CanaryAlertEarly Access
OverviewGetting StartedEmail ForwardingHTTP PingClassification RulesIntegrationsMaintenance WindowsSource GroupsReportsDigest Notifications

Classification Rules

Classification rules tell the AI how to interpret each notification email. Write them in plain English — no regex, no code.

How classification works

When an email arrives, CanaryAlert sends its subject, sender, and body to an AI model (Claude by Anthropic) along with your rules. The AI reads the email and returns:

  • SUCCESS— The email matches your success rules
  • FAILURE— The email matches your failure rules
  • WARNING— The email matches your warning rules
  • UNKNOWN— The email doesn't clearly match any rules

Each classification includes a confidence score (0-100%) and a reasoning explanation so you can see why the AI made its decision.

Writing effective rules

Be specific about what to look for

Bad

"The backup worked"

Good

"Email subject contains 'Success' and body shows all VMs completed with 0 errors"

Reference specific words and patterns

Mention exact words or phrases that appear in your emails. The AI is looking for these in the email content.

Bad

"Something went wrong"

Good

"Email contains 'Error', 'Failed', 'Exception', or a non-zero exit code"

Cover edge cases in warning rules

Warning rules help catch partial successes and degraded performance before they become failures.

Example

"Job completed but with retry attempts, skipped items, partial completion, or duration exceeding 2 hours"

Examples by software

Veeam Backup & Replication

Success

"Email subject contains "Success" and the body shows all VMs completed with 0 warnings and 0 errors"

Failure

"Email subject contains "Failed" or "Error", or any VM shows a non-zero error code"

Warning

"Email subject contains "Warning" or "Success" but the body mentions skipped items, partial completion, or retry attempts"

SQL Server Agent

Success

"Job completed successfully with "The job succeeded" message and 0 retries"

Failure

"Email contains "The job failed" or mentions an exception, timeout, or step failure"

Warning

"Job succeeded but with retry attempts or longer-than-expected duration"

Linux Cron / Shell Scripts

Success

"Exit code is 0, no lines containing ERROR or FATAL, and the expected output pattern is present"

Failure

"Exit code is non-zero, or output contains ERROR, FATAL, FAILED, or stack trace"

Warning

"Exit code is 0 but output contains WARNING, WARN, or unexpected duration"

Windows Task Scheduler

Success

"Task result code is 0 (0x0) and the message says "completed successfully""

Failure

"Task result code is non-zero or the message mentions failure, timeout, or access denied"

Warning

"Task completed but with unusual result codes or partial output"

AI-powered rule suggestions

Not sure what rules to write? Let the AI help. When adding a new source, click the "Suggest from email" button and paste a sample notification email. CanaryAlert's AI will analyze the email format and generate tailored success, failure, and warning rules for you.

How to use it

  1. 1. In the Add Source wizard, go to the Classification Rules step
  2. 2. Click "Suggest from email"
  3. 3. Paste the full text of a notification email from the system you're monitoring
  4. 4. The AI detects the software (Veeam, SQL Agent, cron, etc.) and generates rules tuned for that format
  5. 5. Review and edit the suggested rules before saving

Suggestions are a starting point — always review them and adjust for your specific setup. You can also use them on existing sources by editing the source configuration.

Testing your rules

Use the "Test Classification" button on any source detail page to test your rules before going live:

  1. 1. Open a source and click "Test"
  2. 2. Paste the raw text of a notification email
  3. 3. Review the classification result, confidence score, and reasoning
  4. 4. Adjust your rules if the result isn't what you expected

Next guide

Integrations