Alerts
Argus evaluates alerts on metric data and notifies users when trigger thresholds are exceeded. Alerts are scheduled and executed per the CRON entry specified when the alert was created. An alert is associated with at least one trigger and one notification. You can associate a trigger with more than one notification, and a notification with more than one trigger.
You can access the Alert interface via the link at the top of the Argus interface.
An alert includes the CRON entry, enabled, expression, Id, missing data notification, name, and owner.
| Component | Description |
|---|---|
| CRON entry | Time and frequency of the alert, based on the Quartz CronTrigger format |
| Enabled | Enabled for evaluation |
| Expression | Metric expression to evaluate trigger conditions |
| ID | ID of the alert |
| Missing data notification | Notify owner if metric expression doesn't contain data |
| Name | Alert name |
| Owner | Owner of the alert |
| shared | A boolean variable that indicates if the alert is visible by other users |
- On the Argus home page, click the Alerts tab.
- Click the Add button.
The alert is created with default data under Alert List. - To change the default values, click the alert name.
The Argus scheduler uses CRON Entry (based on the Quarts CronTrigger format) to schedule the alert. The CRON Entry represents the time and frequency of alert evaluation.
Expression is used to retrieve the metric data to evaluate the triggering conditions.
A trigger includes the ID, inertia millis, name, primary threshold, secondary threshold, and type.
| Component | Description |
|---|---|
| ID | ID of the trigger |
| Inertia millis | Number of milliseconds for which the conditions must be met before the trigger is fired. At least 2 qualifying data points must occur within the inertia window to fire the trigger. |
| Name | Trigger name. Variable interpolation is available. |
| Primary threshold | Operand for the comparison |
| Secondary threshold | Operand for the BETWEEN and NOT_BETWEEN operators. BETWEEN is inclusive of both operand values, while NOT_BETWEEN is exclusive of those values. |
| Type | Trigger comparison operator. Choose "no data" to trigger on missing data and be able to use configured notifications (normally missing-data alerts go only to the alert owner). The threshold and inertia fields are not used with the "no data" type. |
A trigger defines a threshold value. The trigger types are less than, greater than, and equal to the time period. When the trigger condition is met, a notification is sent. You can associate a trigger with more than one notification.
- Click Triggers and then click the Add button.
- A Trigger is created by selecting the Triggers tab then clicking the add button.
The trigger is created with default data. - To change the default values,
Inertia Mills is a time period in milliseconds for which the trigger condition is meant to send a notification.
A notification includes the cooldown millis, ID, metrics to annotate, name, and subscription.
| Component | Description |
|---|---|
| Cooldown millis | Number of milliseconds the notification is on cooldown after a notification is sent |
| ID | ID of the notification |
| Metrics to annotate | Metric identifiers |
| Name | Notification name |
| Subscriptions | Subscribers of the alert |
| Type | Notification type |
A notification defines how you are notified when a trigger fires. You can associate a notification with more than one trigger.
- Click the Notifications tab.
- Click the Add button. The notification is created with default data.
- Click the Triggers button to display all triggers.
- Select the triggers to associate with the notification by selecting their’ IDs.
Metrics to Annotate is a comma-separated list of metric expressions. When a notification is sent, an annotation is created. You can retrieve annotations via a REST endpoint or view them on a dashboard.
Cooldown Mills defines the time period in milliseconds when the notification is not sent. No further notifications are sent for the notification until it comes off of cooldown. The cooldown expiration is a read only field.
Argus supports the following notifiers:
- Audit Notifier: This notifier writes the notification to the Argus database. Subscriptions field is left blank.
- Email Notifier: This notifier sends an email to subscribers whose email addresses are mentioned in the Subscriptions field. The subscription field contains a comma-separated list of email addresses.
- Salesforce Chatter Notifier: This notifier sends an alert to a Chatter group or a list of Chatter groups as specified in the subscriptions field. The subscription field contains a comma-separated list of Chatter group ids.
- Callback Notifier: This notifier send the templated notification to HTTP end point!
When an alert is triggered, a notification is delivered (according to the selected notifier) and also logged in the alert History tab. The notification shows the metric expression associated with the alert, along with a link to the triggering event. Even if the original metric expression used relative time, the logged metric expression contains the actual timespan for the triggering event.
Interpolated variables include the following kinds:
-
${scope}— The expression’s scope -
${metric}— The expression’s metric -
${tag.<tag_name>}— Any tags used in the expression(Except for device tags. Currently you can use ${device} instead of ${tag.device}) -
${alert.name}— Name of the alert -
${alert.expression}- The expression associated with the alert -
${alert.cronEntry}- The cron entry associated with the alert -
${alert.enabled}- Boolean value which tells if alert is enabled or not -
${trigger.name}- Name of the trigger -
${trigger.type}- Type of trigger: [Eg: >, <, <>, ><, nodata] -
${trigger.threshold}- Primary Threshold of the trigger -
${trigger.secondaryThreshold}- Secondary Threshold of the trigger (if exists) -
${trigger.inertia}- Inertia of the Trigger -
${triggerValue}- Value at which the trigger was fired -
${triggerTimestamp}- Timestamp at which the trigger was fired -
${notification.name}- Name of the notification -
${notification.cooldownPeriod}- cooldownPeriod of the notification in milliseconds -
${notification.SRActionable}- Boolean to state if SR should be actionable or not -
${notification.severityLevel}- Severity level of the trigger
Argus uses freemarker internally as the templating engine. You can use all the functionality of v2.3.28 with Argus.
For example, if your expression is
-1h:argus.core:alert.evaluation.kpi.*{host=bar, tagA=*}:min
the trigger name can be written as
trigger-${scope}-${metric}-${tag.host}-${tag.tagA}
If the trigger fires on a metric called “alert.evaluation.kpi.foo”, and tagA has the value “baz”, notifications will show the trigger name as
trigger-argus.core-alert.evaluation.kpi.foo-bar-baz
For example if you write something like this in your Custom Text in the Notification tab
Alert Name = ${alert.name?upper_case},
Alert Expression = ${alert.expression},
Alert cronEntry = ${alert.cronEntry},
Alert enabled = ${alert.enabled?then('alert enabled', 'alert not enabled')},
Alert Expression = ${alert.expression},
Trigger Name = ${trigger.name},
Trigger type = ${trigger.type},
Trigger threshold = ${trigger.threshold},
Trigger secondaryThreshold = ${trigger.secondaryThreshold},
Trigger Inertia = ${trigger.inertia},
Trigger Value = ${triggerValue},
Trigger Timestamp = ${triggerTimestamp?datetime?iso('GMT')},
Notification Name = ${notification.name?cap_first},
Notification cooldownPeriod = ${notification.cooldownPeriod},
Notification SRActionable = ${notification.SRActionable?then('SR Actionable','Not SR Actionable')},
Notification severityLevel = ${notification.severityLevel}
it will get evaluated as
Alert Name = ALERT-123-argus.core,
Alert Expression = -1h:argus.core:alert.evaluation.kpi{host=*}:avg,
Alert cronEntry = * * * * *,
Alert enabled = alert enabled,
Alert Expression = -1h:argus.core:alert.evaluation.kpi{host=*}:avg,
Trigger Name = trigger-1234-argus.core,
Trigger type = GREATER_THAN,
Trigger threshold = 60,
Trigger secondaryThreshold = 0,
Trigger Inertia = 10,000,
Trigger Value = 240,000,
Trigger Timestamp = 2018-10-05T21:15:00Z,
Notification Name = New-notification-1531947321887,
Notification cooldownPeriod = 10,
Notification SRActionable = Not SR Actionable,
Notification severityLevel = 2
It supports conditional statements as well. For example, if we write the following in the Custom Text within the Notification tab
<#if trigger.threshold <= 4> Primary Threshold is less than 4 </#if>,
<#if (trigger.secondaryThreshold == 7.1)> Secondary Threshold is 7.1 </#if>,
<#if trigger.inertia == 5 && (trigger.threshold > 5)> Inertia is 5, Primary Threshold more than 5 <#elseif (trigger.threshold > 5)>Primary Threshold more than 5 <#elseif trigger.inertia == 5> Inertia is 5 </#if>,
<#if trigger.name?matches('trigger_name') && triggerValue < 2.0> Trigger name matches and trigger value is < 1 </#if>,
<#if triggerValue?round == 2> Trigger fired rounded value is 2 </#if>,
<#assign dt = triggerTimestamp?datetime> Trigger fired date-time: ${dt?iso('GMT')},
Time before 2.5 hrs of firing: ${dt?iso('GMT-02:30')}
It might get evaluated as
Primary Threshold is less than 4 ,
Secondary Threshold is 7.1 ,
Inertia is 5 ,
Trigger name matches and trigger value is < 1 ,
Trigger fired rounded value is 2 ,
Trigger fired ate-time: 2014-12-11T17:40:00Z,
Time before 2.5 hrs of firing: 2014-12-11T15:10:00-02:30
Thus it could be used to configure your notification text according to your convenience.
