Alert rules let you create complex conditions for an integration, by matching fields like alert_type, summary, description of the alert, and payload sent by the alert.Alert Rules also lets you take custom actions like assign a custom EP or a user to an integrations. Custom actions include:
- Change alert type
- Change incident urgency
- Add note to incident
- Assign incident tag
- Assign incident role
- Change entity_id
- Change incident title
- Change incident summary
- Assign incident to a user
- Route incident to an escalation policy other than the default policy associate with the service
- Suppress the incident
The basics of Alerts and Incidents in Zenduty
Every alert coming from an integration source has an entity_id. An entity_id points to a resource or an entity in the source application. For example, the entity_id of an alert from Jira will be its ticket ID. An alert from Datadog will be its alert ID in Datadog. An alert from Grafana will be its Rule ID. Multiple alerts can have the same entity_id. entity_id is how Zenduty groups multiple alerts about the same problem into one single incident.
Each alert comes with an alert_type. There are six types of alert types: critical, acknowledged, resolved, info, warning, error. If the alert source is sending an entity_id, and the alert type is either critical, error or warning, then that alert(depending on your integration level settings) will trigger a single incident. Every alert after that with the same entity_id will be attached to the earlier created incident. An 'acknowledged' alert_type from the source will automatically acknowledge an incident and a 'resolved' alert_type will automatically resolve the incident.
For example:
Alert 1: entity_id:database-alpha, alert_type:critical, message:DB is down
The above alert will create a new incident(Let's call it 'I'). Incident 'I' is now in the triggered state
Alert 2: entity_id:database-alpha, alert_type:critical, message:DB is still down
The above alert will not create a new incident, but will be appended as an alert to incident 'I'. Incident 'I' is will continue to be in the triggered state
Alert 3: entity_id:database-alpha, alert_type:resolved, message:DB is up
The above alert will automatically resolve the incident 'I' and will also be appended as an alert to incident 'I'
Alert 4: entity_id:database-alpha, alert_type:critical, message:DB is down
The above alert will create another new incident(Let's call it 'I2'). Incident 'I2' is now in the triggered state.
And so on...
Adding an Alert Rule
Go to the service your integration is in. Click on “manage”.
Go to the “integrations” tab and choose the integration for which you want to add Alert Rules. Click on “configure”.
- Click on the “Alert Rules” tab. Click on the “Create Alert Rule” button.
- Give your alert a name and set conditions and actions. Save to apply.
- All saved Alert Rules associated with a particular integration will show up in the Alert Rules tab in the configuration settings of the integration.
JSON Path Search
You can assign payload with specific rules to your alerts. Follow the JSON path syntax here while describing these rules. For example, if the payload sent by the alert is {“project”:”zd-test”, “release”:{“version”:”2.3.1”}}, then to compare the release version, your JSONPath will be $.release.version.
Time window Search
Note: If you’re adding a condition to check if the timing of the alert is between two times say A and B, please ensure that the B>A. If not, then split the alert rule into to conditions with an OR operator between them like this:
A <= T <= "23:59:59" OR "00:00:00" >= T >= B
For example, if you want to create an alert rule where the alert time is between 7pm and 5am, then you need to write two conditions with an OR operator:
- Alert time between 7:00:00 pm and 11:59:59pm
- Alert time between 12:00:00 am and 05:00:00am
If you want to create an alert rule where the time is between 9am and 5pm, then you can do with a single condition:
- Alert time between 09:00:00 am and 05:00:00pm
Modifying incident title and summary, adding notes
You can modify the incident titles and summaries by using the alert data(message, summary, payload, entity_id) within placeholders like The incident message is , or The incident summary is or High priority incident in release .
[IMPORTANT] The Add Note action is only applied for the first alert that triggered the incident. For any alert that does not directly trigger an incident, the Add Note action will not be applied.
Editing an Alert Rule
Simply click the “view” button next to the rule you want to edit.
Deleting an Alert Rule
Simply click the “delete” button corresponding to the rule you want to delete.
##FAQs:
- For time window based operations that cross into the next day, for example 18:00:00 to 04:00:00, split the time in the conditions as 18:00 to 23:59:59 and 00:00:00 to 04:00:00