Event Management
Methods to reduce unwanted alert noise
Event management reduces notification fatigue and helps your team focus on tackling the right issues at the right time. The following methods are effective ways to prevent notification fatigue:
- Deduplicate Alerts: Services with Events API integrations deduplicate alerts with matching
dedup_keyvalues. - Event Orchestration: Automatically deduplicate or suppress alerts.
- Dynamic Notifications: Notify responders based on an alert's severity.
- Email Management Settings: Configure email integrations to only trigger alerts when the conditions you specify are met.
Deduplicate Alerts
For services with an Events API v2 integration, you can group related alerts into the same incident by including a value for dedup_key, a case-sensitive string that identifies an alert, in the trigger event's body. Subsequent alerts with a matching dedup_key value will be deduplicated into the same incident, and the new event will be appended to the incident's Alerts log as an additional Trigger log entry.
If there are no open incidents with this key, PagerDuty will create a new incident. Additionally, if two events with the same dedup_key are sent to two different integrations on the same service, they will not be deduplicated and separate incidents will trigger.
If the event key field is blank or not present, PagerDuty will automatically open a new incident with a unique key.
To learn more about how to deduplicate incidents, please visit our Developer Documentation.
Alerts table
Events API v1
The v1 version of the Events API also supports alert deduplication, but it uses a field called
incident_key. Please read our developer documentation Events API v1 for more information.
Event Orchestration
Event Orchestration allows users to define automated actions, based on conditions that apply to the information in an inbound event's payload. Event Orchestration can perform actions such as deduplicating or automatically suppressing alerts. Please read Event Orchestration for more information.
Alert Suppression
Availability
Alert Suppression is available on accounts with AIOps, or Enterprise for Incident Management and Digital Operations (Legacy) pricing plans. Please contact our Sales team to upgrade to a plan with this feature.
Alert suppression, as opposed to setting an alert's severity, allows you to send events to PagerDuty without triggering any notifications. Suppressed alerts are stored in PagerDuty and are available for forensics, analysis, and context, but do not create incidents. You can view suppressed alerts in the Alerts Table.
To suppress an alert when it matches a given set of conditions, select Suppress as the action to take when you are configuring an Event Orchestration.
View Suppressed Alerts
Suppressed alerts can be viewed in the Alerts Table under Incidents Alerts.
Suppressed alerts are filtered out of the incidents dashboard by default, including the incidents page for the service where they were triggered. Moreover, because suppressed events do not trigger incidents, they will not be visible in the mobile app.
Below is an example of a suppressed alert. It looks very similar to other alerts, but has Triggered (Suppressed) in the Current Status field and is not assigned/assignable.
Alert details
Dynamic Notifications
Dynamic notifications allow you to generate alerts with a severity field. When an incident is generated from an alert, the alert’s severity field allows you to configure how responders are notified, if they are notified at all. Please read Dynamic Notifications for more information.
Email Management Settings
If you use an email integration you can change your incident creation settings so that incidents are only triggered when the condition you specify are met.
The following incident creation settings allow for alert deduplication and reduce the number of notifications responders will receive:
- Open a new alert for each new trigger email subject: Alerts are deduplicated based on trigger emails' subject line. For example, if two emails with the same subject are sent to this service's email address, the first opens a new incident, and the second is appended to the incident.
- Open a new alert only if an open incident does not already exist: An email sent to the service's email address will only open a new incident if the service has no open incidents; otherwise, the email will be appended to the open incident.
- Create and resolve alerts based on custom rules: Use regular expressions to parse incident triggers and resolves.
To configure these settings, please read Email Management: Filters and Rules.
FAQ
What is the difference between deduplication and suppression?
- Deduplication: Alerts with the same incident key are grouped together into the same incident and do not generate multiple notifications.
- Suppression: Alerts are filtered by the conditions set in your Event Orchestrations, and those that match your criteria are suppressed and stored in PagerDuty to be available for forensics, analysis, and context. The major difference between suppressed and deduplicated alerts is that suppressed alerts do not create incidents. Suppressed alerts can be viewed in the Alerts table. It should also be noted that suppressed alerts can be deduplicated.
Will suppressed alerts still show up on a service with no one on call?
If you send an event to a service where no one is on call, the event will not create an incident. However, if the event is meant to be suppressed, it will still go to the service if no one is on call.
Updated 9 days ago