Time-Based Alert Grouping
Automatically group incoming alerts into open incidents based on the time they are generated
Time-Based Alert Grouping will automatically add alerts to an open incident for a predetermined period, which can be helpful for services that generate many alerts. Grouped alerts mean fewer incidents and interruptions for responders, richer context for triggered incidents, and lower resolution times.
Availability
This feature is available with our PagerDuty AIOps add-on, or with Legacy Event Intelligence. If you would like to sign up for a trial of PagerDuty AIOps features, please read PagerDuty AIOps Trials.
Required User Permission
Users with the following roles can edit a service’s Alert Grouping settings:
- Account Owner
- Admin and Global Admin
- User
- Manager base role and team roles
- Manager team roles can only manage services associated with their team.
Enable Time-Based Alert Grouping
To enable Time-Based Alert Grouping:
- In the web app, navigate to Services Service Directory select your preferred service’s name.
- Select the Settings tab and click New Grouping in the section Reduce Noise.
- Select Time only and a duration from the dropdown.
- Click Save Settings.
Disable Time-Based Alert Grouping
To select a different grouping method, or to disable Alert Grouping all together, in the web app:
- Navigate to Services Service Directory select the name of your desired service.
- Select the Settings tab and click Edit in the section Reduce Noise.
- In the bottom-left, click Delete.
- In the confirmation modal, click Yes, turn off.
Time-Based Alert Grouping Behavior
With Time-Based Alert Grouping enabled on a service, the first alert will create a new incident. Subsequent alerts will be added to that incident for the specified time period while the incident is open (i.e., triggered or acknowledged). If the incident is resolved within the selected time period, the timer will stop, and no subsequent alerts will be grouped into that incident.
The timer restarts when the next alert on that service comes in and triggers a new incident. If the incident is not resolved within the selected time period, the incident will remain open and the next alert will trigger a new incident. All subsequent alerts will be grouped into the most recent incident.
If you choose until incident is resolved for the timer’s duration, the first alert on the service will trigger an incident and all subsequent alerts will be grouped into that incident until it is resolved, regardless of how much time passes.
Time-Based Alert Grouping Example
For this example, we have a service, Shopping Cart, that is using Time-Based Alert Grouping with a 1-hour time window.
Time | Example 1: Incident resolves within 45 minutes | Example 2: Incident does not resolve within 1 hour |
---|---|---|
10:00 a.m. | TRIGGER: An alert triggers and creates Incident 1. The 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 1 until 11:00 a.m. | TRIGGER: An alert triggers and creates Incident 1. The 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 1 until 11:00 a.m. |
10:07 a.m. | ACK: Responder acknowledges Incident 1. | ACK: Responders acknowledges Incident 1. |
10:15 a.m. | Five more alerts come in on Shopping Cart. Incident 1 now has 6 alerts, all triggered. | Five more alerts come in on Shopping Cart. Incident 1 now has 6 alerts, all triggered. |
10:45 a.m. | RESOLVE: The responder resolves Incident 1. New alerts on the Shopping Cart service will no longer be grouped into incident 1. | Incident 1 is still in the acknowledged state, grouping alerts. |
10:55 a.m. | TRIGGER: An alert comes in on Shopping Cart service, creates Incident 2. A new 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 2 until 11:55 a.m. | An alert comes in on Shopping Cart, is grouped into Incident 1. Incident 1 now has 7 alerts: 3 resolved, 4 triggered. |
11:00 a.m. | Incident 1 is resolved, Incident 2 is triggered with 1 alert. | Incident 1 is still acknowledged, the timer expires and PagerDuty will no longer actively group new alerts into Incident 1. Incident 1 is acknowledged with 7 alerts. |
11:10 a.m. | An alert comes in, is grouped into Incident 2. Incident 2 now has 2 alerts. | TRIGGER: An alert comes in, creates Incident 2. The The 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 2 until 12:10 a.m. |
FAQ
Can I move a grouped alerts to another incident?
If Time-Based Alert Grouping adds an alert to an incident that would belong better as part of a different incident, there is an option to move the alert to another incident or to create a new incident. For more information, please read Move Alerts to Another Incident.
How does Time-Based Alert Grouping affect incident notifications?
While Time-Based Alert Grouping is designed to reduce notification noise, the number of times you are notified will be based on your notification rules. If a service has an acknowledgement timeout, for example, responders will receive notifications when the timeout expires.
How is Time-Based Alert Grouping different from alert deduplication?
Deduplication is configured on a per-integration basis, and often requires specific event information, like an incident key or an alert key. When you have multiple integrations on a service, or your monitoring tool generates a variety of alerts, it can become increasingly difficult to manage. Time-Based Alert Grouping allows you to group all incoming events on the service into the current incident for the specified time period, regardless of the monitoring tool (i.e., integration) that generates the original alert.
How is Time-Based Alert Grouping different from snoozing incidents?
Snoozing keeps an incident from moving up the chain of an escalation policy and notifying users. Snoozing does not prevent new incidents from being generated on a service.
Can I manually merge incidents?
Yes, Time-Based Alert Grouping does not affect your ability to manually merge incidents or move alerts from one incident to another. All grouped alerts will be merged into a single incident.
Can manually triggered incidents be grouped with Alert Grouping?
Manually triggered incidents do not create alerts and do not participate in Alert Grouping.
Will incidents created by REST API be grouped by Time-Based Alert Grouping?
No, like manually created incidents, the Incidents REST API bypasses alerts and directly creates incidents. With this in mind, it is not affected by Time-Based Alert Grouping.
This also applies to any integrations that create incidents via REST API, for example ServiceNow Enterprise integration.
Events sent via the Events API v2 will still work with Time-Based Alert Grouping.
Is there a limit to how many alerts can group into a single incident?
Yes, incidents are limited to 1,000 alerts. After this limit is reached, a new incident will be created and subsequent alerts will continue grouping into the most recent incident.
Updated 4 days ago