Time-Based Alert Grouping will automatically add alerts to an open incident for a predetermined period, which can be helpful for services that generate many alerts. Grouped alerts mean fewer incidents and interruptions for responders, richer context for triggered incidents, and lower resolution times.
Required User Permission
Users with the following roles can edit a service’s Alert Grouping settings:
- Account Owner
- Admin and Global Admin
- Manager base role and team roles
- Manager team roles can only manage services associated with their team.
Please make sure your desired service is configured to create alerts and incidents. If the service is configured to create incidents only, the option to enable Time-Based Alert Grouping will not be available.
To enable Time-Based Alert Grouping:
- In the web app, navigate to Services Service Directory select your preferred service’s name.
- Select the Settings tab and click New Grouping in the section Reduce Noise.
- Select Time only and a duration from the dropdown.
- Click Save Settings.
To select a different grouping method, or to disable Alert Grouping all together, in the web app:
- Navigate to Services Service Directory select the name of your desired service.
- Select the Settings tab and click Edit in the section Reduce Noise.
- In the bottom-left, click Delete.
- In the confirmation modal, click Yes, turn off.
With Time-Based Alert Grouping enabled on a service, the first alert will create a new incident. Subsequent alerts will be added to that incident for the specified time period while the incident is open (i.e., triggered or acknowledged). If the incident is resolved within the selected time period, the timer will stop, and no subsequent alerts will be grouped into that incident.
The timer restarts when the next alert on that service comes in and triggers a new incident. If the incident is not resolved within the selected time period, the incident will remain open and the next alert will trigger a new incident. All subsequent alerts will be grouped into the most recent incident.
If you choose until incident is resolved for the timer’s duration, the first alert on the service will trigger an incident and all subsequent alerts will be grouped into that incident until it is resolved, regardless of how much time passes.
If Time-Based Alert Grouping adds an alert to an incident that would belong better as part of a different incident, there is an option to move the alert to another incident or to create a new incident. For more information, please read Move Alerts to Another Incident.
Manually triggered incidents do not create alerts and do not participate in Alert Grouping.
While the feature is designed to reduce notification noise, the number of times you are notified will be based on your notification rules. If a service has an acknowledgement timeout, for example, responders will receive notifications when the timeout expires.
For this example, we have a service, Shopping Cart, that is using Time-Based Alert Grouping with a 1-hour time window.
|Time||Example 1: Incident resolves within 45 minutes||Example 2: Incident does not resolve within 1 hour|
|10:00 a.m.||TRIGGER: An alert triggers and creates Incident 1. The 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 1 until 11:00 a.m.||TRIGGER: An alert triggers and creates Incident 1. The 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 1 until 11:00 a.m.|
|10:07 a.m.||ACK: Responder acknowledges Incident 1.||ACK: Responders acknowledges Incident 1.|
|10:15 a.m.||Five more alerts come in on Shopping Cart. Incident 1 now has 6 alerts, all triggered.||Five more alerts come in on Shopping Cart. Incident 1 now has 6 alerts, all triggered.|
|10:45 a.m.||RESOLVE: The responder resolves Incident 1. New alerts on the Shopping Cart service will no longer be grouped into incident 1.||Incident 1 is still in the acknowledged state, grouping alerts.|
|10:55 a.m.||TRIGGER: An alert comes in on Shopping Cart service, creates Incident 2. A new 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 2 until 11:55 a.m.||An alert comes in on Shopping Cart, is grouped into Incident 1. Incident 1 now has 7 alerts: 3 resolved, 4 triggered.|
|11:00 a.m.||Incident 1 is resolved, Incident 2 is triggered with 1 alert.||Incident 1 is still acknowledged, the timer expires and PagerDuty will no longer actively group new alerts into Incident 1. Incident 1 is acknowledged with 7 alerts.|
|11:10 a.m.||An alert comes in, is grouped into Incident 2. Incident 2 now has 2 alerts.||TRIGGER: An alert comes in, creates Incident 2. The The 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 2 until 12:10 a.m.|
Deduplication is configured on a per-integration basis, and often requires specific event information, like an incident key or an alert key. When you have multiple integrations on a service, or your monitoring tool generates a variety of alerts, it can become increasingly difficult to manage. Time-Based Alert Grouping allows you to group all incoming events on the service into the current incident for the specified time period, regardless of the monitoring tool (i.e., integration) that generates the original alert.
Snoozing keeps an incident from moving up the chain of an escalation policy and notifying users. Snoozing does not prevent new incidents from being generated on a service.
No, like manually created incidents, the Incidents REST API bypasses alerts and directly creates incidents. With this in mind, it is not affected by Time-Based Alert Grouping.
This also applies to any integrations that create incidents via REST API, for example ServiceNow Enterprise integration.
Events sent via the Events API v2 will still work with Time-Based Alert Grouping.
Yes, incidents are limited to 1,000 alerts. After this limit is reached, a new incident will be created and subsequent alerts will continue grouping into the most recent incident.
Updated 2 days ago