Time-Based Alert Grouping

Automatically group incoming alerts into open incidents based on the time they are generated

Time-Based Alert Grouping will automatically add alerts to an open incident for a predetermined period, which can be helpful for services that generate many alerts. Grouped alerts mean fewer incidents and interruptions for responders, richer context for triggered incidents, and lower resolution times.

📘

Availability

This feature is available with our PagerDuty AIOps add-on, or with Legacy Event Intelligence. If you would like to sign up for a trial of PagerDuty AIOps features, please read PagerDuty AIOps Trials.

🚧

Required User Permission

Users with the following roles can edit a service’s Alert Grouping settings:

  • Account Owner
  • Admin and Global Admin
  • User
  • Manager base role and team roles
    • Manager team roles can only manage services associated with their team.

Enable Time-Based Alert Grouping

📘

Prerequisite

Please make sure your desired service is configured to create alerts and incidents. If the service is configured to create incidents only, the option to enable Time-Based Alert Grouping will not be available.

To enable Time-Based Alert Grouping:

  1. In the web app, navigate to Services Service Directory select your preferred service’s name.
  2. Select the Settings tab and click New Grouping in the section Reduce Noise.
  3. Select Time only and a duration from the dropdown.
Enable Time-Based Alert Grouping

Enable Time-Based Alert Grouping

  1. Click Save Settings.

Disable Time-Based Alert Grouping

To select a different grouping method, or to disable Alert Grouping all together, in the web app:

  1. Navigate to Services Service Directory select the name of your desired service.
  2. Select the Settings tab and click Edit in the section Reduce Noise.
  3. In the bottom-left, click Delete.
  4. In the confirmation modal, click Yes, turn off.

Time-Based Alert Grouping Behavior

With Time-Based Alert Grouping enabled on a service, the first alert will create a new incident. Subsequent alerts will be added to that incident for the specified time period while the incident is open (i.e., triggered or acknowledged). If the incident is resolved within the selected time period, the timer will stop, and no subsequent alerts will be grouped into that incident.

The timer restarts when the next alert on that service comes in and triggers a new incident. If the incident is not resolved within the selected time period, the incident will remain open and the next alert will trigger a new incident. All subsequent alerts will be grouped into the most recent incident.

If you choose until incident is resolved for the timer’s duration, the first alert on the service will trigger an incident and all subsequent alerts will be grouped into that incident until it is resolved, regardless of how much time passes.

Move Alerts to Another Incident

If Time-Based Alert Grouping adds an alert to an incident that would belong better as part of a different incident, there is an option to move the alert to another incident or to create a new incident. For more information, please read Move Alerts to Another Incident.

Manually Triggered Incidents

Manually triggered incidents do not create alerts and do not participate in Alert Grouping.

Incident Notifications

While the feature is designed to reduce notification noise, the number of times you are notified will be based on your notification rules. If a service has an acknowledgement timeout, for example, responders will receive notifications when the timeout expires.

Time-Based Alert Grouping Example

For this example, we have a service, Shopping Cart, that is using Time-Based Alert Grouping with a 1-hour time window.

TimeExample 1: Incident resolves within 45 minutesExample 2: Incident does not resolve within 1 hour
10:00 a.m.TRIGGER: An alert triggers and creates Incident 1. The 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 1 until 11:00 a.m.TRIGGER: An alert triggers and creates Incident 1. The 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 1 until 11:00 a.m.
10:07 a.m.ACK: Responder acknowledges Incident 1.ACK: Responders acknowledges Incident 1.
10:15 a.m.Five more alerts come in on Shopping Cart. Incident 1 now has 6 alerts, all triggered.Five more alerts come in on Shopping Cart. Incident 1 now has 6 alerts, all triggered.
10:45 a.m.RESOLVE: The responder resolves Incident 1. New alerts on the Shopping Cart service will no longer be grouped into incident 1.Incident 1 is still in the acknowledged state, grouping alerts.
10:55 a.m.TRIGGER: An alert comes in on Shopping Cart service, creates Incident 2. A new 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 2 until 11:55 a.m.An alert comes in on Shopping Cart, is grouped into Incident 1. Incident 1 now has 7 alerts: 3 resolved, 4 triggered.
11:00 a.m.Incident 1 is resolved, Incident 2 is triggered with 1 alert.Incident 1 is still acknowledged, the timer expires and PagerDuty will no longer actively group new alerts into Incident 1. Incident 1 is acknowledged with 7 alerts.
11:10 a.m.An alert comes in, is grouped into Incident 2. Incident 2 now has 2 alerts.TRIGGER: An alert comes in, creates Incident 2. The The 1-hour timer begins, set to group all new alerts on the Shopping Cart service into Incident 2 until 12:10 a.m.

FAQ

How is this different from deduplication?

Expand

Deduplication is configured on a per-integration basis, and often requires specific event information, like an incident key or an alert key. When you have multiple integrations on a service, or your monitoring tool generates a variety of alerts, it can become increasingly difficult to manage. Time-Based Alert Grouping allows you to group all incoming events on the service into the current incident for the specified time period, regardless of the monitoring tool (i.e., integration) that generates the original alert.

How is this different from snoozing?

Expand

Snoozing keeps an incident from moving up the chain of an escalation policy and notifying users. Snoozing does not prevent new incidents from being generated on a service.

Can I manually merge incidents?

Expand

Yes, Time-Based Alert Grouping does not affect your ability to manually merge incidents or move alerts from one incident to another. All grouped alerts will be merged into a single incident.

Will incidents created by REST API be grouped by Time-Based Alert Grouping?

Expand

No, like manually created incidents, the Incidents REST API bypasses alerts and directly creates incidents. With this in mind, it is not affected by Time-Based Alert Grouping.

This also applies to any integrations that create incidents via REST API, for example ServiceNow Enterprise integration.

Events sent via the Events API v2 will still work with Time-Based Alert Grouping.

Is there a limit to how many alerts can group into a single incident?

Expand

Yes, incidents are limited to 1,000 alerts. After this limit is reached, a new incident will be created and subsequent alerts will continue grouping into the most recent incident.