Alerts

Enable and view incident alerts

When PagerDuty receives a qualifying event (from a monitoring tool, for example), it triggers an alert, which in turn triggers an incident. Multiple alerts can be aggregated into a single incident for triage, which streamlines incident handoff between teams, centralizes critical information, and reduces notification fatigue. Alerts can move from one incident to another, either manually or via an automated process, such as Alert Grouping.

With a small handful of exceptions, alerts work on any service that integrates with a third-party monitoring tool.

Create Alerts

By default, all new PagerDuty services are configured to create alerts. Alerts are created for inbound events submitted via the Events API, or via the majority of integrations that use an Events API-based integration. You cannot manually create an alert in PagerDuty.

In the most basic terms, events create alerts, and alerts create incidents. The following diagram details this flow:

  1. Monitoring tools send events to PagerDuty
  2. PagerDuty triggers an alert
  3. The alert is associated with with an incident
  4. Incident creation sends out notifications
  5. Users receive notifications
863

Diagram detailing alerts' role in incident creation

📘

Incidents That do not Create Alerts

Incidents created via any of the following processes do not generate alerts. With that in mind, event rules, suppression and other alert-related features are not applicable:

Alerts and Incident Titles

When PagerDuty creates an alert, we create an accompanying incident and give it the same title as the triggering alert. As time passes, it is possible that additional alerts will be added to the incident, either manually or via Alert Grouping. When this happens, the incident’s title will not change — it keeps the title from the original alert. That said, you can manually edit an incident’s title at any time. Alert titles cannot be edited.

View Alerts

You can review alerts associated with an incident in the web app or the mobile app. For an overview of all alerts, please refer to the Alerts Table article.

View Alerts in the Web App

  1. Go to Incidents and select an incident’s Title to go to its detail page.
  2. Select the Alerts tab.
    • Here you can review information about all of the alerts grouped under the incident, as well as Show/Hide Details about the alert.
1452
  1. Select an alert’s Summary for the most detailed view.
    • The Alert Log shows information about when the alert triggered, how it was processed, and any deduplicated alerts.

View Alerts in the Mobile App

  1. On the Incidents screen, select an incident.
  2. On the incident detail screen, scroll down to the Alerts section and select an alert.

Deduplicated Alerts

While an incident is unresolved, any subsequent alerts with a matching dedup_key are deduplicated into the original alert.

Here is an example of what a deduplicated alert might look like in the Alert Log:

844

Deduplicated alert

For more information, please read our developer documentation Alert De-duplication.

Resolve Alerts

Alerts, in contrast to incidents, have two only states: triggered and resolved. You can resolve alerts manually or via the API. If all alerts under an incident are resolved, the incident will likewise become resolved. Conversely, if an incident is resolved then all alerts under the parent incident will be resolved.

Web App

To manually resolve an alert in the web app:

  1. Go to Incidents and select the Title of an unresolved incident.
  2. On the incident detail page with the Alerts tab selected, check the box next to the alert you’d like to resolve.
    • To mass select alerts, select the checkbox at the top in the header row.
  3. Click Resolve.

Mobile App

To manually resolve an alert in the mobile app:

  1. On the Incidents screen, select an incident.
  2. On the incident detail screen, scroll down to the Alerts section and select an alert.
  3. Tap Resolve.

Enable and Disable Alerts on a Service

📘

Accounts Opened Before November 2016

While all accounts created after November 2016 have alerts and incidents enabled by default, it is possible that alerts may not be enabled on individual services, for example if alerts were manually disabled or if the service was created before November 2016 when the alerts featured was released.

By default, PagerDuty services are configured to create alerts. You can administer this setting in the web app:

  1. Go to Services Service Directory click the name of your desired service Integrations tab.
  2. Under Alert and Incident Settings, click Edit.
  3. Select one of the following options:
    • ​​Create both alerts and incidents (Recommended)
    • Create incidents only: If the service is using any features that rely on alerts, you’ll see a message detailing which changes you’ll need to make before selecting this option.
  4. Click Save Changes.
1183

Alert and incident settings

If your account does not have alerts automatically enabled, you will need to change this setting on each service where you’d like this functionality. In other words, enabling this feature on one service will not enable it on all services.

With alerts, the field incident_key is deprecated and becomes null. Deduplication will be based on the alert_key instead of the incident_key. This may cause issues with some bi-directional integrations. Additionally, the summary of the number of alerts is found using the alert_count. You can read more on this in our API Documentation.

Bi-Directional Integration Limitations

With PagerDuty’s alerts and incidents functionality, most services can be migrated to create alerts and incidents. However, bidirectional integrations that allow PagerDuty to send information back out to a tool cannot support alerts and incidents.

Bidirectional integrations help keep outside monitoring tools in sync with PagerDuty acknowledge or resolve actions by sending a webhook when an incident’s state is updated. These third-party tools depend on incident_key being present in the webhook payload to update events on their end. With alerts enabled on a service, webhook payloads do not contain incident_key; instead the field will show as null.

Bidirectional integrations with this limitation include the following:

  • CA UIM
  • CA UIM – Email
  • Desk.com
  • Freshservice (legacy)
  • Glip
  • JIRA Software
  • Kayako
  • Microsoft Teams
  • Nagios and forks of Nagios: Check_MK, Icinga, Nagios XI, Groundwork Nagios, Opsview
  • Riemann
  • ScienceLogic
  • ServiceNow v3.2.1 (v3.5 and beyond are unaffected)
  • Slack to PagerDuty
  • Stackdriver
  • Zendesk

If you are using any of the above bidirectional integration on a service, you will not be able to select the option Create both alerts and incidentsSave Changes will be grayed out. Similarly, if you have more than one integration on a service, and any of the integrations do not support alerts, the option to Save Changes will be grayed out.