Event Management

Reduce notification noise by using alert deduplication, alert suppression, adjusting email management settings and/or using event rules

Reduce the Number of Notifications for the Same Incident/Event

One can prevent notification fatigue with the following actions:

Include an Incident Key in your API Call

If your service is set up as a service type and multiple incidents are triggering for the same issue, your team will be notified for each duplicate incident. To group these incidents you will want to include incident_key in your parameters for triggering incidents.

PagerDuty de-duplicates incidents based on the incident_key parameter — this identifies the incident to which a trigger event should be applied. If there are no open (unresolved) incidents with this key, a new incident will be created.

Depending on which version of the Events API you're using, you may also see this parameter referred to as dedup_key. Please read more about this in our dev docs.

With Alerts enabled on a service, this value is referred to as Alert Key in the web UI.

If there is already an open incident with a matching key, this event will be appended to that incident's alert log as an additional Trigger log entry.

If the event key field isn't provided, PagerDuty will automatically open a new incident with a unique key.

Adjust your Email Management Settings

If you have an email integration you can change your incident creation settings so that incidents are only triggered under certain conditions.

There are four types of email management settings:

  • Open a new alert/incident for each trigger email: Each email sent to the service's email address will open a new incident.
  • Open a new alert/incident for each new trigger email subject: Incidents are de-duped based on the subject of the trigger emails. For example, if two emails with the same subject are sent to this service's email address, the first opens a new incident, and the second is appended to this incident.
  • Open a new alert/incident only if an open incident does not already exist: An email sent to the service's email address will only open a new incident if they service has no open incidents; otherwise, the email will be appended to the open incident.
  • Create and resolve alerts/incidents based on custom rules: Use regular expressions to parse incident triggers and resolves.

The last three incident creation settings are ones that allow alert/incident de-duplication and reduce the number of notifications being sent.

To change the incident creation settings for your email integration service:

  1. Go to Configuration → Services
  2. Click on the name of the service that houses the integration, then select the Integrations tab. You can edit the integration by selecting Edit from the settings cog, or from the integration details page accessible by clicking the name of the integration.
  1. Select the appropriate email management setting from the four options provided.
  2. Click Save changes.

For more in-depth information about email management settings, please visit our article about email filters and email management rules.

Email Size Limits, Truncation, and Attachments

Email Truncation

If an email that triggered an incident is not displayed in its entirety in PagerDuty it may have been truncated due to its size.

If the email (including headers, attachments, etc.) is under 96 KB the message can pass through without truncation. If the message is over 96 KB the following can happen:

  1. Any parts of the email body where the Content-Type are not matched by this regular expression will be discarded:
    This means PDF files and similar attachments will be stripped.
  2. Text parts (Content-Type of text/plain or text/html) will be truncated to 32 KB.
  3. If the resulting total email size (including headers, attachments, etc.) still exceeds 192 KB then we will reject the message.

You can tell if a message has been truncated or had attachments removed by checking for the headers X-PagerDuty-Truncated-Part or X-PagerDuty-Removed-Attachments when viewing the raw message. You can view the raw message on the incident's individual page by clicking View Message on an incident log entry, then View Raw Message.

Email Size Limits

If an email is over 10 MB, our server will reject it.

Incident Key Truncation

If the incident key of the email that triggered an incident is more than 255 characters it will be truncated to 255 characters.

Truncation of Incident Details in Email Notifications

When you receive a PagerDuty email notification, the "Details" section of each incident will be truncated to 500 characters. That said, the entire email body is accessible in the PagerDuty web UI via the incident logs and details, or via the mobile app.

Suppression and Event Rules

Event rules define automated actions to take on alerts created by services, based on conditions that apply to information in the inbound events' payloads. Event rules can perform actions such as setting the severity of the resulting alert, or automatically suppressing the alert altogether.

Event rules can be used on services that use vendor-specific and API-based integrations. For email integrations, we recommend utilizing email management rules.

Service Event Rules

To use event rules on a service, that service must first be set to create Alerts and Incidents for inbound events. To do this, go to Configuration → Services, edit the service and locate the Incident Behavior section at the bottom of the Edit Service page.

Event rules then can be added from the individual service page; the Event Rules tab is located on the far right.

Event Rule Behavior

Rules are evaluated in the order they appear in the Event Rules configuration tab, top to bottom. The first matching rule is applied, and then execution stops. If no rule matches, the event is processed ordinarily, as if there were no rules configured.

The order of rules can be controlled by dragging and dropping the rules in the Event Rules page.

If an event rule is deleted while a suppressed alert is in a "triggered" state, further alerts with the same alert key will continue to deduplicate until the alert is resolved.

If a second event with the same incident key is passed and de-duplicated into the open incident via global or service event rules, the actions are not followed (ie. raising priority, adding a note) except for when severity is raised.

Setting Conditions for Event Rules

When creating a rule, first set the conditions under which the rule will apply. For example:

In the above example, the rule will apply to an inbound event if the value of its «Source» field is exactly prod05-nginx. This particular rule will cause the event to be suppressed, and the severity will be set to Error.

Event rule conditions can be configured for any of the following PagerDuty Common Event Format (PD-CEF) fields:

  • Class
  • Component
  • Group
  • Severity
  • Source
  • Summary
  • Custom Details

The V1 Events API can also be used with Event Rules.

  • Custom Details can be used as the details parameter of the V1 Events API and the custom_detail parameter of V2 Events API
  • Summary can be used as the description parameter of the V1 Events API and the summary parameter of V2 Events API

Accessing Custom Details

In order to access nested components of the custom details property for a comparison in a condition, enter the full namespace path, in dot notation, from the root of the details property, into the Key name in details field. For example, if the details property is {"foo": {"bar1": {"baz": "thevaluehere"}, "bar2": 2}}, the value in the comparison for foo.bar1.baz would be thevaluehere.

Once you have selected a field to examine, select a condition, such as equals or contains, and (if applicable) enter a value to compare the field against.

In addition to basic comparison, regular expressions can be used. Note, PagerDuty uses the RE2 flavor of RegEx to evaluate expressions.

Equals vs contains in event rule conditions

If you use equals in your event rule, this field must be passed in as a string and match it exactly in order to satisfy the rule.

If the event is added to an incident, notes can be seen on that incident's details page. Notes added by event rules will list the first account user as the author.


Suppression, as opposed to setting the severity of alerts, allows you to send events to PagerDuty without triggering any notifications. Suppressed alerts are stored in PagerDuty and available for forensics, analysis, and context, but do not create incidents. Suppressed alerts can be viewed in the alerts list as well as the Infrastructure Health Application.

To suppress an alert if it matches a given set of conditions, select «Suppress» as the action to take when you are building your Event Rules.

Viewing Suppressed Alerts

Suppressed alerts are filtered out of the incidents dashboard by default, including the incidents page for the service on which the suppressed alerts were triggered. Moreover, because suppressed events do not trigger incidents, they will not be visible in the mobile app.

They can be viewed in the Alerts tab at the top of your screen, between Incidents and Configuration.

Here is an example of a suppressed event. It looks very similar to other alerts, but has "Suppressed" in the severity field and is not assigned/assignable.

The incident below was triggered on a service with alerts set up to trigger as suppressed by default.

Infrastructure Health Application

Suppressed alerts may also be viewed in the Infrastructure Health Application. Suppressed alerts will be shown in gray to differentiate them from the rest of your alerts and incidents.

Event Management

Reduce notification noise by using alert deduplication, alert suppression, adjusting email management settings and/or using event rules

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.