Why Incidents Fail to Trigger
Troubleshooting common issues related to incident trigger failure
There are a few reasons why an incident would fail to trigger:
- A Service Was Disabled or in a Maintenance Window
- Event Orchestration is Configured to Suppress Certain Alerts
- Email Integration Filters
- Email Management Rules Were Appending Triggers to Existing Incidents
- No One Was On-call
A Service Was Disabled or in a Maintenance Window
When a service is disabled or in a Maintenance Window, new incidents will not trigger or be created on that service. Because an incident is not created, PagerDuty will not send notifications to the service's on-call responders.
Event Orchestration is Configured to Suppress Certain Alerts
If you have Event Orchestration configured on your account, you may have rules that are set to suppress alerts that meet certain criteria. In those cases, an incident will not be triggered.
Navigate to AIOps Event Orchestration and select the orchestration that contains the action rule that you would like to check. While reviewing which actions are taken, check Incident Data Basic Event to see if the checkbox Suppress incident and notifications is enabled.
Email Integration Filters
If you have regex filters set up on your email integration service, check whether your email filters may be filtering out emails that you want to trigger incidents.
For example, if you have the following regex filter:
Trigger an alert if All of the following conditions apply: The email subject matches the regular expression CRITICAL
Then emails with the subject line "PROBLEM" will not trigger incidents in PagerDuty, because they are filtered out based on your regex rules.
- Regex Examples
- Test your regular expressions with Regex101 (Use the Golang flavor with the
m
ands
flags to see exactly how PagerDuty will interpret your regex).
Email Management Rules Were Appending Triggers to Existing Incidents
Email integration settings have email management rule settings, which are separate from email filters. While email filters determine which emails trigger incidents for your service, incident creation settings determine how emails create new incidents.
If you have either of these two email management rules for your email integration service:
- Open a new alert only if an open incident does not already exist
- Open and resolve alerts based on custom rules
...then an incident was most likely appended to an existing incident, and could explain why a new incident was not created. When a trigger event is appended to an incident it will appear in the incident's timeline. Learn more about incident creation settings.
No One Was On-call
If no one is on-call on a service's escalation policy, PagerDuty does not have a user to assign an incident to, and we will not create a new incident.
For example, if your escalation policy only has the following schedule associated with it, where one user is on-call from 00:00 - 08:00, if a trigger event comes in between 08:01 - 23:59, PagerDuty will not create an incident no one will be on-call during that time.
To address this, check the Escalation Policy associated with your service and make sure that someone will be on-call when you need incidents to trigger.
Trigger an Incident in the Web App
If you try to trigger an incident through the web app while no one is on-call on the escalation policy associated with that service, you will receive the error Incident could not be assigned.
Updated about 1 month ago