Here are some solutions to issues that may come up when working with email management rules:

• Replies and Email Forwards Trigger a New Incident
• Trigger Emails are not Accepted
• Resolve Emails are not Resolving Incidents
• Capture Group Error Message
• A Regular Expression is Not Matching the Email Body as Expected

Replies and Email Forwards Trigger a New Incident

By default, PagerDuty email integrations accept any messages from any sender, including messages addressed to multiple recipients. When someone replies to all recipients on an email with recipients that include the email integration address, they also send an email to the email integration address. This can trigger a new incident, in addition to the original incident.

You can configure an email management trigger rule to prevent replies and forwards from triggering duplicate incidents:

1. Configure the rule to trigger an alert/incident if Any of the following conditions apply.
2. Configure a condition: The email subject matches the regular expression: .*
3. For the incident/alert key, have the rule state: In the email subject, match this regular expression: (?:(?i)re: |fwd: )?(.*).
4. Click Save changes.

📘

Case Insensitive

This regular expression is case insensitive, so any version of "RE:" or "FWD:" will deduplicate the email trigger into the open incident.

Replies and forwards will now be appended to the existing incident’s Alert Log .

Trigger Emails are not Accepted

👍

Check for Deduplication

If you are able to trigger some incidents but not others, check to see if the alerts created by your emails are being deduplicated into an existing alert within the Alert Log.

You can view the Alert Log by navigating to an open incident on your service, scrolling down to the Alerts tab and clicking the Summary for the incident’s alert. Deduplicated alerts will appear as additional Triggered entries in the Alert Log.

By default, PagerDuty email integrations accept any messages from any sender. If you are using email filters, check to see if your configuration is discarding desired emails. PagerDuty evaluates email filters before management rules in order to reduce noise and ensure all emails come from a verified source.

If you think the filters might be discarding desired emails, try setting them to Accept all incoming email and test again. Keep in mind that changes to your email filters will not affect events that were received before the changes were made.

If you are using email filters, double-check your regular expressions to ensure that desired emails are accepted.

If emails are still not triggering incidents as you’d expect, check the integration’s email management rules. If you are using the Open and resolve alerts based on custom rules setting, check the default behavior; Create a generic alert should be selected. This is the default setting, but if another user changed it, for example, any unmatched emails will be discarded.

Resolve Emails are not Resolving Incidents

Three things are required to successfully resolve an incident:

1. A rule that triggers an incident in PagerDuty.
2. A rule that resolves an incident in PagerDuty.
3. A shared alert key that can be extracted from both emails.

Verify that your rules are correct. PagerDuty will display the extracted alert key on an incident’s alert. Use this piece of information to verify that the key is the same in both the trigger and resolve emails.

Specific reasons a resolve email might not be successfully resolving:

• There is an issue with a regular expression extracting the incident/alert key.
• There is an issue with the email body’s encoding (for example, if it was UTF-8 encoded, but contained the byte \xFF).
• There is a trailing space, for example, in either your trigger rule or resolve rule, but not in both. This means that one email has an alert key with an additional space, and therefore will not match.

Capture Group Error Message

If you receive the following error, it means that you need to specify a capture group in your regular expression:

Email handler email action rules value extractors matcher must have at least one capture group

Oftentimes, this can be resolved by enclosing your regular expression in parentheses.

A Regular Expression is Not Matching the Email Body as Expected

If a regular expression that should match the email body is not working for some reason, check to see whether emails are sent in both plain text and HTML, or plain text. If the emails are only sent in HTML, the HTML tags may be interfering with the regex’s pattern matching.

If the email also includes a plain text version, then the rule will still be able to find a match. However, if the emails are being sent as HTML-only, note that the regular expression may not match because there are HTML tags and elements interspersed within the text content that makes up the searched-for pattern. It may help to view the raw email that was checked for a match in the email rules.

📘

View the Email Source Code

Go to View Message in the incident detail view. This will take you to a page displaying the full details of the original event sent in. The URL should be formatted as follows:

https://{subdomain}.pagerduty.com/incidents/{incident_id}/log_entries/{log_entry_id}

Next, click View Raw Message to see the email’s source code, including headers.

While viewing the source code, note that email management rules that operate on the body of the email use the raw body as input, after decoding (i.e., if the body’s Content-Transfer-Encoding is base64, it will be decoded from Base64, or if it's quoted-printable, then QP escape sequences will be replaced with the actual characters that represent the message’s underlying content). With this in mind, to view the true source of the email that the regular expressions are compared against, one must decode the text.

Base64-encoded Text

Base64-encoded messages will show the following information in their header:

Content-Transfer-Encoding: base64

Base64-encoded messages are not humanly readable, but are easily decoded into plain text using readily available software. We recommend the base64 command line program, which is found on many Linux-based and Unix-based systems, including macOS.

📘

Example of Base64-encoded text

Decoded (Regex runs on this text):

All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy.
The quick fox jumps over the lazy brown dog.

Encoded (Regex does not run on this text):

QWxsIHdvcmsgYW5kIG5vIHBsYXkgbWFrZXMgSmFjayBhIGR1bGwgYm95LiBBbGwgd29yayBhbmQg
bm8gcGxheSBtYWtlcyBKYWNrIGEgZHVsbCBib3kuClRoZSBxdWljayBmb3gganVtcHMgb3ZlciB0
aGUgbGF6eSBicm93biBkb2cuCg=

Decoding with base64:

base64 -D < file-containing-base64-message.txt
# or this:
cat file-containing-base64-message.txt | base64 -D

Quoted-Printable Text

Quoted-Printable messages will show the following information in their header:

Content-Transfer-Encoding: quoted-printable

Messages with Quoted-Printable encoding can be easier for humans to read, but a variety of tools are available for decoding them.

When reading raw Quoted-Printable encoded text, please be aware of escape sequences that constitute part of the encoding, or that are a representation of a character and not the literal character. Non-ASCII characters will be represented as escape sequences, as will equals signs (=).

Furthermore, lines are limited to 76 characters. If a line’s length needs to be limited, a line may be encoded as two or more lines with a line break and an equals sign at the end of every line except the last in the series.

📘

Example of Quoted-Printable encoding

Decoded (Regex runs on this text):

All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy.
The quick fox jumps over the lazy brown dog.
E = mc^2

Encoded (Regex does not run on this text):

All work and no play makes Jack a dull boy. All work and no play makes Jack=
 a dull boy.
The quick fox jumps over the lazy brown dog.
E =3D mc^2