Event Orchestration is a highly customizable suite of features and capabilities. The examples on this page are taken from real-world customer use cases, which you're welcome to replicate in your own PagerDuty account.

We have included the following examples to assist you with your own orchestrations:

• If Events Match Certain Conditions
• On a Recurring Weekly Schedule
• Advanced Configuration with the and Operator
• Capture Unrouted Trigger Events
• Standardized Triage
• Notification Management for Specific Incidents
• Route Alerts with Dynamic Service Routes
• Automated Remediation for Specific Incidents

If Events Match Certain Conditions

In this example, we will create a routing rule that routes events to a service when matching conditions are met.

1. Create a routing rule and, under When should events be routed here?, select If events match certain conditions.
2. Indicate the event conditions that you would like the orchestration to match using one of the following methods:

3. In the middle dropdown, select how the event should be filtered.

📘

Negative Operations

Rules with negative operations, such as does not contain or does not equal, will match events that do not contain your specified value and events that do not contain the field at all. As an example:

• severity field does not equal critical
  • This will match events where the severity field does not equal critical and events that do not contain a severity field at all.

If you'd like to avoid this, you must add an additional condition that matches only when the field exists. For example:

When all conditions are true:

• severity field exists
• severity field does not equal critical

Note: You must select all conditions must be true for the rule to match.

4. In the second value field, input the value that should be met from the payload. This can be a string or regular expression.

📘

Case Sensitivity

Condition values in Event Orchestrations are case-insensitive.

For example, if a condition is set with Summary matches part DOWN, this will match if the Summary contains Down, down and other variations of the word.

5. When additional conditions should be added, use the following options:

• + And: Additional conditions should be met.
• + New Condition: Create another set of conditions that should be met. A new condition block creates an OR operator of conditions that will also be evaluated alongside other blocks.

6. Click Save to save your configuration.

📘

Condition Limits

You can create up to 25 condition blocks within a rule, and you can have up to 64 operators (i.e., AND, OR), or a maximum of 2048 bytes (e.g., if you're using PCL) in a single condition block.

On a Recurring Weekly Schedule

📘

Availability

Scheduled conditions are available with Advanced Event Orchestration.

In this example, we will create a routing rule that routes events to a service on a weekly recurring schedule.

1. Create a routing rule and, under When should events be routed here?, select On a recurring weekly schedule.
2. Enter appropriate times for Start and End, select your preferred Days of the week, and pick your Timezone.
3. Click Save.

Advanced Configuration with the and Operator

📘

Availability

Scheduled and threshold conditions are available with Advanced Event Orchestration.

This example details how to edit PCL to create an advanced condition that the UI does not natively allow for.

If you select one of the following options for When should events be routed here?, the UI does not offer a way to set an and condition:

• On a recurring weekly schedule
• During a scheduled date range
• Depends on event frequency

You can work around this, however, by directly editing the PagerDuty Condition Language (PCL).

Say you'd like to route events to a service when the following conditions are met:

• The event's source matches mainframe.
• It is Wednesday or Sunday between 11:00 a.m. and 12:00 p.m. Eastern.

The following PCL statement will meet these conditions: event.source matches 'mainframe' and now in Wed,Sun 11:00:00 to 12:00:00 America/New_York.

Capture Unrouted Trigger Events

Orchestration service routes include a catch all rule for events that do not match the configured rules. The default behavior creates a suppressed alert from unrouted trigger events, however, you have the ability to route these events to a specific service instead. It is important to note that in order to support event deduplication across all services, the ability to resolve any open alert previously created by the orchestration when a matching dedup_key exists, regardless of what service the open alert belongs to.

In this example, we will create a routing rule that uses these trigger events to create incidents on a catch all service.

1. Create a routing rule and, under What service should events route to?, select the service you’d like to use as a catch all for these events.
2. Under When should events be routed here?, select If events match certain conditions. Configure the following condition:
   • If event.event_action matches part (contains) trigger
3. Click Save.
4. If necessary, reorder the rule so that it is directly above the default catch all rule.

This configuration can raise the visibility for trigger events that do not match the existing routing rules while also preserving the default resolve behavior described above.

Standardized Triage

📘

Availability

Global Orchestrations, incident suppression and variables are available with Advanced Event Orchestration.

Use Global Orchestration rules to standardize the triage process across your entire organization.

1. Create a Global Orchestration. You will send your events to the orchestration’s integration.
2. Create an Event Count Cache Variable. You can use this to count the number of specific events received within a designated timeframe.
3. Create a rule with an event condition to suppress any known false positive events:
   1. Under When should events be routed here?, select If events match certain conditions.
   2. Define a condition that will match to your false positive events. E.g., If event.summary matches part (contains) [No error]. Then click Next.
   3. Under What action(s) should be applied?, select Suppress incident and notifications.
   4. Click Save.
4. Create an Else rule with an event condition to evaluate the current cache variable count.
   1. Under When should events be routed here?, select If events match certain conditions.
   2. In the field selector, choose Cache Variable and add your cache variable’s name. Configure your desired threshold for when an incident should be created. E.g., If cache_var.myEventCount is greater than or equal 2. Then click Next.
   3. Under What action(s) should be applied?, you can choose your desired priority and/or severity level for these incidents.
   4. Click Save.
5. Create a final Else rule to set a lower priority and/or severity for incidents that do not pass the event count threshold set in the previous rule:
   1. Under When should events be routed here?, select Always (for all events) and click Next.
   2. Under What action(s) should be applied?, you can choose your desired priority and/or severity level for these incidents.
   3. Click Save.

With this configuration in place, any event sent to the global orchestration will be evaluated by these rules. If the event is a false positive and matches the condition you set in step 3, it will automatically be suppressed and will not create an incident.

If the event does not meet that condition, it will increment the cache variable count, be routed to a service, and then create an incident on that service. The current event count for the cache variable will determine whether the incident has a high or low priority and/or severity.

Notification Management for Specific Incidents

📘

Availability

Dynamic Escalation Policy Assignment is available with Advanced Event Orchestration.

Leverage Dynamic Escalation Policy Assignment to notify a Subject Matter Expert (SME) instead of the current on-call user when an incident requires a special skill set to resolve.

1. Create an escalation policy with the SME user in the first level.
   1. If the SME should be on-call 24/7, you can add them as a user target. Otherwise, you can create a schedule to define what hours the user should be on-call for.
2. In your global orchestration, create the following rule:
   1. Under When should events be routed here?, select If events match certain conditions.
   2. Define a condition that will match to events that indicate a specific SME should be notified instead of the current on-call user. E.g., If event.summary matches part (contains) Database error. Then click Next.
3. Under What action(s) should be applied?, locate Override service's assigned escalation policy with this policy and select your desired escalation policy from the dropdown.
4. Click Save.

When the orchestration receives an event that matches the condition, the escalation policy chosen will be used for the incident, regardless of which service the event is routed to.

Route Alerts with Dynamic Service Routes

📘

Availability

Dynamic routing, dynamic field enrichment and extraction and variables are available with Advanced Event Orchestration.

Utilize the powers of Global Orchestrations to standardize the event format in order to automatically route events to the correct service with a dynamic routing rule.

An orchestration’s dynamic routing rule evaluates all events and does not have conditions. Rather, dynamic routing rules are able to route events to services based on the Service name or Service ID included within the payload. If service details are not included in your initial payload, you can update the payload with dynamic field enrichment and extraction.

1. In your global orchestration, create a rule with the following actions:
   1. Create a rule variable and use regex to extract the desired value from an event field.
   2. Create an event field (e.g., custom_details.service_route) and use the rule variable you created to set the value.
2. In your global orchestration, navigate to the Service Routes tab.
3. Create a new dynamic route and enter the event field you created in step 1.

When the orchestration receives an event with a valid service name in the event field, it will automatically route it to that service. The event can be processed further by the service’s orchestration rules if desired.

Automated Remediation for Specific Incidents

📘

Release Status and Availability

Automation on Alerts is in Early Access for customers with PagerDuty AIOps. Functionality and documentation are subject to change.

To request access, please complete our Early Access form and select Automation on Alerts.

You can pause notifications and suspend alerts for a set duration and simultaneously trigger a remediation automation with an Automation Action. If the automation successfully resolves the issue, an incident will not trigger. If the pause duration expires without a resolution, the incident will trigger as normal. You can configure this within a service orchestration by following the steps below.

1. In your service orchestration, create a rule with your desired conditions.
2. Under Basic Event, enable Pause notifications. Enter a value (in seconds) for Suspend alert for ___ seconds before triggering an incident. We recommend setting a pause duration that is long enough for your entire automation to complete.
3. Under Automation Automation Actions, enable Use Automation Actions if an event reaches this rule.
4. For When should we run this Action?, select Automatically when Alert is suspended/paused.
5. For Automated Action, select your configured Automation Action .