Knowledge Base
API DocsIntegrationsPagerDuty UniversityPagerDuty CommonsContact SupportMy Support Tickets

Event Orchestration

Event Orchestration allows users to route events to an endpoint and create nested rules, which define sets of actions to take based on event content. Event Orchestration is PagerDuty’s modern solution to help tackle your existing and future automation goals.

🚧

Required User Permissions

The following user roles can create/edit/delete Event Orchestrations:

  • User
  • Admin
  • Manager base roles and team roles. Manager team roles can create/edit/delete Event Orchestrations associated with their team.
  • Global Admin
  • Account Owner

Configure Event Orchestration

Step 1: Create an Orchestration

There are three ways to create an Orchestration:

Global Orchestrations

When an incoming event stream has more than one service destination, you can use Global Orchestrations to apply common event actions and route events to different services.

📘

Availability

Global Orchestrations are only available on accounts with the PagerDuty AIOps add-on and the Advanced Event Orchestration pricing tier.

To create a new Global Orchestration:

  1. Navigate to AIOps Event Orchestration and select + New Orchestration.
  2. Enter an Orchestration Name and an optional Orchestration Description.
  3. Click Save.
  4. On the next screen, select Integrations to view the Integration Key, Email Address (for email integrations) and HTTP Endpoint for API that you can use to connect your alerting systems with PagerDuty. Please follow your alerting system’s integration guide for configuration instructions.
  5. Select Global Orchestration to navigate to the Global Orchestration Rules graph.
  6. Click New Rule to create a new rule. Continue to Step 2: Create Rules.

Global Integrations

When an incoming event stream has more than one service destination, you can use Global Integrations to route events to different services and apply Service Orchestration rule actions.

📘

Availability

Global Integrations are available on all packages.

  1. Navigate to AIOps Event Orchestration and select + New Orchestration.
  2. Enter an Orchestration Name and an optional Orchestration Description.
  3. Click Save.
  4. On the next screen, select Integrations to view the Integration Key, Email Address (for email integrations) and HTTP Endpoint for API that you can use to connect your alerting systems with PagerDuty. Please follow your alerting system’s integration guide for configuration instructions.
  5. Next, select the Service Routes tab. Continue to Step 2: Create Rules.

Service Orchestrations

When integrations exist on a service, you can use Service Orchestrations to evaluate your incoming events and perform additional actions.

📘

Availability

Service Orchestrations are available on all packages.

If you have a service that uses Service Event Rules, you can switch to Service Orchestrations at any time. Please read the section Switch to Service Orchestrations for more information.

To create a new Service Orchestration:

  1. Navigate to Services Service Directory click your desired service name and select the Settings tab.
  2. Scroll to Event Management and click Service Orchestration Rules.
  3. Click + New Rule and continue to Step 2: Create Rules.

Create a Service Orchestration from a Global Integration or Orchestration

A Service Orchestration can also be created after a Global Integration, or Global Orchestration is made. This method allows users to create the rules that define what happens after events are routed to a service.

To create a new Service Orchestration from a Global Integration or Orchestration:

  1. Navigate to AIOps Event Orchestration select your desired Orchestration and then click Service Orchestrations.
  2. Select a service from the Service Event Rules dropdown.
  3. You will then be taken to the Service Orchestration graph for the service you selected. Continue to Step 2: Create Rules.

Step 2: Create Rules

With Event Orchestration there are three distinct types of rules that a user can create:

  • Global Orchestration Rules (Available with PagerDuty AIOps only): Global Orchestration rules are optional rules that can be created before a routing rule. They determine which actions are applied to events during event processing as well as how events eventually turn into incidents. Actions applied at the Global Orchestration rule level will apply to all events that match the rule’s conditions, regardless of which service they are eventually routed to. Any actions applied can be overridden by Service Orchestration rules which also match against a given event. The last evaluated rule will take precedence where a single enrichment, alert, or incident action is taken multiple times.
  • Routing Rules (Available on all packages): Routing rules define how events are routed to services. They must be created in order for events to be routed from a Global Integration or Orchestration to a service.
  • Service Orchestration Rules (Available on all packages): Service Orchestration rules are optional rules that can be created after a routing rule. They determine which actions are applied to events during processing, and how events eventually turn into incidents. Service Orchestration rules will have their own conditions and will be linked to rules that precede and follow them.

Recommended Preparation

We recommend sending events to your Orchestration integration key in order to inspect the incoming JSON key/value pairs and build rules from there.

To send events to Orchestrations:

  1. Navigate to AIOps​​ Event Orchestration, select your preferred Orchestration and click Integrations.
  2. On the following screen, select the integration key set you want to work with and copy the appropriate Integration Key to use in your upstream event source, e.g., a monitoring tool or CI pipeline.

Note: An Orchestration can have up to 10 integration keys associated with it. Using any of these integration keys will direct events toward the Orchestration rules and routing rules specified for the Orchestration.

Routing Rules

In order for Event Orchestration to route events to the appropriate service, you will need to create routing rules. Some things to keep in mind while working with your routing rules:

  • There can only be one routing rule per service per Event Orchestration. For example, if Orchestration A routes events to Service A, you cannot configure another rule within the same orchestration to route to same service, however you could configure Orchestration B to route events to Service A.
  • If events are sent directly to a service, this will bypass any rules at the global level and the service’s Service Orchestrations will be evaluated.
  • Routing rule conditions can be based on either PD-CEF-transformed event data or raw event payloads.
  • You can also create Dynamic Routing Rules, which evaluate all events and do not have conditions. Dynamic Routing Rules are able to route events to services based on the Service name or Service ID in the payload. Dynamic Routing Rules are evaluated before any other routing rules on a service router.

Create a Dynamic Routing Rule

To create a dynamic routing rule:

  1. Navigate to AIOps Event Orchestration and select your preferred Orchestration.
  2. Click Service Routes New Dynamic Route.
A screenshot of the PagerDuty UI emphasizing the "New Dynamic Route" button

Select New Dynamic Route

  1. In the modal, select the Event Field where routing information (i.e., either service name or service ID) is located. You can select any single PD-CEF field. For example, event.summary refers to the Summary field. You may also select free text field formats: Raw Event (e.g., raw_event.foo.bar) or Custom Details (e.g., event.custom_details.foo.bar).
  2. In the Format dropdown, select Service name (default) or Service ID. This setting specifies if the Dynamic Routing Rule should attempt to match events to services via its name or ID (e.g., PXXXXXX).
  3. In the Extraction Regex field, enter an RE2 regex pattern that identifies where service information is located in the selected field. If you specify a capture group, the part matched by the capture group will be used to set the variable. Otherwise, the entire match will be used.
  4. Click Save.

📘

Dynamic Routing Rules

Please keep the following information in mind while configuring a dynamic routing rule:

  • If an event does not match to a service based your dynamic routing rule configuration, it will evaluate against the next routing rule in the service's router.
  • If an event contains malformed information in the field targeted for dynamic routing, Event Orchestration will drop the event (e.g., if you have configure a dynamic routing rule based on service ID, but the specified field does not contain data shaped like a service ID).
  • Global Orchestrations execute before Dynamic Routing Rules. Manipulating a CEF field in a Global Orchestration may lead to unintended behavior if a downstream Dynamic Routing Rule reads from that field.
  • Service names are stored as cached data for routing purposes. It can take up to 10 minutes for the service name cache to update after a service’s name has been changed. During this time events will continue to match to services based on the previously configured service names.

Create a Routing Rule

To create a routing rule:

  1. Navigate to AIOps Event Orchestration select your preferred Orchestration Service Routes click New Service Route.
    • Optional: next to , enter a description for your new rule.
  2. From the dropdown What service should events route to?, select a service you’d like to route events to.
  3. Under When should events be routed here?, select one of the following options:
  4. Depending on your selection in the previous step, configure the rule according to your needs.
  5. Click Save.

👍

Event Orchestration Examples

Please see Event Orchestration Examples for more details on both common and advanced routing rule configurations.

Global and Service Orchestration Rules

  • Global Orchestration rules are optional rules that can be created before a routing rule. These rules will have their own conditions and will be linked to rules that precede and follow them.
  • Global Orchestration rules can include any action except those that define how an event is routed to a service.
  • Service Orchestration rules are optional rules that can be created after a routing rule. These rules will have their own conditions and will be linked to rules that precede and follow them.
  • Service Orchestration rules determine which actions are applied to events during processing before an incident triggers.
  • When there is more than one potential global or service orchestration rule that an event could match against, that group of rules is called a set. Events will match against whichever rule in a set has the highest ordinality (i.e., appears first, either in the UI or API).
  • When creating a Global or Service Orchestration rule, any conditions can be created and combined, similar to a routing rule, however, threshold-based conditions can only ever exist on their own. If a user wants to create a rule with a threshold-based condition they must make a specific service rule that only has that condition present.

📘

Ruleset Limits

A set of rules grouped vertically in the web app can be referred to as a "ruleset." Rulesets can have up to 25 rules. As needed, you can nest rules, which will allow you to create more than 25 rules in an orchestration, however we advise against making orchestrations too complex, as this can make it difficult to maintain, or for other team's to understand your orchestration's logic.

Configure a Global or Service Orchestration Rule

  1. Navigate to AIOps Event Orchestration select your preferred Orchestration select Global Orchestration or Service Orchestrations (and select a service), and click New Rule on an empty Orchestration rule page or on an Orchestration rule page with existing rules.
  2. A modal will appear on the page. Select the type of condition, then specify the appropriate conditions for this new rule.
  3. Click Next.
  4. Specify the actions that should be applied in the following menus:
  1. Click Save to save this new rule.

Incident Data

Basic Event

Determine how the incident should be created, or override default incident creation.

Incident Action OptionInstructions and Details
Create an alert but pause notifications

(Advanced Event Orchestration)		An incident can trigger after a predetermined amount of time. Enter the amount of time you would like to pass before an incident and notifications are created in the Suspend alert for _ second(s) before triggering incident field. See Pause Incident Notifications for more details.
Suppress alert

(Advanced Event Orchestration)		Alerts that match the routing rule conditions will suppress, meaning they will not create PagerDuty incidents or notify responders. You may want to suppress alerts if you don't want responders to be notified outside of support hours, or within any specific timeframe. Suppressed alerts are visible in the Alerts table.
Drop incident and stop processing

(Advanced Event Orchestration)		Note: Only available for Global Orchestration rules.
Events that match this rule will be dropped out of PagerDuty’s event ingestion pipeline. They will generate no data within the PagerDuty platform, will not create Incidents, and will not be turned into Alerts. It is not possible to audit PagerDuty for events that were dropped.
Set PriorityTo set priority, check the Set Priority checkbox and select your desired priority level from the dropdown.

Incident priority allows the classification of incidents based on a level of prioritization. Incident priority must be enabled on your account before it can be set with orchestrations.
Override Assigned Escalation PolicyTo override a service's assigned Escalation Policy , select your preferred escalation policy from the dropdown. The on call responder associated with the selected escalation policy will receive notifications when an incident triggers, instead of the responder associated with the service's configured escalation policy.
Add a note to the incidentTo add a note, under Additional Context, enter the text of the note that you would like to be added to an incident that meets your rule’s criteria.

Notes can be used to help responders resolve incidents quicker by including information or links related to the system that the event comes from.

Event Fields

Event Fields allow you to copy important data from any combination of source event fields into any PagerDuty Common Event Format (CEF) field. Translate difficult machine terms and code into helpful context for responders so they can effectively respond to the problem. Event Fields can also be used to enhance PagerDuty AIOps capabilities by customizing alerts, influencing the Intelligent Alert Grouping and Intelligent Triage machine learning algorithms.

Enrich and update any CEF field or even create new key values within the Custom Details object using custom variables. When events are sent to PagerDuty, they are transformed into Common Event Format. You can replace any CEF field or even create new key values within the Custom Details object.

📘

Availability

Event Fields are available with Advanced Event Orchestration.

Event Orchestrations transformations

Event Orchestrations Event Fields

  1. Click Add Event Field.
  2. Perform the following:
Replace Event FieldInstructions and Details
Common Event Field (CEF)Select your preferred Common Event Field (CEF) from the Event field to set dropdown.
Right dropdownSelect whether you would like to use Regex or a Template from the Replace using dropdown.
Template optionIf you chose a text template from the dropdown, you can reference a variable defined in the Create Custom Variables section above using {{ }} brackets (e.g., {{class}}) in the Replace with value field.
Regex optionIf you chose the Regex option from the dropdown, enter a Value and the origin Source.

Event Fields are available with both Global Orchestrations and Service Orchestrations. For Service Orchestrations, custom variables must extract from CEF fields of the event payload, where for Global Orchestrations, custom variables can extract from any field of the event payload.

Deduplication

The dedup_key field is used to merge events into a single alert. Events with the same dedup_key can update the status of the alert they are automatically merged into.

To set a deduplication key:

  1. In the Event Fields section of your rule, under Define Custom Variable, create a variable from event fields by entering a Variable Name, Event Field and the Extraction Regex. You may use Sample Events on the right to create these variables.
  2. Next, select Dedup Key from the Event field to set dropdown. Select whether you would like to use Regex or a Template from the right hand dropdown. If you use a Template, you can reference a variable defined in step 1 using {{variable name here}} brackets in Replace with value field. If you are using Regex, enter a Value and the Source.

🚧

Using Event Fields on Email Events

Event Fields are only available for email events sent to Global Integrations. Users can extract data from the email event using regex matching into multiple variables. Currently, only Summary and Dedup_Key Common Event Fields can be replaced for email events.

Custom Fields

Replace or fill incident Custom Fields with static text, event fields or variables. This allows you to bring event data into your incidents and use event-driven automation in your incident management. For example, you may wish to pass the component, class, or other custom details event fields into incident custom fields. Once a value is in a custom field, it can be used elsewhere in our product such as Status Update Templates or as part of our ServiceNow integration, which supports custom fields bi-directional syncing.

📘

Availability

  • Business Plans and above are able to set Custom Fields to static text values through Event Orchestration (#3 in table below)
  • Enterprise for Incident Management, Digital Operations (Legacy), Event Intelligence or AIOps plans are able to dynamically populate custom field values with Event Orchestration. This includes replacement with both event fields and variables (#1 and #2 in table below).

📘

Custom Fields Set on New Incidents

Custom fields will only be set when the event triggers a new incident. Subsequent events and alerts that are grouped or deduplicated into the incident do not update Custom Fields at this time.

  1. Click Add Custom Incident Field.
  2. Perform the following:
Custom field to setReplace with value
Select your preferred Custom FieldThere are a few options for how to replace values:

1. Replace with event field: Use double curly brackets to reference event fields. E.g.: {{event.component}}.
2. Replace with variable: Use double curly brackets to reference both cache variables and rule variables. Note: variables must first exist before they can be referenced. E.g.: {{variables.env}} {{cache_var.foo_bar}}
3. Replace with static text: Simply use static text, such as foo-bar in the above image.

Please see the Event Fields section for more information.

Field Types Supported
Custom Field TypeExpected Event Orchestration Format Example
Text As isHello World
Single Select As-is OptionA
Multiple Select Array with quoted strings[“OptionA”, “OptionB”]
TagArray with quoted strings[“TagA”, “TagB”]
Date-Time ISO-8601 formatted date-time with timezone. Microseconds are accepted but not required.

- yyyy-mm-ddThh:mm:ss.sss±hh:mm

- yyyy-mm-ddThh:mm:ss±hh:mm
- yyyy-mm-ddThh:mm:ss.sssZ
- yyyy-mm-ddThh:mm:ssZ 		- 2024-01-25T16:17:18.315+0500
or

- 2024-01-25T16:17:18+0500
or
- 2024-01-25T16:17:18.315Z
or
- 2024-01-25T16:17:18Z
Decimal As-is 101.234
IntegerAs-is 1234
BooleanTrue or False

Note: If left empty, defaults to False		True

Create Custom Variables

You can define custom variables to capture snippets from a source event using regex matching for use with Event Fields.

Create Custom VariablesInstructions and Details
Variable NameEnter a name for the variable.
Event FieldEnter the full path of a CEF or non-CEF field to read the variable from. For example,
use event.summary for the Summary CEF field, or raw_event.fieldname to read from the original event's fieldname data.
Extraction RegexEnter an RE2 regular expression, which will match the part of the field you want to extract. If you specify a capture group, the part matched by the capture group will be used to set the variable. Otherwise, the entire match will be used.

You may use Sample Events on the right as a reference for variables by clicking Show Details. You may add multiple variables by clicking Add Variable in the upper right of the Customize Event Fields panel, or delete variables by clicking the trashcan icon directly to the right of the variable.

Alert Data

Alert ActionInstructions and Details
Set SeverityTo set severity, check the Set Severity checkbox and select info, error, warning or critical from the dropdown.

Severity can be used for Dynamic Notifications, which are defined by the service settings. To use Dynamic Notifications, the events must be routed to a service that sets Dynamic Notifications based on severity levels.

With severity controlled via Event Orchestrations and the service configured to use dynamic notifications, you can control the incident urgency.
Set a custom trigger/resolve action

(Advanced Event Orchestration)		If you would like to automatically trigger or resolve an alert based on your event rule's conditions, in the section Override the given event_action behavior, check the box for either Always trigger an alert or Always resolve an alert.

Automation

Webhooks

Webhook Actions allow response teams to easily define a dynamic webhook through an Orchestration and send a custom payload to a specified endpoint. This enables responders to automate actions like restarting a server, clearing logs, and reverting bad deploys.

📘

Availability

Webhooks are available with Advanced Event Orchestration.

The payload can be set using a combination of several predefined variables, created using Event Fields. Automated Triggers can be created using Service Orchestration rules.

OptionInstructions and Details
Enable webhookCheck the checkbox to enable and then select if the webhook should be manually or automatically triggered.
NameEnter a button name.
API EndpointEnter the API endpoint for the payload.
HeadersSelect the + Header Field button and enter the Name and Value for the webhook header.
JSON Body Fields (Optional)Enter a name and value using the variables created using the Event Fields feature in the preceding section to further enrich the custom action.

Automation Actions

Runbook Automation Self-Hosted allows users to specify which PagerDuty Automation Action should be run based on an incoming event’s payload. Responders can initiate automated diagnostics or remediation as soon as an incident is created, saving critical time during major incidents, or preventing major incidents altogether.

Event Orchestration is able to automatically trigger any Automation Action that has already been created and is allowed for use on the service Event Orchestration is being used on. Please read our Automation Actions article to learn more about how to configure actions and their permissions.

Automation Actions can only be enabled for Service Orchestration Rules.

📘

Availability

This feature is available for Business, Enterprise for Incident Management and Digital Operations (Legacy) plans that meet the following criteria:

  • Business Plans: Accounts must have the PagerDuty AIOps and PagerDuty Automation Actions add-ons, as well as Advanced Event Orchestration.
  • Enterprise for Incident Management and Digital Operations (Legacy) Plans: Accounts must have the PagerDuty Automation Actions add-on and Advanced Event Orchestration.

Please contact our Sales team if you are interested in PagerDuty AIOps, and you may fill out this form if you are interested in Automation Actions.

To select an Automation Action:

  1. Select an Automation Action from the Automation Action dropdown. Initially this dropdown will say “No automation selected” until an Automation Action is specified.
  2. Only one Automation Action can be specified per rule, however multiple Automation Actions can be triggered during incident creation by nesting rules one after another.

📘

Tip

If an Event Orchestration rule has an Automation Action specified that is no longer valid due to it being deleted or not available, the rule with the Automation Action will persist and still function, however the Automation Action will no longer trigger. Rules where this occurs will have a yellow icon on the Automation tab when you edit the rule, to indicate that the Automation Action is no longer available.

Copy Service Rules

📘

Copied Service Rule Limitations

Copied service orchestration rules will persist for 5 minutes after being copied or until they are manually deleted via the successful rule copy icon located in the top right of the service orchestration canvas.

Copying Event Orchestration Routing Rules is not currently supported.

To copy an existing service orchestration rule:

  1. Click the menu shown and select Copy Rule.
  2. An icon will appear in the top right of the service orchestration canvas indicating that a rule has been copied when this is successful. Copying a rule copies all of its conditions, actions, and action configurations.
  3. Copied rules can be pasted after, before, or between any existing service orchestration rules. To paste a rule, you may either click the button shown on the service orchestration canvas between existing rules, or click the menu for an existing rule and select Paste Rule.
  4. A rule creation modal will appear and it will contain all the conditions, actions, and action configurations for the rule that was copied. These conditions, actions, and action configurations can be edited, or they can be left as they are. Click Save.

Manage Integrations

When you create an Event Orchestration, PagerDuty will automatically create a default integration set, which consists of an Events API v2 integration key and an integration email address.

If you delete or migrate all integration keys associated with an orchestration, the orchestration's rules will persist, however they will not be able to receive or evaluate events.

Create an Integration

  1. In the web app, navigate to AIOps Event Orchestration.
  2. Select your desired Orchestration from the list.
  3. Select Integrations, click New Integration and select Create new integration.
  4. Enter a name for the integration and click Save.

📘

Integration Limit

One orchestration can have up to 10 integrations.

Rename an Integration

  1. In the web app, navigate to AIOps Event Orchestration.
  2. Select the Orchestration with the integration you'd like to rename from the list.
  3. Select Integrations.
  4. On the top-right of the integration you'd like to rename, click .
  5. Enter a new name for the integration and click Save.

Delete an Integration

  1. In the web app, navigate to AIOps Event Orchestration.
  2. Select the Orchestration with the integration you'd like to delete from the list.
  3. Select Integrations.
  4. On the top-right of the integration you'd like to delete, click .
  5. In the confirmation modal, click Delete.
    1. Note: This action cannot be undone.

Migrate an Integration from an Existing Ruleset or Orchestration

In order to assist with the transition from Rulesets to Event Orchestration, you may wish to manually migrate an integration from Rulesets to a new orchestration.

Migrate an Integration from a Ruleset

  1. In the web app, navigate to AIOps Event Orchestration.
  2. Select your desired Orchestration from the list.
  3. Select Integrations, click New Integration and select Permanently migrate integration from existing Ruleset.
  4. From the dropdown in the modal, select the Ruleset you'd like to migrate.
  5. Click Migrate.

Note: This action is permanent and cannot be undone.

Migrate an Integration from an Orchestration

  1. In the web app, navigate to AIOps Event Orchestration.
  2. Select your desired Orchestration from the list.
  3. Select Integrations, click New Integration and select Migrate integration from existing Orchestration.
  4. From the dropdowns in the modal, first select the Orchestration with the desired integration, then select the Integration.
  5. Click Migrate.

Manage Rules

The following information details how to manage the three rule types that are relevant to Event Orchestration:

Manage Global Orchestration Rules

Edit, Disable or Reorder a Global Orchestration Rule

  1. Navigate to AIOps Event Orchestration.
  2. Select the Orchestration from the list that contains the Global Orchestration rule you wish to edit or disable.
  3. Select Global Orchestration.
  4. On the following page:
    1. To edit: Click in the top-right of your desired rule. In the modal, make any changes to conditions or rule information and click Save.
    2. To disable: Click in the top-right of your desired rule and select Disable. The rule will be immediately disabled, and will appear in the rule's description. To enable the rule again, click and select Enable.
    3. To reorder: Click in the top-right of your desired rule and select an action, as appropriate (e.g., Move to Top, Move Up, Move Down, Move to Bottom).

Delete a Global Orchestration Rule

  1. Navigate to AIOps Event Orchestration.
  2. Select the Orchestration from the list that contains the Global Orchestration rule you wish to delete.
  3. Select Global Orchestration.
  4. On the following page, click in the top-right of your desired rule and select Delete.
  5. In the confirmation modal, click Delete.
    1. Note: This action cannot be undone.

Manage Routing Rules

Edit or Disable a Routing Rule

  1. Navigate to AIOps Event Orchestration.
  2. Select the Orchestration from the list that contains the routing rule you wish to edit or disable.
  3. Select Service Routes.
  4. On the following page:
    1. To edit: Click to the right of your desired routing rule. In the modal, make any changes to conditions or rule information and click Save.
    2. To disable: Click to the right of your desired routing rule and select Disable. The rule will be immediately disabled, and will appear in the rule's description. To enable the rule again, click and select Enable.
    3. To reorder: Click in the top-right of your desired rule and select an action, as appropriate (e.g., Move to Top of All Routes, Move Up, Move Down, Move to Bottom of All Routes).

Delete a Routing Rule

  1. Navigate to AIOps Event Orchestration.
  2. Select the Orchestration from the list that contains the routing rule you wish to delete.
  3. Select Service Routes.
  4. On the following page, click to the right of your desired routing rule and select Delete Rule.
  5. In the confirmation modal, click Delete.
    1. Note: This action cannot be undone.

Manage Service Orchestration Rules

There are two places where you can manage Service Orchestration rules:

Manage a Service Orchestration Rule in Event Orchestration

Edit, Disable or Reorder a Service Rule

  1. Navigate to AIOps Event Orchestration.
  2. Select the Orchestration from the list that contains the service rule you wish to edit or disable.
  3. Select Service Orchestrations and select your desired service from the dropdown.
  4. On the following page:
    1. To edit: Click to the right of your desired routing rule. In the modal, make any changes to conditions or rule information and click Save.
    2. To disable: Click to the right of your desired routing rule and select Disable. The rule will be immediately disabled, and will appear in the rule's description. To enable the rule again, click and select Enable.
    3. To reorder: Click in the top-right of your desired rule and select an action, as appropriate (e.g., Move to Top, Move Up, Move Down, Move to Bottom)

Delete a Service Rule

  1. Navigate to AIOps Event Orchestration.
  2. Select the Orchestration from the list that contains the service rule you wish to delete.
  3. Select Service Orchestrations and select your desired service from the dropdown.
  4. On the following page, click in the top-right of your desired rule and select Delete.
  5. In the confirmation modal, click Delete.
    1. Note: This action cannot be undone.

Manage a Service Orchestration Rule on a Service's Settings Tab

Edit, Disable or Reorder a Service Rule

  1. Navigate to Services Service Directory and select your desired service.
  2. Select the Settings tab, scroll to the Event Management section and click Service Orchestration Rules.
  3. On the following page:
    1. To edit: Click to the right of your desired routing rule. In the modal, make any changes to conditions or rule information and click Save.
    2. To disable: Click to the right of your desired routing rule and select Disable. The rule will be immediately disabled, and will appear in the rule's description. To enable the rule again, click and select Enable.
    3. To reorder: Click in the top-right of your desired rule and select an action, as appropriate (e.g., Move to Top, Move Up, Move Down, Move to Bottom)

Delete a Service Rule

  1. Navigate to Services Service Directory and select your desired service.
  2. Select the Settings tab, scroll to the Event Management section and click Service Orchestration Rules.
  3. On the following page, click in the top-right of your desired rule and select Delete.
  4. In the confirmation modal, click Delete.
    1. Note: This action cannot be undone.

Send Events to a Global or Service Orchestration

Send Events to Global Orchestrations

  1. Once you have configured a Global Orchestration, navigate to AIOps Event Orchestration click the name of your Global Orchestration.
  2. Click Integrations to navigate to the Integrations section of the Event Orchestration. Each Event Orchestration will have the following routing options for for each integration key set:
    • Integration Key: You may use the integration key with many of our tool integrations. You may also use it to send events via API (see below).
    • Email Address: You may use the email address with our Email Integration.
    • HTTP Endpoint for API: You may use the integration key and our Events API V2 to send events to this endpoint.

Send Events to Service Orchestrations

  1. Add your preferred tool integration or a Custom Event Transformer integration to the service where you configured the above Service Orchestration rule(s).
  2. Once you have added the integration to your service, you will be directed to the Integrations tab. Find the integration that was just added in the list, click to the right and then copy its Integration Key.
  3. Use this integration key to complete your integration according to its guide, and then send test events to your service.

Switch to Service Orchestrations

If your account has services that are currently using Service Event Rules, you may choose to switch to Service Orchestrations, which is a more robust feature. When you switch to Service Orchestrations, events that land on the service, regardless of source, will be evaluated by Event Orchestration instead of Event Rules.

Note: Email events are an exception. If an email event comes in via a service-level email integration, the Service Orchestration will be ignored, and only the email filters and rules are applied. In contrast, if an email comes in on a Global Orchestration, then the Global Orchestration rules are applied, followed by Service Orchestration rules, if present.

To switch to Service Orchestrations:

  1. Navigate to Services Service Directory and select your desired service.
  2. Select the Settings tab, scroll to Event Management and then click Switch to Service Orchestrations.

The service can be switched back to processing events using Event Rules using the same process.

Pause Incident Notifications

As a feature of Event Orchestration, you can configure rules to create alerts in a Suspended state, which will pause incident creation and notifications for a predefined amount of time. During the pause period, new alerts will be viewable in the Alerts Table with a Suspended status. You can take one of two actions on a Suspended alert: either Trigger it, which will immediately create an incident and send out responder notifications, or Resolve it, which prevents the incident and notifications from being created altogether.

📘

Availability

Pausing incident notifications is available as part of the Advanced Event Orchestration package. Please contact our Sales Team if you would like to upgrade to a plan with this feature.

Pausing incident notifications is only available on incidents generated by our Events API, and does not work with those generated from our REST API.

To configure paused incident notifications:

  1. While configuring a Global or Service Orchestration Rule, select the Incident Data tab and check the box next to Pause notifications.
  2. Enter a value (in seconds) for Suspend alert for ___ seconds before triggering an incident.
Select pause notifications

Select pause notifications

  1. Click Save.

View and Take Action on Paused Alerts

To view alerts in a paused state, navigate to the Alerts table in the upper menu of the web app. Alerts with paused incident notifications will have a Suspended status:

Suspended alert

Suspended alert

To take action on a paused alert:

  1. Click the incident title to view its details page.
  2. On the right hand side, click Trigger to trigger an incident and send notifications, or Resolve if you would like to resolve the alert without triggering an incident/notifications. You will see a green dialog that confirms that the alert has been triggered or resolved.
Alert details

Alert details

The pause period and action taken will also appear in the Alert Log timeline, accessible by clicking an alert's title to view the detail page:

Alert log

Alert log

PagerDuty Condition Language (PCL)

You can write your Global Orchestration, routing, or Service Orchestration rules in PagerDuty Condition Language (PCL) if you need more control over the logic than the UI offers. Select while creating or editing a rule to access this feature.

Edit a routing rule using PagerDuty Condition Language (PCL)

Edit a routing rule using PagerDuty Condition Language (PCL)

For more information and examples, please see our developer documentation PCL Overview.

Event Orchestration Pricing Tiers

Event Orchestration has two pricing tiers that come with their own set of features and actions:

Basic Event Orchestration

All pricing plans have access to Basic Event Orchestration features. Basic Event Orchestration comes with the following:

Feature SuiteFeaturesPricing Plan Availability
Event Orchestration- Service Event Orchestrations

- Service Routing
- Global Integration Keys		All plans
Incident Actions- Set Priority

- Add a Note		All Plans
Alert Actions- Set Severity

- Custom Trigger/Resolve Actions		All Plans

Advanced Event Orchestration

Advanced Event Orchestration includes all features in Basic Event Orchestration, plus additional features in the table below. Please contact our Sales team if you are interested in changing your pricing tier.

Feature SuiteAIOpsLegacy plans: Event Intelligence/Digital Operations
Global Event OrchestrationAvailableNot available
Event Orchestration- Rule Nesting
-Scheduled/Recurring/Threshold Conditions
-Variables
- Dynamic Routing
- Dynamic Escalation Policies		- Rule Nesting
- Scheduled/Recurring/Threshold Conditions
Incident Actions- Suppress Incident
- Pause Incident
- Drop Incident		- Suppress Incident
- Pause Incident
Alert Actions- Set Severity
- Custom Trigger/Resolve Actions		- Set Severity
- Custom Trigger/Resolve Actions
Event FieldsDynamic Field Enrichment and ExtractionDynamic Field Enrichment and Extraction
Runbook Automation Self-HostedEvent-driven automation with PagerDuty Automation (requires add-on)Event-driven automation with PagerDuty Automation (requires add-on)
WebhooksAvailableAvailable

FAQ

What's the difference between Event Rules and Event Orchestration?

Many of the new features in Event Orchestration are enhancements on Event Rules, however, there are also some substantial departures from existing event rule functionality:

  • With Event Orchestration, rules are now evaluated iteratively. They start with Global Orchestration rules, which define actions to be performed before routing and end with the definition of Service Orchestration rules, allowing individual teams to define their own incident creation behavior at the service level. This is substantially different from the earlier approach, which evaluated Global Event Rules (GERs) and Service Event Rules (SERs) at the same time. This change has allowed us to remove both GERs and SERs from Event Orchestration.
  • As an intentional part of this change, Event Orchestration now also supports evaluation of conditions against either raw or PD-CEF-formatted event payloads.
  • Another substantial change with Event Orchestration is the introduction of different rule types, as well as a rigid rule structuring. This improves observability, ensures context is established when actions are taken, and streamlines rule creation.
  • Event Orchestration has made improvements to integration key management by adding the ability to migrate integration keys between rulesets and orchestrations as well as the ability to combine multiple integration keys together into a single Event Orchestration, reducing the number of Global Orchestrations and Routers required to ensure events are properly normalized and directed to the right teams.

How many services can an Event Orchestration route to?

You can route events to as many as 1,000 services in one orchestration. However, each service can only have one routing rule.

Can a service have more than one set of Service Orchestration rules?

No, each service can only have one set of Service Orchestration rules (orchestration rule canvas). Here, the same set of orchestration rules will be applied to all events that land on the service, which streamlines the Event Management process.

Can events be routed to a service from the Unrouted rule with Event Orchestration?

Yes, selecting a service to route events that do not match with any of the routing rules in an orchestration is possible with Event Orchestration.

Does Event Orchestration support email events?

Yes, Event Orchestration supports email events. Rules may be based on the content of an email by entering the email field as custom details in the event field event.custom_details.subject. Having email events as custom fields means email events are treated as CEF events that you can apply Event Fields to.

Does Event Orchestration support the "contains/does not contain" filter?

Yes, Event Orchestration supports the "contains/does not contain" filter. It has, however, been reworded and replaced with the "matches part/does not match part" filter.

Are condition values case sensitive with Event Orchestration?

No, condition values with Event Orchestration are case insensitive.

Does Event Orchestration support time-based conditions?

Yes, you can set time-based conditions with Event Orchestrations that avoid trapping events that don't match condition pairs. With Event Orchestration, events outside the set schedule will be evaluated by the next rule.

To configure more advanced time-based conditions with an and operator, you'll need to edit the PagerDuty Condition Language (PCL) directly. Please see the section Advanced Configuration with the and Operator for more information.

Is there a payload size limit for an Event Orchestration?

Yes, the maximum payload size for an Event Orchestration is 4 MB. Orchestrations greater than 4 MB will return 413 Payload Too Large errors and will not save. This limit is most relevant to users who manage Event Orchestration via the REST API or with infrastructure-as-code tools, such as Terraform.

How many integrations can be added to an Event Orchestration?

Users can associate up to 10 integrations with one Event Orchestration.

Updated about 1 month ago