Rulesets
Route events and take action based on predefined event rules
The Rulesets feature allows you to route events to an endpoint and create collections of event rules, which define sets of actions to take based on event content. Accounts with Free, Starter (legacy), Team (legacy), and Professional plans will have access to a single global ruleset, while Business, Enterprise for Incident Management and Digital Operations (Legacy) plans will now have the ability to create multiple, unlimited global rulesets. For these higher-tier plans, each ruleset will have an individual endpoint and they can be associated with specific Teams, giving Team Managers edit-access to their rules.
End-of-Life
Rulesets and Event Rules will end-of-life on January 31, 2025. We highly recommend that you migrate to Event Orchestration as soon as possible so you can take advantage of the new functionality, such as improved UI, rule creation, APIs and Terraform support, advanced conditions, and rule nesting.
Ruleset Types
There are two ways to organize event rules in your account, based on your integration use case:
- Global Rulesets: When a new integration event stream has more than one service destination, you can use global event rules and a global Integration Key to ingest and route your events to the right service. If your existing service integrations have the same event stream integrated into multiple services, consider migrating those integrations to a global ruleset and use routing rules to direct events to the correct services. Global rulesets support event rules for email integrations, but do not support Custom Event Transformers.
- Service Event Rules: When integrations already exist on a Service, you can use Service Event Rules to evaluate your events instead of migrating them to a global event rule. Service event rules support Custom Event Transformers, but do not support event rules for email integrations.
Global Rulesets
Events sent to a global ruleset Integration Key or emails sent to a global email address will be evaluated by the destination global ruleset. In order to create alerts and incidents on a destination service, a global event rule must have Create an incident on a service selected as one of its actions. When an event is routed to a service that contains service event rules, then it will also be evaluated by service event rules at the time of event ingestion, after global event rules have been evaluated.
All plans will have access to a Default Global Ruleset that is available out-of-the-box. Accounts on Business, Enterprise for Incident Management, and Digital Operations (Legacy) plans have the option to create multiple additional rulesets, and they can opt to assign these additional rulesets to Teams.
To access your Default Global Ruleset:
Free, Professional, Starter (legacy), and Team (legacy) plans: Your account will come with a single global ruleset, which you can access by navigating to AIOps Event Rules. Click New Event Rule to begin creating event rules.
Business, Enterprise for Incident Management and Digital Operations (Legacy) plans: Navigate to AIOps Event Rules Default Global Ruleset. Click New Event Rule to begin creating event rules. If you would like to create additional rulesets outside of this Default Global Ruleset, follow instructions in the next section.
Create Multiple Rulesets and Assign Teams
Required User Permissions
- Account Owners, Admins and Global Admins can create/edit/delete rulesets by default. Users with Team Manager roles can create/edit/delete rulesets for their assigned Team.
- Business, Enterprise for Incident Management and Digital Operations (Legacy) plans have the ability to create multiple, unlimited global rulesets, each of which can be assigned to a Team. Note: If you do not assign a new ruleset to a team, then it will behave as an additional global ruleset.
- Account Owners and Admins have view access to all rulesets. Users below Admin will only be able to view rulesets associated with Teams they are assigned to.
- Navigate to AIOps Event Rules Create Ruleset.
- Enter a Name, optionally select a Team and click Create Ruleset.
- Click New Event Rule to begin creating event rules. If you would like to create multiple rulesets with unique, individual endpoints, repeat the above steps as needed.
Service Event Rules
Events that are sent to any service Integration Key will be evaluated by the event rules present on that service at the time that an event is ingested. This behavior is different from global rulesets, which apply rules to all inbound events sent to a global integration key. Service event rules are available on all plans.
Create Service Event Rules
Requirements
In order to use service event rules, the service must be configured to create both Alerts and Incidents.
To create a service event rule:
- Navigate to Services Service Directory click the name of the service where you would like to configure your event rule.
- Click the Settings tab. If you have not configured any rules for this service, click Add One. If you have configured other rules for this service, click Manage Service Event Rules and then click the +New Event Rule button.
- Follow the instructions in steps 1-4 of Create an Event Rule.
Create Event Rules
Event rules allow you to set actions that should be taken on events that meet your designated rule criteria. You can choose whether you would like criteria-satisfying events to trigger incidents, or if you would like them to be suppressed and viewable in your Alerts table. Depending on your plan and account add-ons, you may have additional actions to take on incidents or suppressed alerts.
Event rules must be created in a global ruleset or on a service. Follow the instructions in the above sections to determine where you are creating your event rules, and then proceed with the following steps.
Both Global Event Rules and Service Event Rules can also be managed via API.
Event Rule Limitations
Each ruleset has a limitation of 25,000 event rules. Once the limit of 25,000 rules is met, users will not be able to add any further event rules to the ruleset.
Recommended Preparation
We recommend sending an alert to your PagerDuty account in order to inspect the alerts’ JSON key:value pairs and build rule conditions from there. Please read our API documentation for more information on how to send a simple alert into PagerDuty.
Example in CURL:
curl --location --request POST 'https://events.pagerduty.com/v2/enqueue' \
--header 'Content-Type: application/json' \
--data-raw '{
"payload": {
"summary": "Example alert on host2000.example.com",
"timestamp": "2020-06-17T08:42:58.315+0000",
"source": "monitoringtools:cloudvendor:central-region-dc-01:002000987:cluster/api-stats-prod-003",
"severity": "info",
"component": "postgres",
"group": "prod-datapipe",
"class": "deploy",
"custom_details": {
"ping time": "1600ms",
"load avg": 0.90
}
},
"dedup_key": "uniquededupkey",
"routing_key": "R015DM1NF3UYE4NSWN8WBF6FS062ZZZZ",
"images": [{
"src": "https://www.pagerduty.com/wp-content/uploads/2016/05/pagerduty-logo-green.png",
"href": "https://example.com/",
"alt": "Example text"
}],
"links": [{
"href": "https://example.com/",
"text": "Link text"
}],
"event_action": "trigger",
"client": "Sample Monitoring Service",
"client_url": "https://monitoring.example.com"
}'
This alert will be visible in the Base conditions from the incoming JSON events below section when you are configuring a rule.
![image.png Create rule](https://files.readme.io/f50cfc0-image.png)
Create rule
Within the sample alert JSON body, you have the option to click the decided key:value pair, and this will automatically populate the relevant rule’s conditions.
Step 1: When events match these conditions
Depending on your preferences, indicate the condition(s) that should be met when this event rule searches inbound events.
![rulesets-conditions.png Enter conditions for event rule](https://files.readme.io/100cb18-rulesets-conditions.png)
Enter conditions for event rule
- Select if you would like this rule to be run when All or Any of the following conditions are met. If you select All, every condition must be met for the rule to run. If you select Any, then only one condition must be met for the rule to run.
- In the first condition box, select the event field that you would like the rule to search for from the following options:
- Base conditions from the incoming JSON events below: Depending on your account's activity, you may have recent events appear on the right side of the screen. To add fields from these events, click Show Details and click on the field you wish to use for this condition. This will auto-populate the field in the left hand condition box, where you may also edit and/or delete if necessary.
- Events Sent Through the API: You can use the JSON field names directly, i.e.
description
. For nested fields, separate names with a dot (.), i.e.payload.summary
. If you are sending data through additional fields, enter them exactly as they are sent to PagerDuty. For example, if your events have atags
field, enter that field name in your rule condition astags
. - Events Sent Through Email (Note: Only available in global rulesets, not with service event rules): Rules may be based on the content of an email by entering the appropriate email field as the event field. The most common email fields are:
headers.from.0.address
(the from address)
** Note:.0.address
refers to the 1st position in a list of emails. If you would like to generally search through a list of emails (either in the "to" field or the "from" field), please set toheaders.from
orheaders.to
headers.subject
(the subject line)body
(the email body)
If your monitoring tool sends emails with custom headers, you can create rules based on those headers by including the prefix headers in front of the email header name when you enter that field name in your rule condition.
- In the middle dropdown, select how the above event field should be filtered:
- The field contains/does not contain a value.
- The field equals/does not equal a value (this operation requires the field to be passed in as a string).
- The field exists/does not exist.
- The field matches/does not match a regular expression. Regular expressions must use RE2 syntax. Note that unlike service-level event rules, global rules only support one (e.g. the first) capture group within the regex.
Negative Operations
Rules with negative operations, such as does not contain or does not equal, will match events that do not contain your specified value and events that do not contain the field at all. As an example:
- severity field does not equal
critical
This will match events where the severity field does not equal
critical
and events that do not contain a severity field at all.If you'd like to avoid this, you must add an additional condition that matches only when the field exists. For example:
When all conditions are true:
- severity field exists
- severity field does not equal
critical
Note that you must select that all conditions must be true for the rule to match.
- In the last condition box (where applicable), enter a value to apply to the event field and its filter.
Note
The value field is case-sensitive.
- If you only have one condition, proceed with the following steps. If you would like to add more than one condition, click Add Condition and repeat steps 1-4 as many times as needed. When you are finished adding conditions, proceed with the following steps.
Step 1.1: Customize Event Fields (Optional)
Dynamic Field Enrichment & Extraction
Dynamic Field Enrichment & Extraction is a tool used to normalize event payloads using Event Rules. This capability allows you to copy important data from any combination of source event fields into any PagerDuty Common Event Format (CEF) field. Translate difficult machine terms and code into helpful context for responders so they can effectively respond to the problem. Dynamic Field Enrichment & Extraction can also be used to enhance PagerDuty AIOps capabilities by customizing alerts, influencing the Intelligent Alert Grouping and Intelligent Triage machine learning algorithms.
Custom Event Fields vs. Custom Fields on Incidents
This feature is intended to customize event fields within Alerts, as opposed to creating custom fields in the incident UI. To create the latter, please read Custom Fields on Incidents.
Availability
This feature is available with the PagerDuty AIOps add-on. Please contact our Sales Team if you would like to purchase this product.
![dad0bfe-DFEE_Configuration.png Customize event fields](https://files.readme.io/c52eec6-dad0bfe-DFEE_Configuration.png)
Customize event fields
Dynamic Field Enrichment & Extraction is composed of two functions:
- Create Custom Variables: Capture snippets from a source event using regex matching for later use.
- Replace Common Event Format Fields: Enrich and update and CEF field or even create new key values within the Custom Details object using custom variables. When events are sent to PagerDuty, they are transformed into Common Event Format. Previously, Event Rules could only replace Summary or add a Deduplication Key, and these abilities were limited to a direct mapping from one field as a full replacement. Now, this ability is extended to replace any CEF field or even create new key values within the Custom Details object.
Dynamic Field Enrichment & Extraction is available with both Global Event Rules and Service Event Rules. For Service Event Rules, custom variables must extract from CEF fields of the event payload, where for Global Event Rules, custom variables can extract from any field of the event payload.
Global and Service Event Rules are evaluated against the original event. Fields created from Global Event rules via Dynamic Field Enrichment & Extraction cannot be evaluated by Service Event Rules.
Configure Dynamic Field Enrichment & Extraction
-
In the Customize Event Fields section of your event rule, click Add Variable to the right of Define Custom Variable. Enter a short descriptive Name to represent the value that you will use later on to define a CEF field. Then enter a Value for the variable and the origin Source. Values are defined using valid RE2 regular expression syntax, and you can add as much complexity as you want with Regex to customize what part of each field you capture into a variable. You may also use Sample Events on the right as a reference for variables by clicking Show Details. You may add multiple variables by clicking Add Variable in the upper right of the Customize Event Fields panel, or delete variables by clicking the trashcan icon directly to the right of the variable.
-
Next, under Replace Event Field, select your preferred Common Event Field (CEF) from the left dropdown. Select whether you would like to use Regex or a Template from the right hand dropdown. If you are using a text Template, you can reference a variable defined in step 1 using
{{ }}
brackets (e.g.{{class}}
) in the Value field. If you are using Regex, enter a Value and the origin Source.
![d88fb46-DFEE_Alert_Logs.png Alert log](https://files.readme.io/0ab9e22-d88fb46-DFEE_Alert_Logs.png)
Alert log
Set a Deduplication Key with Event Rules
The dedup_key
field is used to merge events into a single alert. Events with the same dedup_key
can update the status of the alert they are automatically merged into.
To set a deduplication key:
- In the Customize Event Fields section of your event rule, under Define Custom Variable, create a variable from event fields by entering a Name, Value and the Source. You may use Sample Events on the right to create these variables.
- Next, under Replace Event Field, in the Event Field (CEF) field select Dedup Key from the dropdown. Select whether you would like to use Regex or a Template from the right hand dropdown. If you are using a Template, you can reference a variable defined in step 1 using
{{variable name here}}
brackets as the Value. If you are using Regex, enter a Value and the Source.
Add or Replace the Summary Field with Event Rules
The summary
field is used to create the incident title. Titles should be descriptive and tell responders what is happening.
To set a summary field:
- In the Customize Event Fields section of your event rule, under Define Custom Variable, create a variable from event fields by entering a Name, Value and the Source. You may use Sample Events on the right to create these variables.
- Next, under Replace Event Field, in the Event Field (CEF) field select Summary from the dropdown. Select whether you would like to use Regex or a Template from the right hand dropdown. If you are using a Template, you can reference a variable defined in step 1 using
{{variable name here}}
as the Value. If you are using Regex, enter a Value and the Source.
Using Dynamic Field Enrichment & Extraction for Email Events
Dynamic Field Enrichment & Extraction capabilities are available only for email events sent to Global Event Rules. Users can extract from the email event using regex matching into multiple variables. Currently, only Summary and Dedup_Key Common Event Fields can be replaced for email events.
Service Event Rules do not apply to emails because they aren’t CEF and therefore, dynamic field enrichment & extraction capabilities are not available for email service integrations.
Enhancing Alert Grouping Capabilities
Dynamic Field Enrichment & Extraction can be powerfully leveraged for improving the accuracy for Intelligent Alert Grouping and Content Based Alert Grouping. This capability can add and replace key information from the raw event payload to create more contextualized alerts, allowing for more flexible and accurate alert grouping options.
Step 2: Do these things
Select which actions you would like this rule to take when an event meets the condition(s) set in Step 1.
![rulesets-actions.png Do these things](https://files.readme.io/0d07741-rulesets-actions.png)
Do these things
- Select whether you would like to Create an incident or Suppress Alert (Note: Alert Suppression is only available with our PagerDuty AIOps add-on. Please contact our Sales Team if you would like to purchase this product.):
-
Create an incident on a service (Global Rulesets)/Create an incident (Service Event Rules): On the right side of the screen, you will see the Once your conditions are met, what would you like to happen? section appear. Indicate your preferred settings in the following sections:
- Route to a Service (Required for global rulesets only): In this dropdown, select the service where events that satisfy your conditions should be routed to.
- Actions (Optional): Depending on your plan and add-on packages, you may have additional optional incident actions to choose from, such as creating an incident immediately or with trigger thresholds, setting severity, setting priority, adding notes to incidents, setting a deduplication key and/or adding/replacing the description field.
Once you have configured your desired actions, proceed to Step 3.
-
Suppress Alert (do not create an incident) (Note: Only available with our PagerDuty AIOps add-on): Suppressing alerts means that alerts created by events which match your specified conditions will not create incidents or notify responders. These alerts are viewable in the Alerts table as suppressed. You have the option to choose from the following actions:
-
Route to a service (Optional for global rulesets): To route the suppressed alert to a service without creating an incident, check this checkbox and select your preferred service from the dropdown.
-
Actions (Optional): Depending on your plan and add-on packages, you may have additional optional actions to choose from, such as setting severity, setting priority, setting a deduplication key and/or adding/replacing the description field.
Once you have configured your desired actions, proceed to Step 3.
Event Rule Actions
In order to set an event rule to perform custom actions on incidents or suppressed alerts that meet the rule’s conditions, you must first follow the instructions in Step 1. Once completed, you will find these optional actions on the right side of the screen under Once your conditions are met, what would you like to happen?.
![rulesets-event-rule-actions.png Once your conditions are met, what would you like to happen?](https://files.readme.io/1d9c271-rulesets-event-rule-actions.png)
Once your conditions are met, what would you like to happen?
Paused Incident Notifications
As a feature of Rulesets, paused incident notifications allows rules to be set to create alerts, but pause incident creation and notifications for a predefined amount of time. During the pause period, new alerts will be viewable in the Alerts table with a Suspended status. These suspended alerts have new actions, allowing users to Trigger an incident from the alert during the pause period, or Resolve the alert, preventing the incident and its notifications from being created altogether.
Availability
- The Paused Incident Notifications functionality is part of our PagerDuty AIOps product, which is purchased separately from the core PagerDuty platform. Please contact our Sales Team if you would like to purchase this product.
- Paused Incident Notifications is only available on incidents generated by our Events API, and does not work with those generated from our REST API.
To configure paused incident notifications:
- In Step 2: Do These Things, select Create an incident on a service. In the right hand panel in Incident Creation section, select Create alert but pause notifications as an event rule action. Enter the amount of time you would like to pass before an incident and notifications are created in the Suspend alert for _ second(s) before triggering incident field.
![paused-incident-notifications-rule-actions.png Create alert but pause notifications](https://files.readme.io/c8e8c2c-paused-incident-notifications-rule-actions.png)
Create alert but pause notifications
- Click Save Rule.
View and Take Action on Paused Alerts
To view alerts in a paused state, navigate to the Alerts table in the upper menu of the web app. Alerts with paused incident notifications will have a Suspended status:
![delayed-incident-notif-suspended-alert.png Suspended alert](https://files.readme.io/0057fea-delayed-incident-notif-suspended-alert.png)
Suspended alert
To take action on a paused alert:
- Click the incident title to view its details page.
- On the right hand side, click Trigger to trigger an incident and send notifications, or Resolve if you would like to resolve the alert without triggering an incident/notifications. You will see a green dialog that confirms that the alert has been triggered or resolved.
![delayed-incident-notif-trigger-resolve.png Alert details](https://files.readme.io/43399fa-delayed-incident-notif-trigger-resolve.png)
Alert details
The pause period and action taken will also appear in the Alert Log timeline, accessible by clicking an alert's title to view the detail page:
![delayed-incident-notif-alert-log-entry.png Alert log](https://files.readme.io/1cd7572-delayed-incident-notif-alert-log-entry.png)
Alert log
Suppress, but Create Triggering Thresholds with Event Rules
Receive PagerDuty notifications only when your customized alert conditions breach your specified limits. In this way, responders effectively reduce alert noise without missing critical issues.
Availability
The Threshold Alerts functionality is part of our PagerDuty AIOps product, which is purchased separately from the core PagerDuty platform. Please contact our Sales Team if you would like to purchase this product.
To set a threshold, enter the number value of alerts that should be suppressed within a number value of minutes, hours or days before an incident is created. For example, if we set the following threshold:
![rulesets-threshold-alerts.png Suppress, but create with triggering thresholds](https://files.readme.io/b82fe02-rulesets-threshold-alerts.png)
Then if 6 or more alerts come in within a 10 minute time window, the threshold will be exceeded and an incident will be created.
Set Severity with Event Rules
For events that do not have a severity level (such as emails), users can define one, or a new severity level can be set. All current pricing plans include incident severity.
To set severity, under Alert Behavior, check the Set Severity checkbox and select info, error, warning or critical from the dropdown.
Set Incident Urgency with Event Rules through Dynamic Notifications
Severity can be used for Dynamic Notifications, which are defined by the service settings. To use Dynamic Notifications, the events must be routed to a service that sets Dynamic Notifications based on severity levels.
With severity controlled via event rules and the service configured to use dynamic notification, you can control the incident urgency.
Set Priority with Event Rules
Incident priority allows the classification of incidents based on a level of prioritization. Incident priority must be enabled on your account before it can be set with event rules.
To set priority, under Alert Behavior, check the Set Priority checkbox and select your desired priority level from the dropdown.
Add a Note to an Incident with Event Rules
Notes can be used to help responders resolve incidents quicker by including information or links related to the system that the event comes from.
Notes
- Adding notes via event rules is available with our PagerDuty AIOps product, which is purchased separately from the core PagerDuty platform. Please contact our Sales Team if you would like to purchase this product.
- The note is added when an incident is created. If a new alert is deduplicated into the incident, then the deduplicated alert will not add an additional note nor change the original.
To add a note with event rules, under Additional Context, enter the text of the note that you would like to be added to an incident that meets your rule’s criteria.
Set a Custom Trigger / Resolve Action with Event Rules
If you would like to automatically trigger an alert or resolve an alert based on your event rule conditions, under Enrichment & Transformations, check Set a custom trigger / resolve action and select Always trigger an alert or Always resolve an alert. All pricing plans include incident custom trigger/resolve actions with event rules.
Automated Triggers
Automated Triggers allow response teams to easily define a dynamic webhook using event rules, sending a custom payload to a specified endpoint. This enables responders to automate actions like restarting a server, clearing logs, and reverting bad deploys. The payload can be set using a combination of several predefined variables, created using Dynamic Field Enrichment and Extraction. Automated Triggers can be created using both global event rules and service level event rules.
Availability
Automated Triggers are a feature of our PagerDuty AIOps product, which is purchased separately from the core PagerDuty platform. Please contact our Sales Team if you would like to purchase this product.
Configure Automated Triggers
- In the Do these things section of the event rule creation process, select the Create an incident on a service option and select the Advanced tab on the right.
- Under Custom Action (Webhook), check the Enable Custom Action Button checkbox and fill out the details by specifying a Button Name, the API Endpoint for the payload, and the Parameters to include in the payload.
- Optional: Button names and parameters can take advantage of the variables created using the Dynamic Field Enrichment and Extraction feature in the preceding Customize Event Fields section to further enrich the custom action.
- Optional: Check the Trigger automatically on incident creation checkbox to send this webhook automatically when an incident is created.
- Continue configuring the event rule and click Save Rule.
![event-triggered-webhooks-config.png Rule configuration](https://files.readme.io/20a0879-event-triggered-webhooks-config.png)
Rule configuration
Use Automated Triggers
Once the rule is created, a custom push-button action will be available the next time the rule is applied. Users can find the Automated Triggers action in the More... dropdown on the incident details page, and activity related to the custom rule action will be visible in the incident timeline.
There is a limit of one Automated Trigger per rule. If Intelligent Alert Grouping is enabled on a Service, each additional alert grouped to the incident can add an additional unique action button, up to a limit of 5 buttons.
![event-triggered-webhooks-custom-incident-action.png Automated trigger](https://files.readme.io/748ee5e-event-triggered-webhooks-custom-incident-action.png)
Automated trigger
A modal will appear when the Automated Triggers action is clicked, and users can confirm the details of the webhook payload (fields were set in the rule configuration phase). For all webhooks, the PagerDuty incident ID will be included as a mandatory field. If the action is triggered, a response log will be shown in the incident timeline to let users know if the webhook was successfully received.
![event-triggered-webhooks-trigger-action-modal.png Trigger automated trigger](https://files.readme.io/6e9a933-event-triggered-webhooks-trigger-action-modal.png)
Trigger automated trigger
![event-triggered-webhooks-incident-timeline.png Incident timeline](https://files.readme.io/a555ab4-event-triggered-webhooks-incident-timeline.png)
Incident timeline
Step 3: At these times
In this step, select when you would like this rule to be active. Once you have made your selection, proceed to Step 4.
![rulesets-when-should-rule-active.png Rule active at these times](https://files.readme.io/68592bf-rulesets-when-should-rule-active.png)
Rule active at these times
Always
If you select Always, this rule will always run (24/7/365) when your conditions are met. Click Save Rule to create or update your event rule and proceed to Step 4.
Scheduled Event Rules
You can gain a greater degree of control over your event rules by detailing a single specific time in the future in which they will be active. This can be useful for testing rules and for planned maintenance. Users can then specify what actions are to be performed within this specific time window. For instance, if you schedule a rule to be active during a single hour window, and the action set was to suppress alerts, alerts would be suppressed during that hour, and once it ended they would begin generating incidents again.
Availability
Scheduled Rules functionality is available with our Business, Enterprise for Incident Management and Digital Operations (Legacy) plans, or as part of our PagerDuty AIOps product, which is purchased separately from the core PagerDuty platform. Please contact our Sales Team if you would like to purchase this product.
If you select At a Scheduled Time, you will be prompted to select a Starting Date and Time and an Ending Date and Time. Click Save Rule to create or update your event rule and proceed to Step 4.
Recurring Event Rules
If there are specific hours of the day or day(s) within a week when you would like an event to follow a particular rule, you can set its activity on an automatic, weekly recurring schedule. This feature is specific to each individual event rule and you can make additional changes, such as changing severity/priority, based on time of day.
Availability
The Recurring Event Rules functionality is available with our Business, Enterprise for Incident Management and Digital Operations (Legacy) plans, or as part of our PagerDuty AIOps product, which is purchased separately from the core PagerDuty platform. Please contact our Sales Team if you would like to purchase this product.
To set a recurring schedule for an event rule, select On a Recurring Schedule and enter a Starting Time, an Ending Time and select the Days of Week in which you would like the rule to be active. Click Save Rule to create or update your event rule and proceed to Step 4.
Step 4: Send Test Events
Unsupported Integration Types
The following types of integrations cannot use ruleset integration keys:
- Integrations that connect via account authorization are not supported.
- Custom Event Transformer integrations are not supported.
- Integrations that use the REST API are not supported.
Send Events to a Global Ruleset
Each global ruleset will have an Incoming Event Source page that will display your routing Integration Key, as well as an HTTP endpoint for API and Email address that you can use for routing.
To access the Incoming Event Source page:
Free, Professional, Starter (legacy) and Team (legacy) plans: Navigate to AIOps Event Rules.
Business, Enterprise for Incident Management and Digital Operations (Legacy) Plans: Navigate to AIOps Event Rules click the name of your desired ruleset to access its individual Incoming Event Source page.
Use this information to send events based on your preferred integration method:
- Use the Integration Key to integrate with your preferred tool.
- Use the Integration Key to send an event through one of the Events APIs.
- If using the V2 events API, use your routing Integration Key as your
routing_key
value in your event JSON. - If using the V1 events API, use your routing Integration Key as your
service_key
value in your event JSON.
- If using the V2 events API, use your routing Integration Key as your
- Use the Email address to send email events.
Note
Integration Keys are case-sensitive.
This will allow you to test how your event rules will modify them and take the actions configured in previous steps. Your event rule is now complete! If you would like to create more event rules, repeat steps 1-4 as needed.
![rulesets-incoming-event-source.png Incoming event source](https://files.readme.io/5422079-rulesets-incoming-event-source.png)
Incoming event source
Send Events to Service Event Rules
Add your preferred tool integration or a Custom Event Transformer integration on the service where you configured the above rule(s). Once you have added the integration to your service, you will be directed to the Integrations tab, where you can access its Integration Key:
![copy-integration-key.png Integration key](https://files.readme.io/c1154f2-copy-integration-key.png)
Integration key
Use this Integration Key to complete your integration and send test events to your service. This will allow you to test how your event rules will modify them and take the actions configured in previous steps. Your event rule is now complete! If you would like to create more event rules, repeat steps 1-4 as needed.
Edit Rulesets and Event Rules
Edit a Global Ruleset's Event Rules
Free, Professional, Starter (legacy) and Team (legacy) plans:
- Navigate to AIOps Event Rules.
- If you would like to add a new event rule: Click New Event Rule to begin creating event rules.
If you would like to edit an existing event rule: Click the to the right of your desired rule and click Edit Rule. You may edit the rule’s conditions, actions and times when it should be active. - Click Save Rule.
Business, Enterprise for Incident Management and Digital Operations (Legacy) plans:
- Navigate to AIOps Event Rules click the desired ruleset’s name.
- If you would like to add a new event rule: Click New Event Rule to begin creating event rules.
If you would like to edit an existing event rule: Click the to the right of your desired rule and click Edit Rule. You may edit the rule’s conditions, actions and times when it should be active. - Click Save Rule.
Edit a Global Ruleset’s Details
Business, Enterprise for Incident Management and Digital Operations (Legacy) plans:
- Navigate to AIOps Event Rules click the desired ruleset’s name Edit Ruleset. Note: You cannot edit details of your Default Global Ruleset, you can only edit the additional rulesets that you have created.
- Edit the ruleset’s Name and/or Team details and click Save Ruleset. Note: If you are adding the ruleset to a team, this will grant Team Managers edit-access to the ruleset. If you are removing the ruleset from a Team, this will revoke edit-access for Team Managers and it will behave as an additional global ruleset.
Edit Service Event Rules
- Navigate to Services Service Directory click the name of your desired service click the Settings tab.
- If you would like to add a new event rule: If you have not configured any rules for this service, click Add One. If you have configured other rules for this service, click Manage Service Event Rules and then click the +New Event Rule to begin creating event rules.
If you would like to edit an existing event rule: Click Manage Service Event Rules, click the to the right of the desired rule and select Edit Rule. You may edit the rule’s conditions, actions and times when it should be active. - Click Save Rule.
Duplicate Event Rules
If you would like to duplicate and create a new rule within a ruleset, click the dropdown next to the rule and select Duplicate Rule. A new event rule with the same settings will be automatically created beneath the duplicated event rule. This functionality works for both Global Rulesets and Service Event Rules.
Disable Event Rules
If you would like to pause an event rule’s activity, you can disable and re-enable it at a later time. This feature allows you to manually initiate integration-specific disablement during maintenance or testing on a tool.
To temporarily disable a rule:
Free, Professional, Starter (legacy) and Team (legacy) plans:
- Navigate to AIOps Event Rules, then click the dropdown next to the rule and click Disable Rule.
- While the rule is disabled, you will see a
Disabled pill marker in the Matching conditions section of the rule.
Business, Enterprise for Incident Management and Digital Operations (Legacy) Plans:
- Navigate to AIOps Event Rules click the name of your desired ruleset, then click the dropdown next to the rule and click Disable Rule.
- While the rule is disabled, you will see a
Disabled pill marker in the Matching conditions section of the rule.
If you would like to enable the rule again, click the dropdown and click Enable Rule.
Rulesets API
You may also create, update, order and delete rules via our Rulesets API for Global Event Rules and our Service Event Rules API for Service Event Rules.
Global Event Rules API v1 (Legacy)
Our legacy Global Event Rules API v1 is no longer receiving new features, and it will be fully deprecated at the end of 2020. We recommend replacing any integrations using the Global Event Rules API v1 with new integrations using our new Rulesets API to prepare for this change and to access new features.
Troubleshooting Event Rules
If None of Your Rules Are Working
This may be because you have not set up the integration key correctly in your monitoring tool. For event rules to work, you must use the integration key for the event rules you have configured.
Service Event Rules (All Plans): You will find your integration key by navigating to Services Service Directory click the name of the service that has your desired event rule configured select the Integrations tab. You will see your integration key to the right of the integration’s name.
Free, Professional, Starter (legacy) and Team (legacy) plans, Global Rulesets: You will find this key under AIOps Event Rules and it will be listed under Incoming Event Source.
Business and Global Operations plans, Global Rulesets: You will find this key under AIOps Event Rules click your desired ruleset name and it will be listed under Incoming Event Source.
If you are sending events through the Events API, use this integration key in the service_key
field (Events API V1) or the routing_key
field (Events API V2).
If One of Your Rules Is Not Working
This may be because you have not entered the field name correctly in the rule conditions. The field name is based on the data that is sent to PagerDuty, and is not necessarily the same as the field name visible in the alert or incident in PagerDuty. Refer to the data in your monitoring tool or integration for the correct field name.
For nested fields, format the field name with periods separating each level. For example, field_group.sub_field.another_sub_field
.
If you have confirmed your field names are set up correctly, you may have a rule that is too broad that is ordered before the rule that is not working. Re-order your rules so that more specific rules are listed first.
You may also want to check for rules with negative conditions ("does not contain" or "does not equal"), as these rules can match many events that do not contain the specified field.
For nested fields, format the field name with periods separating each level of the object path. For example, field_group.sub_field.another_sub_field
.
Switch to Service Orchestrations
Service Orchestrations provide updated functionality to Service Event Rules. If you would like to switch from Service Event Rules to Service Orchestrations, please read our Event Orchestration documentation.
FAQ
Are email integration filters/rules, or custom event transformers supported?
This depends on where you are creating event rules:
- Global rulesets: Email filters/rules are supported. Custom event transformers are not supported, however, global event rules will autodetect and transform events sent to a global endpoint.
- Service Event Rules: Only basic email filters are supported, while email rules are not supported. Custom event transformers are supported.
Are events transformed into our Common Event Format (CEF)?
- Events that are routed are transformed into CEF.
- Global Rulesets: Custom Event Transforms (CETs) are not supported.
- Service Event Rules: Custom Event Transforms (CETs) are supported.
What happens to events that don’t match any rules?
- Global Rulesets: Any events that are sent through event rules and do not match any rules are suppressed, and you can find them in the alerts table. You can change this default behavior by editing the catch-all rule at the bottom of your rules list.
- Service Event Rules: Any events that are sent through service event rules and do not match any will turn into incidents and will not be suppressed.
Is there a way to bulk add/delete/edit global event rules?
Yes. You can leverage the Rulesets API to allow this functionality.
Once an event is routed to a service from my global event rules, will the event rules I set up on my service be respected as well?
Yes. Service event rules can be used in conjunction with your global event rules this way.
Can I send a second event with the same dedup_key
and have it follow different actions?
dedup_key
and have it follow different actions?If a second event with the same incident key is passed and de-duplicated into the open incident via global, team, or service event rules, the actions are not followed (ie. raising priority, adding a note) except for when severity is raised.
What happens if I do not create a deduplication key for an email sent to a ruleset?
As emails are processed, the email subject will be used as a default deduplication key. In other words, emails with the same subject line will automatically be deduplicated. To change this behavior, add a custom deduplication key with an event rule action.
Is there a difference between the types of events processed by global event rules vs. service event rules?
Yes, global event rules are written against raw/untransformed events, whereas service event rules are written against transformed/PD-CEF events.
Why wasn't my event suppressed by a rule?
If there is an existing unresolved alert with the same dedup key as the incoming event, this event will be deduped into that alert and the "suppress" action in the event rules will be ignored.
Updated 22 days ago