ServiceNow is a powerful platform-as-a-service, which offers advanced automation and process workflow for the enterprise environment. We offer the following resources to assist with installing, customizing and troubleshooting your ServiceNow integration:

• Installation: For installation instructions, please refer to the ServiceNow v7 Provisioning.
• Advanced Configuration: For more in-depth information about customizing how the ServiceNow integration behaves, please refer to Advanced ServiceNow v7 Configuration.
• FAQ: Answers to frequently asked questions are found in ServiceNow v7 Frequently Asked Questions.
• Troubleshooting: For a guide to addressing some common issues, see the ServiceNow v7 Troubleshooting Guide.

This article is an overview of the components and concepts that drive the ServiceNow + PagerDuty integration, and will give you a deeper understanding of how the integration works at a fundamental level.

🚧

Unsupported Changes

If you modify any core components of the integration, application upgrades may not work as expected. Customizations are generally not supported by PagerDuty.

Introduction

With the ServiceNow + PagerDuty integration, you can leverage PagerDuty's robust event management, on-call scheduling, escalation and notification functionality and keep PagerDuty incidents in sync with your ServiceNow tickets.

In ServiceNow Enterprise, the source code for many of the integrated PagerDuty app’s components is not only visible, but can also be modified. All of the source code that performs the actions underpinning the integration is written in JavaScript, and can be viewed by going to the Advanced section or tab of any component's view. The application's components can be accessed by going to PagerDuty Configuration Configuration files. There you will find a page with the following tabs:

• Business Rules
• Script Actions
• Script Includes
• UI Actions
• Processors
• UI Pages
• Macros
• Application Files

Terminology

Assignment versus Escalation

The term assignment has a slightly different meaning in PagerDuty than it does in ServiceNow. When an incident is assigned to a user in PagerDuty, it is escalated to them and they are the agents who will be notified. When an incident is first triggered, it is considered to be assigned to the first targets in the escalation policy until escalated. However, it will not assign the incident to anyone in ServiceNow until someone acknowledges it, in which case the corresponding ServiceNow incident will be assigned to them.

When an incident in ServiceNow is assigned to a user, that similarly escalates the incident to that user in PagerDuty.

The integration will also assign the ServiceNow incident to the user who acknowledged it in PagerDuty, since it is assumed that the user who acknowledged it is the one who will be working on it.

Provisioned versus Linked

The term provisioned refers to when the item in ServiceNow has an analogous record in PagerDuty, identified by one or more metadata fields, referred to as the integration fields.

An incident is considered linked if it is associated with a PagerDuty incident, in which case its status, work notes and other information pertaining to the incident can be synchronized between systems.

Core Components

Most of the tasks performed to synchronize data between PagerDuty and ServiceNow are executed in the following application files, under Script Includes:

• PagerDuty: Utilities for obtaining PagerDuty metadata, and functions for performing actions on incidents, i.e., assigning to a user, synchronizing work notes, and triggering new incidents.
• PagerDutyInbound: Functions that handle receipt and processing of webhooks from PagerDuty
• PagerDutyProvisioning: Handles the creation of objects in PagerDuty from ServiceNow, or in the case of users, finding existing ones that match analogous ones in ServiceNow.
• PagerDuty_REST: Core functions for making REST API requests, and in version 3.2.1 and earlier, Events API requests.

Each of the above defines a class constructor named after the script include. Methods of these classes are called throughout the rest of the application.

Another important file, which handles the importing of webhook data into ServiceNow incidents, is the PagerDuty Webhook Import Table Transform Map, which can be found under the Application Files tab.

Troubleshooting With Verbose Logging

To get the most detailed insight into script behavior, in ServiceNow go PagerDuty Configuration PagerDuty Settings and set Logging verbosity level to debug. With this setting selected, each call to gs.debug in the script source will show up in the application-scope logs, which can be viewed under PagerDuty Support Logs.

Bidirectional Synchronization

The integration uses a number of features to keep objects between PagerDuty and ServiceNow in sync:

• Provisioning: Linking Objects Between Systems
• Business Rules: ServiceNow to PagerDuty
• Webhook Import: PagerDuty to ServiceNow
• Work Note Synchronization

Provisioning: Linking Objects Between Systems

For any user or assignment group or Configuration Item (CI) in ServiceNow to be recognized by the integration, it must be provisioned.

Objects can be provisioned by clicking Provision PagerDuty Service or Provision PagerDuty User under Related Links in the view of the user, group or Configuration Item, or by selecting several from the list view and choosing the option to provision them under the menu actions on selected rows.

This process will make the necessary REST API calls in the PagerDutyProvisioning object. Note the ID of the created (or existing) records, and save them in one of the following fields, depending on the object's type:

• User
• Assignment Group
• Configuration Item

User

Assignment Group

Configuration Item

Auto-Provision Users

Since version 3.2.1, users in PagerDuty can optionally be provisioned as ServiceNow users, or, in cases where they provision an assignment group, the ServiceNow user's record will be linked to a PagerDuty user if one exists with the same email address. The user performing the provisioning is also auto-provisioned and then automatically added as the initial member of the newly-generated escalation policy in PagerDuty.

Special Integration Field Usage

For the bidirectional integration to function properly, the following is how the fields are used:

• Service: When a ServiceNow incident triggers a PagerDuty incident, the assignment group’s Service is used to tell which PagerDuty service to trigger it on.
• Escalation: When an incident in PagerDuty opens a ServiceNow incident via webhook, the integration uses the assignment group’s Escalation field.

We do not recommend associating multiple ServiceNow groups with the same escalation policy. This is because if the webhook transform map finds more than one assignment group with the same escalation policy ID, it will assign the ServiceNow incident to the first assignment group whose x_pd_integration_pagerduty_escalation column value matches the escalation policy ID in the webhook.

Conversely, it is possible to manually associate an assignment group with multiple services which all use the same escalation policy and have the integration work smoothly. This is because it identifies the assignment group by escalation policy ID, and there's one escalation policy for all the services, so the service ID has no bearing in this part of the process (see Webhook Import: PagerDuty to ServiceNow).

In the opposite direction, ServiceNow incidents can only trigger incidents in PagerDuty on one service, i.e., the service whose ID is stored in the assignment group's PagerDuty Service field.

One can find any PagerDuty object’s ID by going to its detailed view page in the PagerDuty web app and looking at the URL in your web browser’s address bar. The ID should be a string of uppercase alphanumeric characters beginning with P (e.g., PXXXXXX), and will be near the end of the URL. Using this method, you can manually provision objects by setting the appropriate fields in the corresponding ServiceNow objects’ records.

Business Rules: ServiceNow to PagerDuty

All aspects of the integration that concern any flow of data from ServiceNow to PagerDuty are handled by business rules. Business rules begin when an event, such as an insertion, update or deletion, takes place on an incident. If the conditions of a business rule are met, the rule and its associated script are run, which invokes a transaction with the PagerDuty API to perform the necessary actions in PagerDuty.

For more information about ServiceNow business rules, we recommend the following resources, provided by ServiceNow:

• Business Rules (developer documentation)
• Business rules (product documentation)

How PagerDuty Incidents Trigger

As an example, the event named x_pd_integration.trigger_incident, handled by the PagerDuty Trigger Incident script action, is raised by one of the following business rules:

• PD Trigger PagerDuty Incident: By default, this runs when ServiceNow creates an incident that has no corresponding PagerDuty incident, and it is assigned to a PagerDuty-linked group, has Priority 1 or 2, or an existing incident has its priority updated to 1 or 2. You can adjust this behavior to include a wider range of priorities: When is an incident sent to PagerDuty?.
• PD Group Assign Trigger New Incident: An existing incident that was previously tied to a PagerDuty incident, but that was "unlinked" after assigning it to an un-provisioned assignment group (per the business rule PD Unlink if not PagerDuty group).

The setting Choose ServiceNow to PagerDuty mapping dynamically tests whether the assignment group and/or Configuration Item are provisioned. For more information, see the section “When is an incident sent to PagerDuty?”.

To recap, the integration selects the PagerDuty service to trigger an incident on based on the value of the ServiceNow incident assignment group’s PagerDuty Service field. If the ServiceNow incident is associated with a Configuration Item, the integration uses the value specified in the field Default PagerDuty Service ID to determine which PagerDuty service to trigger an incident on.

You can test this in the PagerDuty Script Include, in the function PagerDuty.triggerIncident.

How PagerDuty Incidents Resolve

The following business rules resolve PagerDuty incidents from ServiceNow:

• PD Resolve PagerDuty Incident: The incident's State field must change to 6 or 7 (resolved/closed) and the incident must be linked. This raises the x_pd_integration.resolve_incident event, handled by the PagerDuty Resolve Incident script action. The call to PagerDuty.resolveIncident resolves the incident.
• PD Unlink if not PagerDuty group: If a linked incident is assigned to a non-provisioned assignment group, it is assumed that it will be handled outside of the PagerDuty incident response process. With that in mind, the PagerDuty incident will be resolved. This process happens as follows:
  1. The PagerDuty Incident field on the ServiceNow incident record is set to ---unlinked---.
  2. This activates the business rule PD Resolve Incident on Unlink.

PD Resolve Incident on Unlink raises the x_pd_integration.resolve_on_unreference event, handled by the PagerDuty Resolve on Unreference script action. The PagerDuty Resolve on Unreference calls PagerDuty.resolveOnUnreference to resolve and post a worknote to the PagerDuty incident, indicating its resolve reason.

Reassignment

Incidents in ServiceNow, when assigned to a user or assignment group, may trigger one of the following business rules:

• PD User Assign: If a provisioned user is assigned a linked incident, this escalates the PagerDuty incident to the user in PagerDuty, by raising the x_pd_integration.assign_incident_user event, which is handled by the PagerDuty User Assign Incident script action. This action calls the function PagerDuty.assignIncidentToUser.
• PD Group Policy Assign: If a linked incident’s assignment group changes, and the new assignment group is provisioned, the corresponding PagerDuty incident will be assigned to the escalation policy in PagerDuty specified by the assignment group's PagerDuty Escalation field. This will restart escalation and notify the new assignees in the target escalation policy.

Webhook Import: PagerDuty to ServiceNow

Real-time incident updates in PagerDuty are communicated to ServiceNow through extensions on each PagerDuty service that correspond to a provisioned assignment group in ServiceNow. When a group is initially provisioned, an extension will also be created on the service. In version 3.5 and later, an extension on the service will be provisioned automatically if one does not already exist, whenever a new incident is created.

The extensions, also known as webhooks, specify URLs in your ServiceNow instance, to which JSON-encoded data is sent via HTTPS/POST requests.

Precedence and Order of Operations in Webhook Import

Throughout the webhook import process, automation and record creation in ServiceNow happens in the following order:

1. Run the default import actions
2. Run dynamic field mapping rules
3. Run the custom script, if any

Initial Receiving and Importing of the Webhook

The first point of entry into the ServiceNow application is the Webhook Post Scripted REST Resource, found in the Application Files tab. This component handles the receipt of the payload, validation of the webhook key/endpoint path, and generating the response. The PagerDutyInbound Script Include processes the data, via a call to PagerDutyInbound.processRestWebhook.

PagerDutyInbound handles data saving. It is passed an array, messages, containing one or more webhook event objects as described in Webhooks Overview.

Finally, in the function PagerDutyInbound._processData, an import record is created for each webhook event in the table x_pd_integration_webhook_import. You can review this table’s data and a record of received webhooks under PagerDuty Imports Webhook Import Rows.

Translating Webhook Imports to ServiceNow Incidents

The PagerDuty Webhook Import Table Transform Map, and other transform maps that operate on other tables in ServiceNow (v5 and later), takes each new record inserted in the webhook import table and translates it to either an insert or an update on the ServiceNow table. Which of these two operations happens depends on whether the import transform is able to find an existing ServiceNow incident record with the same PagerDuty incident ID as the one corresponding to the PagerDuty webhook message.

This behavior is dictated by the Field Maps configuration in the Table Transform Map:

After creating or updating the incident in ServiceNow, the integration will automatically set the assignment group (for trigger,acknowledge, escalate and delegate webhook types). The PagerDuty escalation policy ID in the webhook payload is used to determine the assignment assignment group. In other words, if the assignment group declared in the PagerDuty Escalation field (x_pd_integration_pagerduty_escalation) matches the escalation policy ID in the webhook payload, the incident will be assigned to that group.

In version 5 and later, the PdWebhookTransform script contains the logic for all default operations taken when updating or inserting an incident (e.g., additional fields to populate and workflow logic). Within the local scope, the following variables dictate the actions that will be taken on the target record for each webhook import:

By the time the script completes, the incident’s properties will be stored in the target variable. Changes will be saved after the script exits.

Work Note Synchronization

ServiceNow Work Notes to PagerDuty

Work notes are copied from ServiceNow to PagerDuty through the PD Copy worknote to PagerDuty incident business rule, which triggers the x_pd_integration.post_worknote event, which then invokes the PagerDuty.postIncidentNote function. This runs any time a work note changes on a linked incident.

PagerDuty Incident Notes to ServiceNow Work Notes

Starting in version 5, the integration uses the annotate webhook event type to keep incident notes synchronized between PagerDuty and ServiceNow. When a user adds a note to the incident in PagerDuty, the webhook transform will recognize the event, and add the note as a work note with the suffix (PagerDuty: PAGERDUTY_USER_NAME on TIMESTAMP).

In earlier versions, the process that ran in the opposite direction was initiated by a business rule (in addition to the user-initiated UI action Refresh PagerDuty Notes), which appeared as an action in the context menu for any given linked incident. The business rule PD Update Worknotes from PagerDuty Notes polls for updates, and raises the event x_pd_integration.import_notes (handled by the Import Notes from PagerDuty incident script action). In both cases, the functions PagerDuty.getIncidentNotes and PagerDuty.createNoteImports are used to import and save notes from PagerDuty. ServiceNow incident work notes will appear in PagerDuty with the prefix (from ServiceNow:SERVICENOW_USER_NAME).