ServiceNow Troubleshooting Guide

Identifying and resolving common issues with the ServiceNow integration

Troubleshoot common issues and get answers to frequently asked questions about the ServiceNow v8 integration.

📘

Looking for a Different Version?

The Troubleshooting Guide and FAQs for the v7 integration are also available.

Installation and Configuration

PagerDuty App Does Not Appear in the System Menu

  1. If you’re having trouble finding the PagerDuty app in ServiceNow’s system menu, please check the following first:
    1. If you just installed the app, try a hard refresh in your browser, or log out/in again, as the left menu may need to reload after installation.
    2. Make sure the PagerDuty app has been added to your current instance from the ServiceNow store.
    3. Install the app to your current instance by going to System Applications Applications Downloads tab.
  2. If you’ve completed the steps above and are still unable to see the PagerDuty app, check to see if you have any plugins that could prevent the installation from succeeding.
    1. For example, if you have installed a plugin that encrypts attachments in ServiceNow, we’ve seen cases where it looks like the installation completes when it has not actually done so. If you identify such a plugin, please try deactivating it and attempting installation again.

Installation and Configuration FAQs

How can I tell which version of the app I have installed?

  1. Navigate to System Applications All Available Applications All.
  2. Search for PagerDuty and then click the Version dropdown on the right to see the installed version.

How many accounts/instances can be connected?

This integration supports integrating a single PagerDuty instance with multiple ServiceNow instances. However, you cannot integrate multiple PagerDuty instances with a single ServiceNow instance.

Which configuration files are relevant to the integration?

Please see the configuration files below:

  • Business Rules
  • Script Actions
  • Script Includes
  • UI Actions
  • Processors
  • UI Pages
  • Macros
  • Application Files

Who should I reach out to if I have any questions about upgrading/installing/configuring the integration?

Our Support team can assist customers with basic troubleshooting of some common issues that may occur during the installation process. However, Support cannot help customers plan or implement any upgrades.

Our Professional Services team offers the ServiceNow Application Upgrade Accelerator. To see if this service matches your needs, please contact the PagerDuty Sales team.

Integration Health Check

Cross-scope Access Policy Error During Webhook Async Processing Mode Setup

If you receive an error during the Setup webhook async processing mode step, the workaround is to:

  1. Search on the menu for Scripts - Background.
  2. Select scope Global.
  3. Run the following script code: gs.invalidateCache();
  4. Re-run the Integration Health Check (Select All and search and select Integration Health Check. It will run automatically).

Integration Health Check FAQs

What issues are detected by the integration health check?

The Integration Health Check now consolidates and runs the following tests: Test REST API Connection, Test ServiceNow User Authentication, and Test Default User Settings.

Webhook Health Check

Webhook Health Check FAQs

What are some common issues detected by the webhook health check?

MessageCodeValidationStops validation?Action type
PagerDuty Service that is linked to ServiceNow Entity doesn’t existservice_invalidIf PagerDuty Service ID is valid (exists on PagerDuty)Yes🔴 Manual intervention
ServiceNow Entity linked to a PagerDuty Service has an incorrect Webhook IDwebhook_missingWebhook is missing and can be created or updated with existing Service webhookYes🟡Additional review
ServiceNow Entity linked to a PagerDuty Service has an incorrect Webhook ID. There are multiple Webhooks already createdwebhook_invalid_multipleInvalid webhook (missing or mismatched) and there are more than one webhook candidates at the ServiceYes🔴 Manual intervention
ServiceNow Entity linked to a PagerDuty Service has a mismatched Webhook IDwebhook_service_mismatchWebhook doesn’t match the ServiceNow Entity Service IDYes🟢 Auto remediation
ServiceNow Entity linked to a PagerDuty Service has a mismatched Webhook ID. There are multiple Webhooks already createdwebhook_service_mismatch_multipleWebhook doesn’t match the ServiceNow Entity Service ID and there are multiple Webhooks createdYes🔴 Manual intervention
Webhook duplicated for a Service linked to a ServiceNow Entitywebhook_duplicatedIf more than one WebhookNo🟡Additional review
Webhook configuration (sync_mode & task_type) params do not match with ServiceNow entity configurationwebhook_config_mismatchIf Webhook ServiceNow configuration params (sync_mode, task_type) match
OR
If Webhook (V3) is properly configured for sync_mode (triggered event)
No🟢 Auto remediation
Webhook is temporarily disabledwebhook_disabledIf Webhook is temporarily disabled (delivery failure)No🟢 Auto remediation
Webhook is not associated with any ServiceNow Entitywebhook_not_associatedIf Webhook is associated with ServiceNow entity
Note: At least one is expected for the default service x_pd_integration.default_service
No🟡Additional review

Which Webhook Health Check run is being referenced in the results?

The Webhook Health Check results are from the last run.

Provisioning

When Assignment Group or Configuration Item Provisioning Fails

  • Confirm that the object’s name does not conflict with an existing service or escalation policy in PagerDuty.
  • Check that your PagerDuty account has user licenses available if there are new users being provisioned.

Provisioning FAQs

Are ServiceNow objects and PagerDuty objects mapped 1:1?

The ServiceNow integration provides 1:1 mapping out-of-the-box.

Escalation:

  • When an PagerDuty incident opens a ServiceNow Incident via webhook, the integration uses the Assignment Group’s Escalation field.
    • We do not recommend associating multiple ServiceNow groups with the same escalation policy. If there is more than one assignment group with the same escalation policy ID, the ServiceNow incident is assigned to the first assignment group whose x_pd_integration_pagerduty_escalation column value matches the escalation policy ID in the webhook

Service:

When a ServiceNow Incident triggers a PagerDuty incident:

  • With the Assignment Groups map to PagerDuty setting enabled: The Assignment Group’s Service field is used to tell PagerDuty which service to trigger it on.
  • With the Configuration Items and Assignment Groups map to PagerDuty setting enabled: The service is fetched/mapped from the Configuration Item.

ServiceNow Incidents can only trigger PagerDuty incidents on one service, i.e., the service whose ID is stored in the Assignment Group's PagerDuty Service field.

What is the difference between provisioned versus linked?

The term provisioned refers to when an item in ServiceNow has an analogous record in PagerDuty, identified by one or more metadata fields (referred to as the integration fields).

A ServiceNow Incident is considered linked if it is associated with a PagerDuty incident. In which case its status, work notes, and other information pertaining to the incident can be synchronized between systems.

Data from PagerDuty to ServiceNow

PagerDuty Incidents are Not Creating/Updating Tickets in ServiceNow

Run the Integration Health Check

To run the Integration Health Check in ServiceNow, select All and search and select Integration Health Check. The health check will run automatically, and once it is complete, the results will be displayed on this page.

Run the Webhooks Health Check

Select All and search and select Webhooks Health Check. Click Run health check or Re-run health check to identify if there are any inconsistencies which may be causing this behavior.

Check if the Webhook Created a Webhook Import Row Record

In ServiceNow, select All and search and select Webhook Import Rows. Look for records with a creation time that coincides with when the PagerDuty incident was triggered, acknowledged or resolved.

Check Requirements and Restrictions in your ServiceNow Instance

  1. Check to see if there are any required fields in the ServiceNow incident form that aren’t auto-populated via Inbound Field Rules: If you can see that webhooks are getting through from PagerDuty to ServiceNow, but are not creating or updating an incident, try manually closing an incident in ServiceNow. If manually closing an incident requires you to enter values for certain fields, your required fields have likely been customized. Create inbound field rules so that required fields have a default value.
  2. Check for interfering state values: Check to see if you have any custom choices set up for the State field in the ServiceNow Incidents model:
    1. Right-click on the State field’s label (e.g., New or Resolved).
    2. Select Configure Dictionary.
    3. Switch to Advanced view.
    4. Select the Choices tab and look for entries with higher numbers in the Value field.

The integration follows the same convention as ServiceNow’s out-of-box state values. Higher numbers imply a more advanced state (i.e., 6 or higher represents done, resolved, or some variant). If the state of the incident that did not automatically resolve is a higher numeric value, but the incident did not resolve, it is because it was treated as already resolved.

To correct this, you may need to modify custom choice values for the State field to be lower than 6, so that they are treated as an unresolved state.

  1. Check for customizations:
  • Check for customizations to the configuration files.

    🚧

    Warning

    Please note that any customizations which alter the integration’s out-of-the-box behavior are not supported by the PagerDuty Support Team.

  • Are there any modifications or customizations to the fields on the incident form?

  1. Check for interfering data policies:
    Data policies allow ServiceNow administrators to create validation rules on tables. These data policies can stop the webhook import process from committing PagerDuty changes to the ServiceNow Incident record, and thus prevent the integration from automatically resolving incidents.
    When a data policy does this, there will not be any clear indication of it in the application-scoped logs. There will, however, be an error-level log entry in the system logs created at the time of the webhook import, with a message beginning that begins with Data Policy Exception.
    As of ServiceNow version 4, inbound field rules give you the ability to automatically fill incident fields, and this can be used to fill fields as needed in order to satisfy data policies.
    To address this:
    1. Navigate to System Policy Rules Data Policies.
    2. In the list view, create a new filter on the Table field: =incident.
    3. Make note of all the fields and constraints.
    4. Go to PagerDuty Configuration PagerDuty Inbound Field Rules.
    5. For each incident field that was part of the data policy’s conditions:
      1. Create a new inbound field rule.
      2. Set the webhook type to Acknowledge or Resolve, based on which action is experiencing an issue.
      3. Enter a default value for the field, or one dynamically based on the webhook payload.

For more information, please see Inbound Field Rules.

Check Requirements and Restrictions in Your PagerDuty Instance

  1. Check that Assignment Groups match the Escalation Policy:
  • If a ServiceNow Incident isn’t updating after changes happen on the corresponding PagerDuty incident, make sure the relevant PagerDuty escalation policy has a corresponding Assignment Group in ServiceNow.
  • Ensure that the appropriate assignment group has its PagerDuty Escalation field set to the escalation policy’s ID (for example, PXXXXXX).
  • You may need to add the two PagerDuty fields to the group form or list view to review their values.

Check PagerDuty Logs

To get the most detailed insight into script behavior, in ServiceNow go PagerDuty Configuration PagerDuty Settings and set Logging verbosity level to debug. With this setting selected, each call to gs.debug in the script source will show up in the application-scope logs, which can be viewed under PagerDuty Support Logs.

Duplicate Incidents Triggered in ServiceNow

  • Coalesce was not set to true, meaning that any webhook originating from the same PagerDuty incident isn’t getting de-duplicated, and as a result, it’s creating new ServiceNow Incidents.

    • You may see this behavior if the import transform map is not set to coalesce on the incident ID. Verify that coalescing on the incident ID is enabled in the import transform map:
      • In ServiceNow, go to PagerDuty Import Rows Webhook Import Service.
        b. At the bottom of the page, select PagerDuty Webhook Import under Web Service Transform Maps.
      • At the bottom of this page, find the section Field Maps.
      • In the mapping for id x_pd_integration_incident, ensure that Coalesce is set to true.
  • There may be instances where coalescing is true, but duplication is still occurring. Thus, we recommend trying to index the PagerDuty Incident ID field in the ServiceNow incident table incident.x_pd_integration_incident, so that coalescing on that column is more efficient.

PagerDuty Webhooks are Generating a 401 Error

A 401 server error indicates that the accessing user is unauthorized. If you use IP safelisting, make sure you have safelisted PagerDuty’s IPs in your production environment. You can use a tool such as Postman to test remote access.

Please read our developer documentation Webhook IPs for more information about the IPs addresses PagerDuty uses to send webhooks.

Data From ServiceNow to PagerDuty

ServiceNow Incidents and Updates do not Appear in PagerDuty

  1. In ServiceNow, select All and search and select Integration Health Check to run the health check.
  2. Check the Default user ID on the PagerDuty Settings page in ServiceNow:
    1. The Default user ID configuration property in the ServiceNow app needs to be a PagerDuty user ID.
    2. The PagerDuty user associated to this ID must be linked/mapped to a ServiceNow user.
    3. Their login email address in PagerDuty must match their email address in ServiceNow.

ServiceNow Sends Too Many Incident Emails or Worknote Updates

We’ve seen cases where ServiceNow sends hundreds, or even thousands of emails or worknote updates on an Incident. This is usually due to a feedback loop, which can result from ServiceNow custom business rules or automation. It is also possible to use webhooks to inadvertently create a feedback loop between ServiceNow and PagerDuty.

This can happen if there are any business rules or other automated processes that alter the Assignment Group of the ServiceNow incident, and caution is not exercised when manually setting the integration fields. To give a high-level example of how this can possibly happen, let us assume the following configuration:

  • There is a business rule that automatically changes the Assignment Group to B if it is placed into Assignment Group A.
  • Both Assignment Group A and B are tied to PagerDuty escalation policy Z.

A feedback loop occurs in the following scenario:

  1. Incident changes the Assignment Group to, or is created in, B.
  2. A PagerDuty incident triggers and is assigned to escalation policy Z.
  3. A webhook fires and escalation restarts (webhook type incident.delegate or incident.trigger).
  4. Webhook import in ServiceNow assigns incident to assignment group A.
  5. Automation changes assignment to B.
  6. The loop repeats.

If your configuration matches or is similar to the scenario described above, you’ll need to make adjustments so that the loop is broken.

ServiceNow Creates Duplicate PagerDuty Incidents

Assuming that there have been no customizations to the out-of-box PagerDuty integration, the most likely cause is automation. I.e., a separate business rule that is triggering a second automatic update on the incident record after the initial one. The business rule PD Trigger PagerDuty Incident runs with order = 100, meaning that it runs after changes are committed to the database. If two updates happen asynchronously in rapid succession, they risk triggering the business rule that creates a PagerDuty incident twice.

To resolve this, first you will need to find out how the secondary updates take place:

  1. In ServiceNow, go to System Logs Events.
  2. Find events named x_pd_integration.trigger_incident around the time duplicate incidents triggered.
  3. Note the values of:
    1. param1, the last worknotes entry.
    2. param2, a comma-delimited list of extra parameters: current user sys_id, the operation (insert or update), and a boolean for whether the value of the Assignment Group or Configuration Item changed.
    3. User ID
    4. User name
  4. Identify the event that was raised by the automated process based on how the event was submitted, vis-à-vis the available attributes.
  5. Find some attributes of the automated update operation that can be used to distinguish it from others, e.g., if it was made by a human.
  6. Add conditions to the business rule PD Trigger PagerDuty Incident that exclude updates which have the distinguishing characteristics of the automated updates.

Data from ServiceNow to PagerDuty FAQs

When is an incident sent to PagerDuty?

By default, the PagerDuty application triggers a PagerDuty incident when two conditions are met:

By default, the priorities 1 - Critical and 2 - High will be selected. If you wish to trigger PagerDuty incidents for other priorities as well, you can adjust these conditions in ServiceNow under Configuration Files Business Rule PD Trigger PagerDuty Incident business rule.

If you edit to the priorities that trigger PagerDuty incidents, you will likely also want to make the same adjustment to the priority conditions in a second business rule, PD Trigger PagerDuty Incident on update, which covers scenarios where an existing ServiceNow incident (not yet linked to PagerDuty) is updated in a way that meets the conditions to create a PagerDuty incident.

Business Rules

For more information about ServiceNow business rules, we recommend the following ServiceNow resources:

Custom Fields

Custom Fields FAQs

How do I create a new mapping if the “Custom Field limit in PagerDuty has been reached”?

Since you have reached the amount of custom fields available in your PagerDuty pricing plan, you will need to delete an existing PagerDuty Custom Field in order to free up a new creation. The mapping experience today provisions a new PagerDuty Custom Field based on the ServiceNow Incident record field chosen.

Why can’t I edit the newly created PagerDuty Custom Field in ServiceNow after mapping?

To ensure configuration consistency, we are enforcing that fields involved in a mapping should only be editable inside its original system (i.e., within PagerDuty or ServiceNow). Please see our Custom Fields article for more information on how to edit an existing PagerDuty Custom Field.

Why can't I delete the custom field mapping?

You must delete the custom field mapping in ServiceNow before deleting the custom field in PagerDuty. If you are unable to remove a custom field mapping because it was not removed from ServiceNow first, please reach out to [email protected] for assistance.

Why can’t I edit the PagerDuty Custom Fields options for the Single Select data type when it is mapped to ServiceNow?

For the Single Select and Multiple Select data type, we are enforcing that the ServiceNow Incident record field is the source of truth for the options. Option edits and deletions in ServiceNow will be immediately reflected in the PagerDuty UI. Newly added options in ServiceNow will also sync back to PagerDuty until the maximum number of options for the PagerDuty Custom Field is reached. However, if a ServiceNow field has more options available than PagerDuty’s, options beyond that PagerDuty limit will not sync to the PagerDuty UI and the last selected option will be displayed instead.

Can an admin disable or temporarily pause custom field syncing?

At this time, we do not have any features that would allow admins to temporarily pause custom field syncing.

Are duplicate custom field mappings allowed?

You cannot map a ServiceNow Incident record field to an existing PagerDuty custom field.

Can the integration be modified to only sync data in one direction?

At this time, we do not have any features which would allow users to modify the bi-directional sync.

What is the expected behavior if multiple ServiceNow tickets are linked to the incident, or vice versa?

If there are multiple ServiceNow tickets linked to one PagerDuty incident, or multiple PagerDuty incidents linked to one ServiceNow ticket, the custom field value should remain synced with the value of the more recent record.

Inbound Field Rules

Inbound Field Rules not Filling Target Fields with Desired Values

A Business Rule or Process is Overwriting Field Changes

A business rule or some other process is overwriting the changes to the field after it is initially set by the field rule.

  1. In ServiceNow, go to PagerDuty Configuration PagerDuty Inbound Field Rules and verify the following:
    1. For the inbound field rule in question, Enabled is true.
    2. The PagerDuty webhook type is the expected type.
  2. Go to System Definition Business Rules and look for all rules that modify the field as follows:
    1. Enable the field Set field values in the list view.
    2. Search filter for table: =incident.
    3. Filter out all rows where Set field values is blank.
    4. If there is more than one page of results, search for all records where Set field values contains the column name, e.g., *business_service. If any results modify this field, it may explain why the field is not getting set according to the inbound field rule; check that the business rule is not overwriting the field.
    5. If nothing turns up, look at all business rules that run a script and check whether any scripts set the value of the field in question.

The Inbound Rule Only Applies to a Specific event type

Check if the inbound field rule event type matches the event type sent from PagerDuty.

The Inbound Field Rule is Inactive

Ensure that the inbound field rule is active.

New v3 Webhook Payload

Check if your inbound field rule is referencing a payload field that should be updated in order to reference the new v3 webhook payload.