Webhooks

Webhooks

Webhooks allow you to receive HTTP callbacks when incidents are triggered and updated on a service. Details surrounding the event will be sent to a URL that you specify, such as Slack, HipChat, or your own custom PagerDuty webhook processor.

Add a Webhook

  1. Go to the Configuration menu and select Services, then click the name of the service you want to add a webhook to.
  2. By default, you will be directed to Incidents tab for the service. To add a webhook, switch over to the Integrations tab.
  3. Click Add an extension.
  4. For the Extension Type select your webhook type or Generic Webhook.
  5. Enter a unique Extension Name.
  6. Enter your Endpoint URL.
  7. Click Save.

If you'd like to verify your webhook configuration to make sure it's working as expected, trigger a test incident by clicking New Incident on your service, and then check your endpoint service for updates. For example, if you use a service like RequestBin, you can view the POST request sent by PagerDuty after triggering an incident.

HTTP Authentication and Web Server Access

Webhooks can be sent to any publicly accessible web server, on any port, with or without encryption, and can include a username and password for HTTP basic authentication.

Specify a username and password

If your web server uses HTTP basic authentication, you can add the username and password to the URL before the host address. Special characters such as @ in the password should be percent-encoded.

Example: http://username:password@app.acmemonitoring.com/pagerduty.php

Webhook Ports

If you specify http:// in your Endpoint URL, we will try to initiate a standard HTTP connection on port 80. Likewise, entering https:// means we will attempt to make a secure connection on port 443. To override the default port, simply add a colon (:) after the host address with the desired port number, just like you would enter in your web browser.

Example: https://app.acmemonitoring.com:8443/pagerduty.php

Restrict access to just PagerDuty IPs

You can view our list of IPs to whitelist here.

Can I use a self-signed certificate for webhooks?

Yes. Web servers presenting self-signed or expired certificates will still receive webhooks.

Temporary Blocks and Blacklisting

When a site goes down, it is a generally expected behavior that events can't be received to your webhook, or that it will become completely disabled.

When events are sent to your webhook endpoints, PagerDuty expects a 200 response within 5 seconds. If there is no response, or if there is any response other than a 200, PagerDuty will periodically retry sending the webhook call until it receives a 200.

For example, if a 500 response is received, it is considered a transient issue and regular retry logic is applied. First the event is rescheduled for delivery 50 seconds in the future. If it continues to 500, the PagerDuty will back off by a multiple of 1.25^N for 7 more tries (8 retries total). This takes about 20 minutes.

After these retries are exhausted, the endpoint will be temporarily disabled for 30 minutes. PagerDuty will continue to accumulate events for the endpoint while it is temporarily disabled, but they are all scheduled for after the disable has expired. If the payload can't be sent after this, the endpoint will be disabled for exponentially longer periods of time (again, 1.25^N), up to 6 retries total (about 7.5 hours).

If PagerDuty still can't deliver a webhook event at that point, it will blacklist the endpoint and show that the webhook is disabled in the web app. This means that it will discard any queued events and no longer send events to this endpoint until the user manually unblocks it via the user interface.

Send Email Notifications when Incidents Trigger, Acknowledge, or Resolve

Webhooks are small information packets that are sent out via an HTTP POST whenever an event happens in PagerDuty, allowing you to receive event state change notifications at every step of the incident lifecycle. Our Webhooks API documentation describes the format of webhook data, as well as the information contained in each request.

In PagerDuty, webhooks are implemented at the service level under Extensions. Whenever an incident changes state (triggered, acknowledged, resolved, etc.), we'll send an HTTP POST request to your endpoint with all the details about that change to a URL you specify. Once you receive that request at your listener, you can then use and disseminate that information to your users — for example, through email.

You can set up webhooks on a service's Integrations tab, where you'll find a section at the bottom for webhook configuration. To use webhooks, you must enter a URL where those POST requests will be sent, and a name for your endpoint to help you identify it.

You can use RequestBin to see the information that is being transmitted (i.e. the webhook as you would actually receive it) without using your own server. Under the service where you will create the incident you want to follow, enter your RequestBin address (this is the "listener" as it will be listening for webhooks). Then simply trigger an incident, acknowledge, and resolve it, and check your RequestBin; every state change will be shown there, in JSON format.

Send Incident State Change Notifications via Email Using a PHP Script

For this tutorial, you will need to have access to a web server that supports PHP.

  1. Save one of the PHP files listed here: https://gist.github.com/eurica/6034108
  2. Modify the email address listed to the address that you'd like to get emails for all incident status changes within your text editor: $emailAddress = "CHANGEME@example.com";
  3. Upload the PHP file to your web server.
  4. Log in to your PagerDuty account.
  5. Go to Configuration → Services and select the service that you'd like to get notified about on incident updates.
  6. By default, you will be directed to the Incidents tab of the service's individual page. To add a webhook, switch over to the Integrations tab.
  1. Click New Extension.
  2. For the Extension Type select Generic Webhook, then enter a name for your webhook, paste the URL to the PHP script that you uploaded, and click Save.

Now when an incident is triggered, acknowledged, resolved, etc., you will get an email notification. You can customize the script to format the data to your liking. By default it will tell you the incident status, subject, and service, along with a link to the incident. The full JSON payload is sent with the message.

Webhooks