AWS CloudTrail Integration Guide | PagerDuty

Configure the AWS CloudTrail integration

AWS CloudTrail + PagerDuty Benefits

  • The AWS CloudTrail integration enables automated response workflows for activities that could affect compliance and security issues in your AWS ecosystem.
  • PagerDuty provides the opportunity to group alerts alongside other ongoing issues and it has seamless integrations with systems of record like Jira and ServiceNow.

Requirements

To Configure the Integration:

  • In PagerDuty: Managers, Admins, Global Admins and Account Owners can configure the integration.
  • In AWS: Users who set up and manage AWS Config must have full-access permissions. Please read Amazon’s documentation What Is AWS Config? for more information about managing permissions in AWS.

How it Works

  • When a CloudTrail metric goes beyond a predefined ALARM state threshold, a CloudWatch alert sends an event to a PagerDuty endpoint, triggering an incident.
  • When the CloudTrail metric returns to an OK state below the predefined threshold, a resolve event is sent to the same endpoint, resolving the PagerDuty incident.

Version

This guide details configuration of the AWS CloudTrail V1 integration.

Integration Walkthrough

In PagerDuty

There are three ways to integrate AWS CloudTrail with PagerDuty:

Integrate With Event Orchestration

Configure a Global Orchestration Integration

  1. Configure a Global Orchestration in your PagerDuty account.
  2. Navigate to AIOps Event Orchestration and select the name of your preferred Global Orchestration.
  3. Select Integrations and copy the Integration Key to your clipboard.
  4. Once you have your Integration Key, the Integration URL will be:

https://events.pagerduty.com/x-ere/YOUR_INTEGRATION_KEY_HERE

You can now proceed to the In the AWS Management Console section below.

Configure a Service Orchestration Integration

  1. Configure a Service Orchestration in your PagerDuty account.
  2. Create a Generic Events API integration on the same service.
  3. Once complete, copy the Integration Key and paste it into the following URL:

https://events.pagerduty.com/x-ere/YOUR_INTEGRATION_KEY_HERE

You can now proceed to the In the AWS Management Console section below.

Integrate With a PagerDuty Service

Add to a New Service

  1. To add the integration to a new service, navigate to Services Service Directory and click New Service.
  2. Follow the prompts and configure the service to your preferences. On the Integrations screen, select AWS CloudTrail from the search bar dropdown.
  3. Once you are done entering your service settings, click Create Service.
  4. You will now be in the service’s Integrations tab. Find your integration in the list and click the to view and copy your Integration URL. Keep it in a safe place for later use.
  5. You can now proceed to the In the AWS Management Console section below.

Add to an Existing Service

  1. To add an integration to an existing service, go to Services Service Directory and select the service where you would like to configure the integration. Select the Integrations tab and click Add another integration.
  2. Select AWS CloudTrail from the search bar dropdown.
  3. Click Add. Find your integration in the list and click the to the right to view and copy your Integration URL. Keep it in a safe place for later use.
  4. You can now proceed to the In the AWS Management Console section below.

Integrate With Rulesets

❗️

Rulesets End-of-Life

Rulesets and Event Rules will end-of-life in 2024. We recommend using Event Orchestration instead, which offers new functionality, such as improved UI, rule creation, APIs and Terraform support, advanced conditions, and rule nesting.

Configure a Global Ruleset Integration

  1. In the web app, navigate to AIOps Event Rules and select your Default Global Ruleset.
  2. On the Event Rules screen, click the Incoming Event Source dropdown and copy your Integration Key.
  3. Once you have your Integration Key, the Integration URL will be:

https://events.pagerduty.com/x-ere/YOUR_INTEGRATION_KEY_HERE

You can now proceed to the In the AWS Management Console section below.

Configure a Service Event Rules Integration

To use service-level event rules:

  1. Configure service event rules on your preferred service.
  2. Create a Generic Events API integration on the same service.
  3. Once complete, copy the Integration Key and paste it into the following URL:

https://events.pagerduty.com/integration/YOUR_INTEGRATION_KEY_HERE/enqueue

You can now proceed to the In the AWS Management Console section below.

In the AWS Management Console

  1. In the Services search bar, search and select Simple Notification Service. In the SNS dashboard left menu, select Topics and click Create Topic on the right. This topic will be used to route alerts to PagerDuty from AWS.
  2. Select the Standard Topic Type.
  3. Next, perform the following:
    • Name: Enter a name for your topic. You may want to name your topic after your PagerDuty service’s name.
    • Display name (optional): Enter an optional display name.
    • Click Create topic at the bottom of the screen.
  4. Now that your topic has been created, select Subscriptions in the left menu and click Create subscription.
  5. Perform the following:
    • Topic ARN: Select the Topic ARN of the topic you just created.
    • Protocol: Select HTTPS.
    • Endpoint: Paste your Integration URL (generated in steps above).
    • Ensure that the Enable raw message delivery checkbox is unchecked.
    • Click Create subscription.
  6. Your subscription should be automatically confirmed. Refresh the page to make sure the Status is Confirmed and not PendingConfirmation.
  7. Next, search and select CloudTrail in the Services search bar.
  8. Perform the following where necessary:
  1. After configuring CloudTrail to send data to CloudWatch Logs, navigate to the CloudWatch console to configure an alarm based on a CloudTrail finding. Navigate to Logs Log groups in the left menu.
  2. Select the Name of the log group and then click Actions Create metric filter.
  3. Enter the Filter pattern to use. For more information, see Amazon’s documentation on filter and pattern syntax. Click Next to continue.
  4. On the next page, perform the following:
  • Filter name: Enter a filter name.
  • Metric namespace: Enter a name for the CloudWatch namespace where the metric will be published. If the namespace doesn't already exist, toggle Create new to the on position.
  • Metric name: Enter a name for the new metric.
  • Metric value: If your metric filter is counting occurrences of the keywords in the filter, enter 1. This increments the metric by 1 for each log event that includes one of the keywords.
    • Alternatively, enter a token such as $size. This increments the metric by the value of the number in the size field for every log event that contains a size field.
  • Unit (optional): Select a unit to assign to the metric. If you do not specify a unit, the unit is set as None.
  • Click Next to continue.
  1. On the final page, review your metric filter and then click Create metric filter.
  2. After creating the filter, you will see it listed in the Metric filters tab. Select the checkbox on your filter and then click Create alarm.
  3. A new tab will open where you can configure a CloudWatch alarm. On the Specify metric and conditions
    screen, verify your alarm threshold and settings and then click Next to continue.
  4. First, you will configure the In alarm state notification, which will trigger a PagerDuty incident when the metric has met your predefined threshold. Select the In alarm and Select an existing SNS topic radio buttons, and then select the SNS Topic (created above) from the Send a notification to… field.
  5. On the next page, enter an Alarm name and click Next.
  6. Next, you will configure the OK state notification, which will automatically resolve the PagerDuty incident if the metric has fallen back into an OK state (not meeting or exceeding the threshold). Click Configure actions in the left pane. Select the OK and Select an existing SNS topic radio buttons, and then select the SNS Topic (created above) from the Send a notification to… field. Click Next to continue.
  7. An Alarm name and Alarm description from a previous step should be prepopulated. Click Next to continue.
  8. On the Preview and create screen, review your alarm’s details. If you need to edit any details, click Edit to the right of each step. Once you have confirmed all details, click Create alarm. You should now see PagerDuty incidents when alarm thresholds are breached.