Postmortems

Document and learn from past major incident responses

Postmortems are reports that allow users to learn from major incidents by providing a summary of events that transpired, how the response was handled, and what resolution steps were taken. Postmortems curate a timeline of activity in PagerDuty and Slack, which teams can analyze to determine follow-up actions for similar situations in the future and, ideally, help prevent them altogether.

📘

Pricing Plan

The Postmortems feature is available for accounts on our Business and Digital Operations plans. If you are interested in a pricing plan that includes PagerDuty Postmortems, please contact our Sales team for more information.

11581158

Postmortems

Create a Postmortem

🚧

Required User Permissions

Observers, Limited Users, Users, Admins and Account Owners (or their respective advanced permission base roles) can create, edit and delete postmortems .

  1. Navigate to Incidents Postmortems catalog and click New Report, or click the New Postmortem Report button on a resolved incident.
  • If you created a new postmortem from the catalog page: Enter a Report Name and Owner of Review Process. You may optionally enter a Impact Start Time and Impact End Time. Keep in mind that, although multiple people may have been involved in the incident’s response, there can only be a single owner of any one postmortem. Once finished, click Create Report.
  • If you created a new postmortem from a resolved incident: Your report details will be auto populated and you will be assigned as the Owner of Review Process. You may choose to edit these details if necessary. Once finished, click Create Report.
  1. Data Sources: This section allows you to build a timeline of incident log entries and Slack messages related to the incident.
  • PagerDuty: To link incidents to this postmortem, either Select incident between impact start and end times from the dropdown, or Find by incident number by entering incident numbers (one at a time) and click Add This Incident.
  • Slack: If you’d like to include discussions from public Slack channels, you can do so by clicking the Connect to Slack button. Once you’ve authorized PagerDuty to access your Slack account, you will then be prompted to search and select Channels to include in your report.

📘

Slack Data Source

To include Slack content in a postmortem, a contributor must authorize PagerDuty to access Slack on their behalf, which will let that contributor see Slack content in the Available Data listing. Each PagerDuty user will need to authorize with Slack individually.

If your Slack user account does not have permissions to add apps to your team, you will need to first ask a Slack admin to go to Slack's App Directory and authorize the PagerDuty Postmortems app.

Click Save Data Sources to continue.

  1. Timeline: All Slack messages and incident log entries from the above data sources will be listed in the Available Data (left) slush bucket. The incident Trigger and Resolve log entries will be listed in the Included in Timeline (right) slush bucket by default.
  • To add Slack messages or incident log entries to the timeline: Select the event and click the button, or click the to the left of the event to move it from the left to the right slush bucket. You can add multiple items at once by selecting them while holding the shift key and then using the button to move them to the timeline. You can also manually add freeform entries to your timeline by selecting New Timeline Entry. Any Slack entries that are moved from the Available Data listing into the postmortem's Timeline will be visible to anyone looking at the postmortem.
19561956

Add to timeline

  • To remove an event from the timeline: Select the event and click the button, or click the to the left of the event to move it to the left slush bucket.
19581958

Remove from timeline

Click Save Timeline to proceed.

  1. Analysis: This section allows you to summarize and reflect on the curated timeline, including incident root cause analysis and potential steps to prevent or mitigate future incidents.

Fill in the following fields based on your analysis:

FieldDescription
OVERVIEWInclude a short sentence or two summarizing the contributing factors, timeline summary, and the impact; for example, "On the morning of August 14th, we suffered a 14 minute SEV-1 due to a runaway process on our primary database machine. This slowness caused roughly 3% of reports generated during this time to be completed out of SLA."
WHAT HAPPENEDInclude a short description of what happened, usually based on the timeline.
RESOLUTIONInclude a description what solved the problem. If there was a temporary fix in place, describe that along with the long-term solution.
ROOT CAUSESInclude a description of any conditions that contributed to the issue. If there were any actions taken that exacerbated the issue, also include them here with the intention of learning from any mistakes made during the resolution process.
IMPACTBe specific here. Include numbers such as customers affected, cost to business, etc.
WHAT WENT WELL?List anything you think we did well and want to call out. It's okay to not list anything.
WHAT DIDN'T GO SO WELL?List anything you think we didn't do very well. The intent is that we should follow up on all points here to improve our processes.
ACTION ITEMSInclude action items such as:
(1) fixes required to prevent the issue in the future,
(2) preparedness tasks that could help mitigate a similar incident if it came up again,
(3) remaining postmortem steps, such as an internal follow-up email, updating the public status page, etc.
NEW SECTIONIf you would like to add a new section, you may do so here.

📘

Customized Postmortem Templates

The fields above are from the default Postmortem Template. Some fields may be subject to change based on how your postmortems have been customized.

Each analysis field may be edited with the following options by clicking the to the right:

  • Edit Section Title
  • Move to Top
  • Move Up
  • Move Down
  • Move to Bottom
  • Delete Section

You may also add new sections by clicking Add Section.

  1. Once you have completed and reviewed your report, click Save & View Report.

Edit a Postmortem

  1. Navigate to Incidents Postmortems to the right of the postmortem Edit. Or you may select the postmortem’s Name and click Edit Report.
  2. Edit any details necessary and then click Save & View Report.

Delete a Postmortem

  1. Navigate to Incidents Postmortems to the right of the postmortem Delete. Or you may select the postmortem’s Name and click Edit Report and then click Delete Report.
  2. A confirmation dialog will appear. Click Delete to delete your postmortem.

Export a Postmortem

Once you have completed your postmortem report, you can export it as a PDF.

  1. Navigate to Incidents Postmortems and select your desired postmortem's Name.
  2. Click the Save as PDF button.
974974

Save as PDF

Customize the Postmortem Template

The purpose of the postmortem process is to learn how to improve both system resiliency and the response process. As you learn new insights, you can customize your report and add or remove sections that apply to your team's specific use case.

11941194

Customize the Postmortem template

  1. Navigate to Incidents Postmortems Settings tab.
  • To add a new section to your template: Scroll to the default New Section, click the to the right and select Edit Section. You may then enter a new section Title and a Description and then click Save. To continue adding new sections, click the Add Section button at the bottom of the page.
  • To edit existing sections: Click the to the right and select Edit Section. Make any edits necessary and then click Save.
  • To delete sections: Click the to the right and select Delete. A confirmation dialog will appear. Click Yes, Delete Section to delete the section.

Reset the Postmortem Template

You can also reset your postmortem template to the default sections if needed.

📘

Resetting Templates

Resetting your Report Template to its default will only affect future postmortem reports. This change will not affect existing postmortem reports.

  1. Navigate to Incidents Postmortems Settings tab.
  2. Click the Reset Template button at the top of your Report Template. A confirmation dialog will appear. Click Reset Template to reset the template to its default.

FAQ

How can I change the Slack user associated with Postmortems?

Expand

To change the associated Slack user account, you'll have to revoke and replace the current Slack authentication only for the Postmortems feature.

📘

Slack User Limitations

It is currently not possible to choose more than one Slack user specifically for the Postmortem feature.

  1. Within your Slack user account, access the My Authorizations section in the Slack API.
  2. Locate the PagerDuty Postmortems app under the Application & Scopes column.
  3. Click the icon to the right of the PagerDuty Postmortems app to remove it only from your chosen Slack user account.
15361536

Remove PagerDuty Postmortems app

  1. While editing a postmortem in PagerDuty, click the Connect to Slack button under the Data Sources section to authenticate a different Slack user account to the Postmortems feature.

Is it possible to add images or documents to a postmortem?

Expand

It is possible to insert an external link or URL in any postmortem field which hosts the image or document externally. You can insert the link or URL into the section text, timeline entry summary, or in the description of a manual timeline entry within a postmortem. When the link or URL is clicked, then the image or document will open in a new tab. However, it is currently not possible to insert an image directly into a postmortem.