Configure PagerDuty Control Center Alerts for Confluent Platform

PagerDuty opens an incident when it receives a webhook or an email to the designated PagerDuty integration address.

The email notification from PagerDuty contains:

  • The incident numbers created in PagerDuty.
  • View Incident buttons to view an incident directly in PagerDuty.
  • Trigger, topic, cluster, and metric details.
  • Links to view alerts history in Control Center for certain trigger types.
../../_images/c3-pagerduty-email-alert.png

PagerDuty alert example email

Control Center alerts can send email alerts to a PagerDuty account in the following ways:

Send both PagerDuty webhook and email notifications

If you are planning on or already using PagerDuty webhook notifications, it is simple to send email notifications. Setting up an SMTP email configuration in Control Center properties files is not necessary in this scenario. Email notification does not need to be enabled in control.center.properties files because you are using the PagerDuty email integration in conjunction with webhook notifications.

The webhook notification from Control Center to PagerDuty creates an incident but does not send email notification.

Follow these steps to be notified through email that an alert has been triggered through the webhook notification.

  1. Set up PagerDuty webhook notification in an alert action and associate it with a trigger. The PagerDuty webhook activation must be active within both Control Center and PagerDuty.
  2. Set up a PagerDuty email integration following the PagerDuty documentation.
  3. Set up a test trigger and action to quickly generate test alerts with history. See Tips for testing PagerDuty and Control Center email alerts. Point the email action to your PagerDuty email integration address.

Send PagerDuty email notifications through SMTP

PagerDuty opens an incident when it receives an email sent from Control Center to the designated PagerDuty integration address. In this scenario, you must configure your SMTP mail server and Control Center email settings. Follow this procedure if you do not use the Control Center PagerDuty webhook notification.

Prerequisites

  1. Configure your SMTP mail server if it’s not already configured. For non-production testing purposes, you can use FakeSMTP.
  2. Enable email alerts in Control Center. Email alerts are not enabled by default. See Enable email alerts.
  3. Set up your SMTP email configuration in the control.center.properties files. See Configure SMTP email.
  4. Set up a PagerDuty email integration following the PagerDuty documentation.

To test SMTP email alerts to PagerDuty:

  1. Set up a test trigger and action to quickly generate test alerts with history. See Tips for testing PagerDuty and Control Center email alerts.
  2. Set up an email action that points to your PagerDuty email integration address.

Tips for testing PagerDuty and Control Center email alerts

Depending on the trigger condition, it can take some time for a trigger event to fire. To trigger test alerts with history in Control Center right away:

  1. Set up a topic trigger on the pageviews topic created using the Quick Start guides. Set the Bytes in metric to greater than 0 (zero).

  2. Create a corresponding action that sends a notification at a frequency of once per minute.

    After you have generated and tested some alerts, you can pause, disable, edit, or delete your test triggers and actions.

PagerDuty considerations:

  • Set up a temporary escalation policy that automatically resolves a test incident after a few minutes. Follow the PagerDuty documentation to do so.
  • Set up a temporary service in PagerDuty so that you are not spamming your PagerDuty account and colleagues while testing alerts. You can disable (pause) or delete your test service when you no longer need it. Follow the PagerDuty documentation.
  • For more details about how to manually trigger test incidents within PagerDuty, see the PagerDuty Incidents documentation.

SMTP considerations:

  • Use a temporary or alternative email address to test.
  • Use a temporary SMTP server to test. For non-production testing purposes, you can use FakeSMTP.