Incident communications

Context

Communications posted via a public outage page that alerted customers to product incidents which caused degraded performance or outages to their SIEM (security information and event management) service.

Sample

Slack Alert Delivery Issues
November 14, 2022

Identified

We have received reports of alert delivery failures for customers using our Slack webhook integration after deployment of the patch yesterday. Our engineering team is working on an additional patch to resolve these new issues. We will update this incident with more information shortly.

Update

Our team has begun deploying the patch to customer accounts. We will update this incident again after the deployment is complete and our monitoring indicates that the issues have been resolved.

Resolved

The patch containing the fix for this issue has been deployed to all impacted customer accounts. Alert delivery to Slack destinations using webhooks are now functioning as intended.

To check for alerts that failed during this time and manually resend them, please follow the instructions below:

  1. Open the Panther Console and navigate to Alerts.
  2. Click the name of an alert that shows “Destination Delivery Failed”.
  3. Next to “Alert Delivery Failed”, click “Show History”.
  4. Click the retry delivery button.