Webhooks delivery degradation

Severity: Minor
Category: Dependencies
Service: GitHub

This summary is created by Generative AI and may differ from the actual content.

Overview

Webhooks delivery degradation caused delayed writes of webhook delivery records to the database, making many webhook deliveries not visible in the UI or API for over an hour.

Impact

Customers were temporarily unable to request redeliveries for delayed records.

Trigger

Throttling of database writes due to high replication lag

Detection

Not specified in the postmortem report

Resolution

Temporarily disabling delivery history for a small number of very high-volume webhook owners to reduce write pressure and stabilize the service

Root Cause

Throttling of database writes due to high replication lag