Incident with high errors on Git Operations
This summary is created by Generative AI and may differ from the actual content.
Overview
On May 11, 2026, from 14:00 UTC to 14:33 UTC, HTTP‑based Git read operations experienced degradation. The error rate averaged 2.8 % and peaked at 7.5 % of requests. The degradation was caused by resource exhaustion in a networking gateway that sits between GitHub.com’s frontend service for Git operations and the authentication/authorization dependency. The frontend service subsequently entered a stuck degraded state in one data center, extending mitigation time. The incident was mitigated by scaling the networking gateway and redeploying the frontend service. Future mitigation includes adding auto‑scaling to the gateway and fixing the bug that kept the frontend service degraded.
Impact
The incident affected Git read operations, with an average error rate of 2.8 % and a peak of 7.5 % of requests failing. No additional user‑impact metrics or revenue impact were reported.
Trigger
Resource exhaustion in the networking gateway that connects the Git frontend service to the authentication and authorization backend caused the degradation.
Detection
The team became aware of the issue when they began investigating reports of degraded performance for Git operations at 14:25 UTC.
Resolution
Mitigation was achieved by scaling the networking gateway to increase capacity and redeploying the frontend service, which restored normal operation by 14:33 UTC.
Root Cause
The root cause was resource exhaustion in the networking gateway, which led to a degraded state in the frontend service. A bug in the frontend service prevented it from recovering automatically once degraded.