Authentication issues related to API requests
This summary is created by Generative AI and may differ from the actual content.
Overview
Between 15:05 UTC and 16:25 UTC, GitHub API services experienced degraded availability due to sporadic authentication failures affecting roughly 9 % of requests. Customers saw intermittent “logged out” behavior as erroneous 401 responses triggered repeated authentication flows in app integrations, and affected requests incurred about 800 ms of additional latency.
Impact
Degraded availability for API Requests and Issues, with ~9 % of API traffic experiencing authentication failures and ~800 ms added latency. Users encountered intermittent logout experiences in integrated applications.
Trigger
A rollout of a memcached proxy service to the internal API infrastructure caused the authentication service to read an incorrect memcached host configuration, resulting in intermittent authentication lookup failures.
Detection
Internal monitoring observed a rise in 401 error rates and increased latency, prompting the on‑call team to investigate the sporadic authentication failures.
Resolution
A configuration change was deployed to the memcached service to point to the correct host, restoring proper authentication lookups and mitigating the incident.
Root Cause
Incorrect memcached host configuration introduced by the memcached proxy service rollout, which caused the authentication service to query the wrong memcached instance.