Code search experiencing degraded performance
This summary is created by Generative AI and may differ from the actual content.
Overview
Code search experienced degraded performance for approximately 5 hours, with 5-10% of search queries timing out due to a single customer's orchestrated network of accounts executing uniquely expensive search queries. This concentrated load on a single hot shard within the search index, causing widespread latency and timeout issues.
Impact
Lexical code search queries in GitHub.com and the code search API were significantly slowed, with 5-10% of search queries timing out. The incident lasted from 2026-02-23 19:10 UTC to 2026-02-24 00:46 UTC.
Trigger
A single customer created a network of hundreds of orchestrated accounts that executed uniquely expensive search queries, concentrating load on a single hot shard within the search index.
Detection
Initial reports indicated impacted performance for some GitHub services, leading to an investigation into elevated latency and timeouts for code search.
Resolution
The source of the load was identified and traffic was stopped, restoring latency to normal levels. Additionally, improvements were planned to rate limiting, system resilience, tooling for recognizing abusive actors, and emergency load shedding capabilities.
Root Cause
The incident was caused by a single customer's orchestrated network of accounts executing uniquely expensive search queries, which concentrated load on a single hot shard within the search index, leading to elevated CPU and widespread latency and timeout issues.