Incident with API Requests

Severity: Major
Category: Bug
Service: GitHub

This summary is created by Generative AI and may differ from the actual content.

Overview

On October 22, 2025, between 14:06 UTC and 15:17 UTC, GitHub.com experienced an incident where less than 0.5% of web users saw intermittent slow page loads and up to 2% of API requests timed out due to increased latency. This was caused by elevated loads on a database stemming from a poorly performing query. The issue was resolved by optimizing the problematic query and implementing new monitoring for query performance.

Impact

Less than 0.5% of web users experienced intermittent slow page loads on GitHub.com. API requests showed increased latency, with up to 2% timing out. The incident lasted for 71 minutes (14:06 UTC to 15:17 UTC).

Trigger

The incident was triggered by elevated loads on one of the databases, which was directly caused by a poorly performing query.

Detection

Awareness began with reports of degraded performance for API Requests. Users also reported experiencing slow, timing out requests or 'not found' errors when browsing repositories. Subsequent internal investigation identified slowness within the platform.

Resolution

The team identified the source of the elevated load as a poorly performing query and optimized it to restore normal performance. Additionally, new monitors were added for early detection of query performance issues, and the system continues to be closely monitored.

Root Cause

The root cause was an elevated load on one of the databases, directly attributable to a poorly performing query.