Incident Report: 403 / 429 Errors for Some Users
This summary is created by Generative AI and may differ from the actual content.
Overview
At 11:06 UTC, our CDN partner deployed changes intended to detect spurious traffic by observing the 'Referer' HTTP request header. This change caused some requests from the npm CLI to be flagged as suspect by the CDN.
Impact
growing chorus of user feedback
Trigger
CDN partner deployed changes intended to detect spurious traffic by observing the 'Referer' HTTP request header
Detection
To our monitoring systems, this deployment simply made it look like our CDN partner was helping us block harmful or abusive traffic. To our support staff, and to the initial technical investigators, this issue presented as a growing chorus of user feedback.
Resolution
By 13:00 UTC they had deployed a fix, resolving the issue.
Root Cause
requests were being blocked if they contained HTTP Referer headers that were not fully qualified URLs. This led them to reject properly formatted npm traffic, notably 'install' requests.