Cast by joanwestenberg.eth
I went down a rabbit hole reading the post-mortems and status logs from the Oct 21 Railway breach. On the systems side, I don't have much to say re: the engineering response. On the comms side, it's a different story... 🧵 1/10
1 reply
First, a golden distinction for any // all tech crisis comms: When you announce an outage, customers wait. When you announce a breach, customers act. 2/10
At 09:25 UTC, Railway knew enough to ask Stripe and Anthropic to revoke customer keys. Yet they kept calling it an "outage" on the status page until 21:59 UTC. For over 12 hours, customers sat waiting when they should have been rotating production secrets. Because Railway told vendors 12 hours before telling customers, users found out from third parties first. 3/10
Imagine finding out your infrastructure host got popped because Stripe and Anthropic suddenly nuked your production API keys, 40 minutes before your host said a word. Worse: on-chain researchers tied wallet drains starting at 19:36 UTC to keys stored on Railway. Over two hours before the public disclosure. The Catch-22: Railway told every customer to urgently rotate every secret...while builds were still paused. 4/10
