Back to overview

Bob the Bot Service Degradation

May 20, 2026 at 5:00am UTC
Affected services
Bob (Bot Service)

Resolved
May 20, 2026 at 5:00am UTC

Resolved May 19, 2026 at 8:41pm CDT

Date: May 19, 2026 | ⏱ Duration: 3 hours 25 minutes

🧭 Summary
Bob experienced partial failure. While the interface could initiate command responses, critical internal communication between service components and the database failed due to Railway infrastructure issues, resulting in global 404 "Application not found" errors.

🛠️ Resolution
Bob achieved full restoration at 8:41pm once Railway restored internal connectivity for our service components. Note that our service was fully functional well before Railway's platform-wide recovery was complete.

🕑 Timeline
5:16 PM – 🚨 404 "Application not found" errors detected.
5:25 PM – 🛑 Incident triggered.
8:38 PM – ✅ Service monitors recovered across all regions.
8:41 PM – 🏁 Incident auto-resolved; Bob fully functional.
~11:58 PM – 🚇 Railway officially declared the upstream incident resolved.

Root Cause: Railway's GCP account was erroneously suspended by an automated system. This external dependency failure was outside of Railway's direct control, which significantly extended the time required for resolution. Here is what Railway knows so far: https://station.railway.com/community/what-we-know-so-far-may-19th-2026-86354cdd

They will be posting a full post-mortem once they gain a complete understanding of everything that went down.

As always, thanks for your understanding!