A note on incidents: incidents are internal events for our infrastructure team. Incidents often correspond to degraded service on our platform, but not always. This log aims for 100% fidelity to internal incidents, and is a superset both of our status page events and of customer-impacting events on the platform. It includes events reported to subsets of customers on their personal status pages, as well as events without any status page impact.
Trailing 2 Weeks Incidents
Merry Christmas and Happy Holidays! The infra-log got a bottle of bourbon aged in nocino barrels, and a copy of the board game Ark Nova. What did you get? We went deliberately quiet over the holiday, and are just now catching up, so this week’s infra-log update covers almost a month of time.
(Larger boxes are longer, darker boxes are higher-impact; this week, unlike ordinary weeks, all 3 rows of the chart are “fresh.”)
- December 19: GDL Networking Issue (23:00EST): Our Latin American upstream experienced a 10-minute networking hiccup, which we caught with synthetics. We status paged it, but it had been resolved practically by the time the update posted.
That’s it! Normally, even though major incidents are pretty rare, there’s a smattering of little things to post here; “incidents” we flagged internally that didn’t merit a status page update, or that had limited impact. Not this week, though!
These Weeks In Engineering
The lack of incidents to write about isn’t really just happenstance. We locked the platform down in anticipation of the holidays; major changes, such as to our state propagation system, our Anycast routers, or Fly Machine scheduling, were all frozen. We don’t want to get paged on New Years Eve any more than you do.
So! Not many major things to report this last month! Engineering updates should pick up next week.