#July 2: Certificate issuance outage (21:40UTC)

We issue certificates using Let’s Encrypt, who had a networking hiccup when failing over datacenters for maintenance. This caused some requests to fail, making our issuance jobs bail out and re-queue themselves. Apart from a handful of lucky hostnames who squeaked through, certificate issuance was out of action for a few hours. We renew certificates well before they expire, so no existing traffic was affected. Any apps that were in the process of setting up their hostname would have seen a delay in getting their first certificate. Once the upstream incident was resolved, all queued issuances and renewals were processed.