The degradation on the real time collaboration service that occurred on Oct 30th was caused by a self-inflicted DDOS based on our websocket authentication protocol that would retry forever on a timeout. This was triggered by a high server side response time (caused by high load on our servers triggered by a slow deployment process) that made timeout of the authentication handshake to occur more often. We fixed the issue by improving the authentication protocol to allow for more flexibility in server side response time and by adding an "exponential backoff" strategy that would prevent a self-inflicted DDOS in the future.