Users are unable to sign In to MURAL
Incident Report for Mural
Postmortem

What happened?

At 06:25 UTC on September 28 2022 we started to receive notifications of users seeing an error page when accessing their MURAL workspaces. Initial investigations indicated the cause was an issue with our cloud provider. Shortly after we received confirmation from our cloud provider that routine server maintenance had had an unexpected impact on the availability of data hosted in one of their facilities. At 06:55 UTC MURAL implemented a workaround to re-route traffic and restore service via a secondary server. At 07:25 UTC our cloud provider confirmed that full service had been restored on the primary server and we were able to revert the earlier change.

Summary

This incident impacted service availability from 06:25 UTC to 06:55 UTC, for a total of 30 minutes of downtime. No data was lost, however MURAL was not accessible during this time.

What we've done to avoid this happening again

We have improved automated monitoring and notifications to include this particular scenario, to ensure we can restore service faster in the event this should occur again.

Posted Oct 05, 2022 - 18:06 GMT-03:00

Resolved
This incident has been resolved.
Posted Sep 28, 2022 - 05:30 GMT-03:00
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Sep 28, 2022 - 04:29 GMT-03:00
Identified
The issue has been identified and a fix is being implemented.
Posted Sep 28, 2022 - 04:21 GMT-03:00
Investigating
Users are currently unable to sign in to MURAL. We know this is a major service disruption for everyone. We're investigating the issue and will restore regular service ASAP.

Please check our status page for the most up-to-date info 👉 status.mural.co/
Posted Sep 28, 2022 - 03:49 GMT-03:00
This incident affected: Mural Application (Authentication, Realtime collaboration) and Website.