MURAL Notifications Delay
Incident Report for Mural
Postmortem

What happened?

On Friday, June 17 2022 at 22:20 UTC, our email service provider incorrectly identified a security risk with MURAL’s account. As a precaution, our email service provider suspended MURAL’s account. This resulted in MURAL product emails (for example: invitations to join a mural) not sending.

MURAL worked with our email service provider to restore service as quickly as possible. Together with our service provider we implemented a fix at 02:50 UTC on June 18th, which restored full service and allowed the pending emails to be sent. We continued to monitor the results of the fix until 04:00 UTC, at which time the incident was closed.

Summary

MURAL’s platform remained available throughout this incident. There was a delay of 2 hours and 50 minutes sending product emails, but this did not impact service availability.

Posted Aug 23, 2022 - 12:43 GMT-03:00

Resolved
The fix we implemented earlier has been successful in resolving the issue.

We apologize for the inconvenience this interruption caused and will be conducting a full review to avoid a repeat in the future.
Posted Jun 18, 2022 - 00:57 GMT-03:00
Update
We are continuing to monitor the results to make sure the fix has the intended effects.
Posted Jun 18, 2022 - 00:53 GMT-03:00
Monitoring
We've implemented a fix that should resolve the issue. We're monitoring the results to make sure this has the intended effect and will update this message as soon as we've been able to confirm this.
Posted Jun 17, 2022 - 23:51 GMT-03:00
Update
Our team is still working on resolving this issue. We'll update this message as soon as we have more details.
Posted Jun 17, 2022 - 23:01 GMT-03:00
Update
We're continuing to work on resolving this issue.
Posted Jun 17, 2022 - 22:04 GMT-03:00
Update
We have identified the path to return to full service and are working towards this as quickly as possible.
Posted Jun 17, 2022 - 20:48 GMT-03:00
Identified
The issue has been identified and our team is working on fixing it. We'll update this message as soon as we have more details.
Posted Jun 17, 2022 - 19:59 GMT-03:00
Investigating
We're experiencing a service disruption that is causing notifications, including password reset emails, to not be delivered. We're investigating the issue and will restore regular service ASAP.

Please check our status page for the most up-to-date info 👉 status.mural.co/
Posted Jun 17, 2022 - 19:37 GMT-03:00
This incident affected: Mural Application (Notifications).