Previous incidents

We've deployed a fix and are no longer seeing elevated error rates for OmniReport requests.

[Update 7] The issue is now resolved

[Update 6] After deploying the fix to our backend, we are monitoring the systems to check if the solution resolved the issue completely. We will provide a new update by 11:00am PST

[Update 5] On top of the database change previously mentioned, we deployed a new fix to our backend system. We are monitoring the logs to check if it fixed the issue. We will provide a new update by 10:45am PST

[Update 4] We reverted the database change that caused the issue ...

Update 1: Cloudfare has reported the issue is resolved and all internal testing is successful with the Rad AI applications.

Cloudfare currently has a global network outage. We are closely monitoring the situation and will update impacts to Rad AI as soon as our teams complete initial investigation.
https://www.cloudflarestatus.com/

We are stabilized now. We are monitoring closely but message processing is back to normal throughput with no elevated error rates.

Latest AWS Update (Oct 20 2:48 PM PDT):

"We have restored EC2 instance launch throttles to pre-event levels and EC2 launch failures have recovered across all Availability Zones in the US-EAST-1 Regions. AWS services which rely on EC2 instance launches such as Redshift are working through their backlog of EC2 instance launches successfully and we anticipat...