We have published a post-mortem for this incident on our blog.
Posted Dec 04, 2018 - 22:00 PST
Resolved
We have finished processing the backlog and returned to normal data processing capacity.
Posted Nov 30, 2018 - 16:39 PST
Update
We've deployed additional resources to resolve the final backlog.
Posted Nov 30, 2018 - 13:02 PST
Update
We are continuing to monitor for any further issues.
Posted Nov 30, 2018 - 10:42 PST
Update
The backlog has been resolved for most applications. We anticipate that the remaining processing will be resolved within the next day.
Posted Nov 30, 2018 - 07:15 PST
Update
We have finished processing the backlog for the majority of our customers. We expect to be fully caught up in the next few hours and return to the normal data processing capacity for Friday daytime (U.S.) traffic.
Posted Nov 30, 2018 - 01:14 PST
Update
Data processing has caught up for some applications. We continue to work on the backlog for the remaining applications.
Posted Nov 29, 2018 - 17:20 PST
Update
We are continuing to progress and slowly catch up with the backlog. In the meantime, we're investigating ways to process the data more quickly.
Posted Nov 29, 2018 - 12:44 PST
Update
Note, the uptick "Data Processing Delay" is due to raising the maximum value displayed. Previously, 360 was the highest value we would show. We are now displaying the true value.
Posted Nov 29, 2018 - 09:13 PST
Update
Our workers continue to process data for all applications. We still anticipate a full recovery without data loss.
Posted Nov 29, 2018 - 07:58 PST
Monitoring
We've deployed the hotfix to all of our workers and are processing data for all applications. It may take some time before processing is caught up, but we do not anticipate data loss. We continue to work towards a more permanent resolution to the issue.
Posted Nov 28, 2018 - 17:41 PST
Identified
We've identified the issue and have a hotfix partially deployed. Data processing has resumed for some applications and we anticipate that it will resume for others soon. At this point, we do not believe that any data has been lost.
Posted Nov 28, 2018 - 16:17 PST
Investigating
We are currently investigating this issue.
Posted Nov 28, 2018 - 02:37 PST
This incident affected: Data Processing (Application).