The remaining worker has finished processing the backlog as of 19:16 Pacific Time.
Posted May 28, 2019 - 19:33 PDT
Monitoring
The remaining worker has resumed processing data. We will be monitoring its progress.
Posted May 28, 2019 - 18:01 PDT
Identified
Data processing for most customer apps has resumed to normal as of 16:00 Pacific Time. Unfortunately, one of the new workers (affecting around 10% of customer apps) is still having issues. We are taking further actions to address the underlying issue.
Posted May 28, 2019 - 17:23 PDT
Monitoring
We have deployed the upgraded infrastructure. Data processing should gradually catch up over the next few hours for the affected customer apps (~20%). We apologize for the inconvenience.
Posted May 28, 2019 - 14:21 PDT
Update
We are suspending data processing on the affected worker in preparation of deploying the upgraded infrastructure.
Posted May 28, 2019 - 14:08 PDT
Identified
One of our data processing worker is experiencing high load. We are working on provisioning additional resources. In the meantime, around 20% of customer apps may experience up to an hour of data processing delay.
Posted May 28, 2019 - 13:04 PDT
This incident affected: Data Processing (Application).