Backend performance issues

Incident Report for 1build

Resolved

We have released several fixes and have observed the performance has returned to baseline for the most part. We sincerely apologize for the frustration and inconvenience this has caused.
Posted Nov 02, 2021 - 15:53 EDT

Update

We have implemented a fix and are currently monitoring the situation for any additional issues.
Posted Nov 01, 2021 - 18:03 EDT

Update

We are now seeing an increase in performance issues once again, specifically around creating new projects and assemblies. We have identified the backend component responsible for these issues and are working on steps to mitigate.
Posted Nov 01, 2021 - 17:20 EDT

Update

We are still continuing to track down and resolve the remaining performance issues. We currently do not have an estimated time for these fixes to be implemented.
Posted Oct 29, 2021 - 17:29 EDT

Update

We are continuing to monitor the situation as we track down and address additional issues.
Posted Oct 29, 2021 - 11:57 EDT

Monitoring

We have implemented another fix that has improved the performance and stability of the system. We are continuing to monitor the situation.
Posted Oct 28, 2021 - 20:58 EDT

Identified

We have identified one issue and have implemented a fix. This has improved the performance of our backends but we are still working on fixing some remaining issues. We are downgrading this to a partial outage at this point as we continue to work on it.
Posted Oct 28, 2021 - 20:12 EDT

Update

We are continuing to investigate the source of the issue but have determined it is more serious than initially thought, so we are upgrading this to a Major Outage.
Posted Oct 28, 2021 - 18:07 EDT

Investigating

We are currently investigating degraded performance in the 1build Portal
Posted Oct 28, 2021 - 17:05 EDT