Swaps API is unavailable
Resolved
Sep 02 at 10:02am EDT
Production swaps servers were restarted manually, beta servers recovered automatically.
Affected services
Updated
Sep 02 at 08:55am EDT
Umbrasync database was cleared manually and measures will be taken so that automated recovery is less likely to fail in the future. The swaps API should recovery automatically in the next 5 to 10 minutes, if not, they will be restarted manually.
Affected services
Updated
Sep 02 at 08:49am EDT
It is now known why the automated recovery measures failed. The recovery process involved clearing the chain snapshot, which took more than the timeout time given.
Affected services
Created
Sep 02 at 08:20am EDT
At approximately 1AM EDT, Umbrasync started having a noticeable delay in providing real-time data to our swaps solver. Automated recovery measures were attempted at 4AM EDT which failed. The issue is currently being diagnosed
Affected services