API performance regression causing some writes & events requests to timeout

[Jeff_Schneider]

Our infrastructure team has identified performance issues that are causing some mutating and events requests to timeout. If you started receiving a 500 Internal Server Error over the past 2 weeks for mutating/events requests that previously worked, this performance regression is likely the cause.

This regression is especially impacting the GET /1.1/events route. The mutating requests most affected are PUT requests to /tasks and /projects. In some cases, retries may be a solution.

Our eng team is actively investigating the issue. We will update this post as we learn more.

Thank you for your patience while we investigate and remedy the underlying cause. We apologize for the inconvenience.

Best,
Asana DevRel

[Jeff_Schneider]

Update:
The underlying issue was identified and a solution was implemented. As of March 10th, the logs show that it was resolved for API users.

Our eng team determined that the timeouts were being caused by memory starvation. We increased the memory allocated to each web server, fixing the sporadic timeouts.