API requests are slow

English Forum Developers & API
1

Hi.

Any insights why successful requests to Asana API generally take 1-2 seconds to execute? A fresh example:

2020-12-11 17:21:59 California time
PUT https://app.asana.com/api/1.0/tasks/1140113615262698
{"data":{"assignee":"1107514444678950","name":"Test","notes":"12"}}
duration_ms: 1519

POST https://app.asana.com/api/1.0/tasks/1140113615262698/addProject
{"data":{"project":"1140106414902969","section":"1140106414902970"}}
duration_ms: 1510

I wonder if we so something incorrectly (like getting close to rate limiting quotas or so), and if yes, could you please provide some hints on how can we speed up such queries?

If we can’t do anything with it on our end, what’s your general expectation about the API response latency? I mean, is it expected on your end that they take 1-2s to succeed (like you wait for some replica become in sync with the primary or so), or something is not as it’s expected to be? Any bit of info or hints would be helpful here, thanks in advance!

This also applies to GET requests (although in average, GET requests are faster, they often take 300–500ms to finish).

1 Like
2

Hi @Dimitri_Ko,

I don’t think you’re doing anything wrong; your experience matches mine and also others that I know of. Writes are definitely slower than reads. As to why its not faster than it is, I can’t say.

@Ross_Grambo do you have any insights here?

3

Also is it correct to say the API is slower for free accounts and faster for paid ones? or only the rate limits are impacted?

4

I’m not aware that it’s slower for free accounts than paid ones; I’d be surprised if that were the case. But the lower rate limit for free accounts, yes, for sure, that one’s documented: Rate limits

1 Like
5

Thanks. But still, is there a chance to reach some Asana engineer with that question, why the write requests take 1-2s to execute (and sometimes up to 10)? I think it has nothing to do with rate limits (they’re not reached), it’s something else and very technical (my guess is that maybe the server waits until the update is delivered to some/all replicas before responding back with success - that would explain why the delay is 1-10 seconds and is unpredictable).

6

I concur with Phil, it was always like this. Engineers have been pinged above by Phil.

1 Like
7

@Bastien_Siebman, which statement is this referring to?

My feeling is the public API has become nogticably slower over the years. I never attempted to characterize the performance formally but believe this is the case anecdotally with many repeated samples over a long period.

Larry

8

Hey there! Friendly API engineer here :slight_smile: I’m jumping in to hopefully give a high level overview of why API writes are slower than reads.

Between 2008 - 2015, all our code (API, server, web framework) were written in a framework known as Luna. This was a JS framework that used server side simulation to achieve reactivity. This framework made a lot of things that at the time were hard, easier, like reactivity / data loading / etc. But, it was also really complex, and, importantly for this conversation, slow. See this blog post for more details on this framework, why we started moving away from it as a company.

Since then, we have been rewritten our read API routes (GET requests) in Scala with a new framework (it doesn’t have a snazzy name unfortunately - we just call it the Scala API :sweat_smile:). We built this framework from the ground up to be easier to write / understand, and also be faster. I’m having trouble finding the exact numbers, but for most (all?) requests this was a massive perf improvement.

But, we haven’t had the opportunity to rewrite our write API routes yet (PUT, POST, etc. requests). This is due to the tightly coupled nature of the Luna framework: it’s hard to decouple the API code from the writes, and the writes from the business logic to accomplish them. This means these routes have stayed the same / slowed down, since they’ve been stuck in this legacy system.

We’re currently investigating performance here in two lenses:

  • what can we do to improve the performance of the write API: as outlined above, if we got rid of Luna and replaced it with something similar to the Scala API, we’d hopefully get better performance. We have some designs here, and are investigating feasibility.
  • how can we improve performance generally: we’re currently doing a lot of work on getting better instrumentation, observability (e.g. incorporating Lightstep), and figuring out if there’s ways we can improve performance across the API. We’ve already identified a way to improve API performance after deployments that we’re looking to get out soon, which should improve the worst case performance in our API.

These are unfortunately long term solutions, we don’t have any quick changes we can do to fix this. But know that we are investing effort here, and hopefully that will incrementally improve performance.

In the meantime, I’d definitely recommend doing writes in the Batch API. While this doesn’t eliminate the performance problems, it can help with write performance.

7 Likes
9

Thanks for this detailed write-up, @Steven_Rybicki - very good to know!

1 Like