The API is not supporting my many requests (a case study)

English Forum Developers & API
1

Hi :wave:,

I’ve had a very interesting call with a client who was having trouble using the Asana API. They have 3.000 projects and half a million tasks. Each night, they request the API through Integromat for about 200 projects. The code runs for a while, and then stops, with the Asana API basically saying “stop harassing us”. We discussed everything that could go wrong and ways to fix the situation. I was curious to have community inputs as well!

429 errors handling

Integromat might not be handling the 429 rate limit error from Asana. The answer from Asana contains the time you are supposed to wait before the next request. Not handling properly that answer might result in the API stopping you from requesting.

Parallel versus sequential

Requesting 200 projects at the same time, in parallel, might not be a good idea. If that is how your code operates, consider requesting projects one by one, in sequence, to avoid an API overload resulting in errors.

The source times out

Depending on the tools you use for requesting the API, it might have timeouts on its own. Not all code is able to run for several hours in a row without completing.

Too hungry in terms of data

To determine whether or not you hit a rate limit, Asana computes the cost of your request. If you ask for only tasks gid, the cost is low. If you ask for the assignee’s name, email, custom field values… the cost increases. Your requests might be asking for too much and you probably have room for improvements by requiring less.

Repeating yourself

It is better to ask the API for all tasks within a project, and then for the identity of the unique list of assignees, rather than ask for the assignee’s identity for every single task.

Blacklist versus whitelist

Your rate limit depends on the Asana plan your organization is on I believe. A free plan has a lower threshold than a Business plan. What is not said officially is whether or not an account that has been pushing a bit too often above the limit is blacklisted somehow with a lower threshold.

Not using the available libraries

Asana is offering online various client libraries for node, Java, PHP… Those client libraries know how to handle rate limit errors and retry after the correct duration. It is usually a good idea to use one of them instead of re-coding everything from scratch.

Any other ideas about why the API would complain about rate limits or the requesting might stop in the middle?

4 Likes
2

Bonjour!
Sorry for my english as I’m from Québec Canada…
But I’ll try to be clear!

We had to work hard in our application to solve the problems you mention.

The solution we developed is to have a single function where ALL get are managed. Every part of the app is always calling the api from that function. Then, I add every request in a queue. And, I create batch from that queue.

And, knowing that I can do 1500 call / minute as the rate limit, and knowing that 1 batch of 10 is count for 10 calls, that’s what I do:
I have an array to count how many requests did I do in the last minute (push(), then setTimeout pop() after 60 seconds).

Based on that count, I start my queries with 5 parallel threads of batch of 10, that is, 50 simultaneous query. Then, the more I go near my limit of 1500, I reduce batch size and threads. Each thead doing calls with batch of 2 to 10 items at a time.
The more calls I did in the last 60 seconds, the less threads I use and the smaller the batch are.

We don’t have any 429 using that method, and we always use the api to its maximum capacity.

That also make guid renewal easier to handle, when we get a guid expired, we pause all, renew it, relaunch the last query, and continue the batch.

But, it can be very long, when we need to query all details of subtasks also, sometimes it can last a few hours !!!

Also, we never request “names” of subitems. Like, assignee, users, projects, tags, … we only request ids from tasks, and we do separate calls to get subItems details. We can get most of users / projets / tags from a single call to get the complete list once, but sometimes, an old user that is no longer available can be listed in a “story” of 3 years old, then we need to handle it and query it alone. Same for projects / tags / custom fields / …

We did 100% ourself as it’s not very difficult, and we were able to handle that multi-thread our way.

Because we’re 100% client-side in browser (using indexedDB as cache engine), we don’t experience timeout, but we need to be clear to the user about the estimated duration of the action. It’s better to tell it will last 2 hours and do it in 90 minutes, than no telling anything and just wait with a loading icon without information about the progress of the task.

Finally, about the rate limit, I think that most users have access to 1500 query / minute, based on our access right on the workspace VS workspace status VS …

But, based on our experience, people who need our product have access to that 1500/min limit, I don’t think we never had issues with that limit being lower than this.

5 Likes
3

1 more thing… if you have too many 429, you will get that other “stop harassing us” message, and eventually, Asana can block completely your app to access the api, for that user, or your whole application. Before we do all these actions, some of our customers were black-listed from the Asana and we had to contact them to re-activate it! But it never happen again.

4 Likes
4

I recall either Zapier or IFTTT, in the past if not still ongoing, was unable to complete calls due to not respecting paging limits. Bastien reports that Integromat may also have a problem accessing the Asana API due to rate limits. Frederic had to write custom code to manage this, even beyond what Asana’s libraries offer, apparently. Perhaps if two of the major API connectors are having problems, and the recommendation everyone writes the same custom code as Frederic, there’s an opportunity to make this easier to develop with?

Thanks,

Larry

2 Likes
5

I don’t think they do, because the errors arrive after several hours apparently. But I wanted to leave that point here in case people work with other integrations.

Thanks @Frederic_Malenfant for the detailed answer!

3 Likes
6

These questions (and answers) are generic to any other integration with such a service as Asana. Thus, it is a good idea to reuse best practices from other integration services :

• queuing (already mentioned) is a first great idea
• parallelizing (also mentioned) is just like spawning web workers in parallel to do the job
• always check how you are doing versus the current limits : this is a job for the worker orchestrator, to throttle calls when necessary (also mentioned)

2 other points may help :
• compression : consider using the HTTPS header Accept-Encoding = gzip, to reduce volumes travelling over the wire, and thus performance
• asking for diffs rather than loading everything : use the modified_since query filter when getting tasks, for example. If you are syncing the systems every night, tasks do not change every day (especially completed tasks !)

I hope this helps.,
Rupert

3 Likes
7

Thanks @Rupert great points!

1 Like
8

Hi all, not sure where to get the latest on this but what is the current limit of call per minute? we are running a once of custom report and the api stops every now and then.

thanks
Jon

9

I am wondering if they don’t mention this in the doc. @Phil_Seeman any input?

10

@Jonathan_Stern @Bastien_Siebman
As I recall they didn’t used to have a lot of details and specific numeric limits specified in the developer docs, but they are pretty specific these days; see:

2 Likes
11

Thanks @Phil_Seeman and @Bastien_Siebman.
I think we had 1600 to do, we went around by adding a few ms sleep in between each calls.

by our calculations we should not have been affected by the limit. :upside_down_face:

12

That fix is a bit clunky, the best would be to listen to the Rate Limit error that gives you the time you are supposed to wait before retrying :+1:

1 Like