Problems with Asana API rate limitations when accessing sub-tasks

[Christoph]

I am using Asana with several other services which query my Asana tasks and projects via the Asana API. After many months I have been engeged with support from these services in order to fix some problems I had with how my Asana tasks were synced across these services. One thing that turns out to be the source of multiple troubles is how subtasks are accessed via the Asana API. Here is what one developer said to me when explaining why they cannot sync sub-tasks (or sub-sub-tasks) from Asana:

To request subtasks, we have to make X extra requests (where X is the number of tasks) via their [Asana’s] API, which covers almost any limit. Wunderlist, GTasks, Trello etc lets us get at least the info “does the task have any subitems”, or even provide full picture at once. Thus it depends on Asana API team or their restrictions: as soon as they’ll let us request the subtasks without reaching the request limits, we’ll add it to our pipeline.

So, as I understand it, this means that the Asana API is too ineffective when it comes to tasks lower down in the hierarchy. It takes to many API calls to even get information about their existence.

At another service, I was having huge problems with inconsistent behaviour when it comes to sub-tasks and it took the developers quite some time to discover that they were often hitting Asana’s API call rate limits when querying sub-tasks and failed to properly handle that on their side. So while that failure to handle the rate limitation correctly is their fault, I wonder whether such issues could not be avoided if Asana provided a better way of querying sub-tasks (and sub-sub-tasks)?

[Phil_Seeman]

Hi @Christoph,

FYI I moved your post to the “Developers & API” section of the forum since it’s most likely to be seen by the Asana API team there.

In general I can say that as a 3rd party developer I handle the rate limit and have not to-date had any issues in accessing lots of tasks and subtasks. I don’t think it should be an issue if these services properly handle the rate limit.

The Asana API folks may also want to weigh in here.

[Frederic_Malenfant]

Hi @Christoph, you are right that, by default, it can be hard to see if a task have subtasks. for 100 tasks, you need to do 100 “get” just to know.

But… we found a way in our app to know which task have subtasks, without querying them one by one.

Example: we get tasks for project 123, with:
GET /projects/123/tasks.

By default, we get 4 fields per task returned, “gid/id” + name + resource_type.
But, you can ask for more with the “fields” parameter.
We usually use that field parameter to get “root” fields, but it can also be used to query sub-items.

So, your “field” parameter can look like:
fields=id,name,due_on,subtasks.id

(in querystring, it’s ?opt_fields=…)

Now, in your data data response, each task will also contains an array of subtasks object with the id field only.

But, do not abuse… only ask for id and call for others fields only if you really need them after, as your response can be too large, or very long to execute.
And, with abuse comes features disappearing from the API without warning, as they did recently for the stories…

By using this, we were able to make bridge24.com faster when querying for subtasks.

But, I agree that they should provide information fields like “subtask_count”, “attachments_count”, or else to make all our and their app faster.

[Phil_Seeman]

@Frederic_Malenfant, @Christoph,

Yes, I knew about (and used to use) the method that Frederic describes; but I didn’t mention it in my reply to Christoph because it’s not officially supported and Asana does not want us to use it - see:

Enable to fetch sub-task details without task id

Accessing subtasks as attributes of a task by using opt_expand=subtasks or similar is not officially supported in the API because it frequently causes stability issues. The only supported way to fetch the subtasks of a task is to call GET /tasks/task-id/subtasks. We’re aware that fetching all subtasks in this way is painful, and we are thinking about additional alternatives that would be easier to use.

[Joe_Trollo]

Hi @Christoph, the API team here is actively in communication with Unito about the subtask syncing issue. The trouble here is that Unito is using a deprecated method of fetching subtasks that is even worse than the method described by @Frederic_Malenfant, and we’re working to transition them to using the only supported method: calling GET /tasks/<task-id>/subtasks.

[Frederic_Malenfant]

Hi @Joe_Trollo, I hope that if you disable the ability to query subtasks using opt_fields, there will be an efficient alternative, like having the “subtask_count” in the task, to prevent tons of calls to /tasks/xyz/subtasks that will returns empty results!
We don’t want to make useless calls that can take long time for nothing, and you want to keep the charge on your api as low as possible… so I think that having these indicators can helps both of us. assignee_count (or ids?), subtasks_count, attachments_count, …
Thank you!

[Christoph]

@Joe_Trollo

the API team here is actively in communication with Unito about the subtask syncing issue.

Haha, good to know that even more services encounter this issue. I was referring to TimeCamp.

@Joe_Trollo

we’re working to transition them to using the only supported method: calling GET /tasks/<task-id>/subtasks .

I don’t know how Timecamp are doing it, but I assume that they already do it that way. But the fact that this (presumably - I’m not a dev and have not studied the API) only returns subtasks but not sub-subtasks etc seems to be the core of the issues I’m describing in the OP. The thing is: the further down in the task-hierarchy you get, the broader the task tree gets so the more you actually want to get a list of all children, grandchildren and so on with a single csll. But as I understand it, the opposite is the case: the further down you get, the more individual calls you need to do, thus, the further down you get in the task tree, the more likely you are going to hit the rate limit.

@Joe_Trollo

even worse than the method described by @Frederic_Malenfant

But it seems to me that “official way” is not necessarily better than the inofficial one, no?

I apologise in advance if what I’m saying is just nonsense. I’m just trying to get to the heart of the problem.

Anyway, what is the message here to Pomodone (who are not syncing sub-tasks (or was it sus-subtasks?) for the reasons explained above?

[Joe_Trollo]

@Christoph

I was referring to TimeCamp.

Ah, sorry! This forum post came through at the same time as an internal communication about Unito, and there’s been a recent history of users writing to both our support channels and this forum at the same time so I mistakenly assumed they were related.

@Christoph

The thing is: the further down in the task-hierarchy you get, the broader the task tree gets so the more you actually want to get a list of all children, grandchildren and so on with a single csll.

Yes, exactly—the bigger the tree, the less developers want to traverse it manually. But, the bigger the tree, the more computationally expensive it is to provide the entire thing in a single API response. Currently our options are either (1) risk the stability of the API for a large portion of our users or (2) require that apps make more manual requests, and the trade-off between those two has the Asana API leaning heavily away from risking stability.

@Christoph

But it seems to me that “official way” is not necessarily better than the inofficial one, no?

The “official way” is far better in that it does not cause any stability issues, but is unfortunately more difficult to work with (because of having to make so many API calls). We are tracking how often developers encounter this difficulty, and are considering some solutions to see if any of them are technically stable and also solve the needs of our developers.

I’ll have to do more investigation for this specific case to find out what issue TimeCamp was encountering before I can make any recommendation.

[Christoph]

@Joe_Trollo

The “official way” is far better in that it does not cause any stability issues, but is unfortunately more difficult to work with (because of having to make so many API calls).

I guess the question is, what “more difficult” exactly means. Does it “just” require some sophisticated programming (on the third party side) and once that is accomplished, things will work smoothly, or does it mean that whatever you do, you will always hit rate limits if you want to get an overview of your task tree?

[Joe_Trollo]

Using GET /tasks/<task-id>/subtasks takes a little more work to get all the subtasks in the tree because the app has to manage multiple calls to this endpoint and reconstruct the tree client-side. It’s straightforward, but more work.

We can never guarantee that you won’t hit rate limits anyway, because we can’t know what else the app is doing at the same time. (Maybe it’s downloading multiple projects at once, or exceeding our concurrent request limits.) However, limited to just the action of “downloading all tasks and subtasks in a project” then using the “official way” is far far less likely to encounter rate limits.

[Joe_Trollo]

We have just shipped a new field on tasks that will let your app know how many subtasks there are. This allows you to avoid making calls to GET /tasks/<task-id>/subtasks if they would return an empty response.

This field is called num_subtasks and is not included by default—you must use opt_fields to select it.