I need to get every task and subtask for every project in my team.
My current approach is to get all projects for my team using .projects.get_projects(team=team['gid'], opt_fields=fields) then recursively get the tasks and subtasks for each project using .tasks.get_subtasks_for_task(task['gid'], opt_fields=fields)
Unfortunately, using this method and just getting the compact version of each task/subtask can take over an hour due to the sheer number of requests it takes to pull all the data.
I’m sure there’s a better way to do this, but I’m not sure which to use. I’m all for reworking how I store the data from the ground up if it means I can reduce the overall retrieval time.
I don’t think there’s a better method to query all data you need.
Some things you can do to help make the process faster
First, in your “opt_fields”, make sure you add the “num_subtasks” field.
Then, only query subtasks that has num_subtasks > 0.
Also, you can use the “batch” endpoint, that allows you to query up to 10 subtasks at a time using only 1 request (but it will count as 10 query in your quota)
Finally, you can check for recursivity, as each subtask can also have subtasks, up to 5 levels deep.
But, there’s no miracle solution, it will take a very long time!
Thanks for the info. Not great news, but I’ll take it.
A few things, though.
The batch system, while I’m familiar with the concept, I’ve always been confused on how to implement and I can’t find any working examples to use as a reference. Do you happen to have any references I could look at?
Quota? What quota?
I already look recursively for subtasks of subtasks, hence why it takes over an hour to retrieve them. It only takes about 15 minutes if I stick to the top level of tasks. I have considered using the num_subtasks field, but I’m not convinced that would be faster than just trying to pull subtasks as-is. I’ll have to try it out and see if there’s a significant difference.
If 95% of your tasks have subtasks, you are right. But no doing a call is faster than doing it?
If your parent tasks indicates that there’s no subtasks, why query them, when you know that the answer is an empty array?