error using python library to search tasks

Donald_Fast · July 10, 2023, 1:52pm

Hi,
The python Asana library is behaving weirdly. I am stepping through tasks a few days at a time so that I can do some historical reporting. This code will work for a few years then starts failing.

Issues

How do i fix the error “asana.error.InvalidRequestError: Invalid Request: completed_on.after: Should be before completed_on.before” (details below)
How do i add “new_goal_memberships” to your “Asana-Enable” or “Asana-Disable” header? I see nothing in the python library documentation to help
what is the syntax to get 100 (or more?) results at one time? i could not get limit to change getting max 50 results.

System Details: Python 3.11, Windows 11, Premium Asana

Code

    # start_date and end_date are both date variables.
    time.sleep(1.5) # Search can only be called 50 per minute
    task_results = client.tasks.search_tasks_for_workspace(WORKSPACE_GID, 
                {
                'completed_on.after':start_date,
                'text': 'visit', 
                'sort_by': 'completed_at', 
                'sort_ascending': 'true',
                'completed_on.before':end_date,
                }, 
                opt_pretty=True)

Output

daterange:2021-01-02 to 2021-01-07
storing: items num/overall:( 6/49)
daterange:2021-01-08 to 2021-01-13
storing: items num/overall:( 3/52)
daterange:2021-01-14 to 2021-01-19
storing: items num/overall:( 3/55)
daterange:2021-01-20 to 2021-01-25
storing: items num/overall:( 5/60)
daterange:2021-01-26 to 2021-01-31
Traceback (most recent call last):
File “c:\Users\donal\OneDrive\Documents\python dev\asana stats\get_asana.py”, line 94, in
find_tasks(start_date, end_date)
File “c:\Users\donal\OneDrive\Documents\python dev\asana stats\get_asana.py”, line 30, in find_tasks
store_results(task_results)
File “c:\Users\donal\OneDrive\Documents\python dev\asana stats\get_asana.py”, line 40, in store_results
for index, element in enumerate(task_results):
File “C:\Program Files\Python311\Lib\site-packages\asana\page_iterator.py”, line 58, in items
for page in self:
File “C:\Program Files\Python311\Lib\site-packages\asana\page_iterator.py”, line 40, in next
result = self.get_initial()
^^^^^^^^^^^^^^^^^^
File “C:\Program Files\Python311\Lib\site-packages\asana\page_iterator.py”, line 69, in get_initial
return self.client.get(self.path, self.query, **self.options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Program Files\Python311\Lib\site-packages\asana\client.py”, line 174, in get
return self.request(‘get’, path, params=query, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Program Files\Python311\Lib\site-packages\asana\client.py”, line 92, in request
raise STATUS_MAPresponse.status_code
asana.error.InvalidRequestError: Invalid Request: completed_on.after: Should be before completed_on.before

Bastien_Siebman · July 10, 2023, 3:48pm

Hi,

Can we see exactly the date you are passing for before and after params?

Looks like this in Javascript, if that helps.

asana.Client.create({
            defaultHeaders: {
                'Asana-Enable': 'new_user_task_lists',
                'Asana-Disable': 'new_project_templates'
            })

That’s expected, when running a search you have to play with the creation date of the tasks to be able to request the next batch. See Search tasks in a workspace

Donald_Fast · July 10, 2023, 4:07pm

dates as string look like:
start:2018-02-01 end:2018-02-06
start:2018-02-07 end:2018-02-12

When the query works, the dates are working with the right range.

The higher level function that loops through dates is:

start_date = datetime.date.fromisoformat(OVERALL_START_DATE)
next_year = start_date.replace(year=start_date.year + 1)
while start_date < datetime.date.today():
end_date = start_date + datetime.timedelta(days=NUM_DAYS_PER_QUERY)
if end_date >= next_year:
end_date = next_year - datetime.timedelta(days=1)
next_year = next_year.replace(year=next_year.year + 1)
find_tasks(start_date, end_date)

start_date = end_date + datetime.timedelta(days=1)

Here are the dates up to and including an error:

start:2021-01-20 end:2021-01-25
storing items num/overall:( 3/55)
daterange:2021-01-20 to 2021-01-25
start:2021-01-26 end:2021-01-31
storing items num/overall:( 5/60)
daterange:2021-01-26 to 2021-01-31
start:2021-02-01 end:2021-01-31
Traceback (most recent call last):
File “c:\Users\donal\OneDrive\Documents\python dev\asana stats\get_asana.py”, line 95, in
find_tasks(start_date, end_date)
File “c:\Users\donal\OneDrive\Documents\python dev\asana stats\get_asana.py”, line 31, in find_tasks
store_results(task_results)
File “c:\Users\donal\OneDrive\Documents\python dev\asana stats\get_asana.py”, line 41, in store_results
for index, element in enumerate(task_results):
File “C:\Program Files\Python311\Lib\site-packages\asana\page_iterator.py”, line 58, in items
for page in self:
File “C:\Program Files\Python311\Lib\site-packages\asana\page_iterator.py”, line 40, in next
result = self.get_initial()
^^^^^^^^^^^^^^^^^^
File “C:\Program Files\Python311\Lib\site-packages\asana\page_iterator.py”, line 69, in get_initial
return self.client.get(self.path, self.query, **self.options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Program Files\Python311\Lib\site-packages\asana\client.py”, line 174, in get
return self.request(‘get’, path, params=query, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Program Files\Python311\Lib\site-packages\asana\client.py”, line 92, in request
raise STATUS_MAPresponse.status_code
asana.error.InvalidRequestError: Invalid Request: completed_on.after: Should be before completed_on.before

Donald_Fast · July 10, 2023, 4:25pm

ah i found the answer to #1. a bug in my date generations has them generated backwards sometimes.

Anyone have answer to #2 and #3 ?
The syntax given above to add headers is not correct for python and i can’t find the right client function.

#3 - the limit is stated as 100, but i can only get 50 at a time

Bastien_Siebman · July 11, 2023, 6:05am

How do you create your python client? did you look at the python library source code.

To me, it doesn’t really matter, you have to loop through pages anyway even if they were 100 items long don’ t you?

Donald_Fast · July 11, 2023, 2:36pm

i thought i could find the answer without reading source code from the library.
After looking at the source code, i still don’t know how to change headers. Care to point in the right direction.
My python asana code is:

client = asana.Client.access_token(PERSONAL_ACCESS_TOKEN)
…
task_results = client.tasks.search_tasks_for_workspace(WORKSPACE_GID,
{
‘completed_on.after’:start_date,
‘text’: ‘visit’,
‘completed_on.before’:end_date,
},
opt_pretty=True)
…
result = client.tasks.get_task(task_gid)

limit of 50 means the program makes twice the calls and takes twice as long to run. Getting data for 4 days at a time for 5 years means close to 10 minutes runtime. (as the search task can only run 50 times/minute, even on paid platform).

Doubling that limit to 100 would halve the number of calls and halve the time.

Bastien_Siebman · July 12, 2023, 6:45am

I am surprised you don’t have a create step like me asana.Client.create(...)

I don’t see in your code passing a value for limit

Donald_Fast · July 12, 2023, 3:45pm

I followed the code examples and they did not have asana.client.create step
whatever syntax i tried for limit made no changes so i removed it. What is the correct syntax?

task_results = client.tasks.search_tasks_for_workspace(WORKSPACE_GID,
{
‘completed_on.after’:start_date,
‘text’: ‘visit’,
‘completed_on.before’:end_date,
},
opt_pretty=True, limit=100)
or
task_results = client.tasks.search_tasks_for_workspace(WORKSPACE_GID,
{
‘completed_on.after’:start_date,
‘text’: ‘visit’,
‘completed_on.before’:end_date,
‘limit’:‘100’
},
opt_pretty=True)
or something else?

Bastien_Siebman · July 12, 2023, 4:05pm

The GitHub gives you the syntax: GitHub - Asana/python-asana: Official Python client library for the Asana API v1

Bastien_Siebman · July 12, 2023, 4:08pm

Maybe you are not using the latest version because I don’t see the method you used but a different one…

Donald_Fast · July 12, 2023, 5:15pm

Thanks for providing the github link with better documentation than on Asana website. It would be helpful if the Asana documentation pointed here.

I installed with “pip install asana”. The version installed is 3.2.1 – the same version as on github.

the function i call is in tasks.py, line 351:

def search_tasks_for_workspace(self, workspace_gid, params=None, **options):

Donald_Fast · July 12, 2023, 5:18pm

PS - my tasks.py does not have search_in_workspace function

Bastien_Siebman · July 13, 2023, 6:23am

So you got the latest lib but it doesn’t have the function we see on GitHub?
Anyway, the limit is definitely part of the search parameters… I always assumed it worked but maybe all this time I was not getting 100 results

Donald_Fast · July 13, 2023, 3:56pm

Hmm is right. I used python pip to install. i don’t know how to tell where pip sourced the files.

Strange to have 2 versions.

For limit, i tried the other way and limited to 5 results for testing. It always returned more. The git docs refer to using item_limit:# so i will try that later. Note the Asana doc refers to the parameter as limit as per my above code

Bastien_Siebman · July 15, 2023, 8:28am

I always refer back to the source code in cases like these because the doc might not be aligned with the releases…