Question about search api + paging

We plan to use your search api for very specific queries, like “unassigned tasks” or "floating tasks (not in any project).

My question is…

Can I trust the “sort_by” ?
You mention that we can use sort_by parameter, and the default value is “modified_at”.
I tried it, and look at some results I got:
(query 1)
“2019-07-12T06:40:56.938Z”
“2019-07-12T06:40:54.111Z”
“2019-07-12T06:40:51.856Z”
“2019-07-12T06:40:52.185Z” <---- ???
“2019-07-12T06:40:39.872Z”

So, it looks like I can’t use that method for paging.

You also mention that we should use created_at for paging, instead of modified_at.
I tried it with the same data, it looks ok.
(query 2)
“2019-07-12T06:40:54.111Z”
“2019-07-12T06:40:52.550Z”
“2019-07-12T06:40:52.185Z”
“2019-07-12T06:40:51.856Z”
“2019-07-12T06:40:49.754Z”

Is it always “safe” to use the created_at? I tried a few queries, and results looks fine, but I’m not 100% sure they are always.

I really prefer to use the modified_at, as I can compare with my local cache and stop querying when I reach an in-cache task. I can’t do that using created_at ordering.

Did I found a bug, or is it a known issue that we must live with?

1 Like

In short: yes, it is always safe to use created_at to “paginate” in the search API.

The reason there appears to be a discrepancy in the modified_at is because we actually have two “modified at” values per task. There is the raw modified_at field we expose in the API, which reflects the last time the object was updated in the database. Additionally, there’s a “user visible modification time” which reflects the last time the object was updated in a way that would appear as a change to users of the Asana web app. This is because there are certain changes to objects that are not directly visible, and so aren’t useful when web app users want to know what’s been modified recently.

Our search API operates the same way as our web app advanced search feature, and the web app search uses the “user visible modification time” which is not always in sync with the true “modified at” time. What’s happening here is that the search cluster has filtered and ordered the results by one of these timestamp fields, but the API has returned the other timestamp in the JSON object, giving the appearance of out-of-order/inconsistent results.

2 Likes

Do you plan any changes in this mechanism? I want to use modified_at for pagination, because if i would use creaed_at i wil always have to get all the tasks from the scratch - i have thousands of them.
If i can use modified_at for pagination i will always get only changed tasks from the last query date, not all for the projects.

Do you have any news here?

Hi @Nikita_Popov,

No, we do not plan to change this. However, you can both paginate by created_at and filter by modified_at in the same request. This will allow you to perform stable pagination through the search results and see only things that have changed recently.

Ok, it is very sad.
I am not sure that your option will help me - i am still need to get all the tasks. It will not return me the proper order: if i modify any task - it will not move in the order in such way - first sort will be created_at and it will stay in its first order. So, i need to get all the tasks as earlier.