Search API: Lack of pagination leads to limited results

Thanks in advance.

I’ve read through the docs, and in the process of writing some code to use the search API. While writing some tests, I discovered a scenario that leads to lack of information.

We are using the search API to filter through tasks that fall under a particular section, and have a particular modified_at.after & modified_at.before window. We implement a sliding window of time to retrieve more tasks that may exist but are not returned since we are limited by 100 tasks, and no ‘next_page’ field is returned by the search API. This is fine, but then I realized that there exists a scenario the sliding window implementation will not work, and we can not use any other existing filters to capture other tasks.

Let’s say section 1 contains a 101 tasks, and by some chance all of the these tasks were modified at the same time by different users. With my current filter and sliding window implementation, I will never be able to get the 101th > task.

I know the scenario is unlikely, but even unlikely things happen. What do you guys propose I do to capture those hanging tasks?

You should consider using the created_at field as a pivot for your pagination, that will not be impacted by actions from users inside the app. Does it help? :thinking:

1 Like