NodeJS client returning duplicates in streams, going over the page limit


I am using the nodeJS client library to query the search endpoint. Even though I asked for 50 tasks, and the API gave indeed 50 tasks, the nodeJS library, through the stream mechanism, was giving me 51 tasks, with one being a duplicate of another one…

Does it ring a bell?

Hmm, that’s curious. This is reproducible every time? (i.e. it’s not some sort of one-off race condition)?

The streaming code is frankly a bit convoluted - we have async promises waiting on a buffer that may or may not be reading from another buffer that may or may not be filling from pages in our API, if I read the code correctly - so it’d take some time to really dig into what’s going on here, so I’m curious what the “smell” of this potential bug might be.

It happened for some projects, but for those it happened every time…