klsince commented on issue #11948: URL: https://github.com/apache/pinot/issues/11948#issuecomment-1815293485
hey @jhyao thanks for testing that out on your side. Yes, I also noticed the overcounting issue while testing the new changes. Basically, the realtime data ingestion can continue to update existing segments (like the upsert validDocId bitmaps used to identify which docs are updated in each segment) while the query is processing those segments in parallel (and without a deterministic segment processing order). The query could overcount or undercount, depending on the order of segment processing and whether new records got ingested and invalidated existing docs. But this is different from the issue to be fixed by PR https://github.com/apache/pinot/pull/11978, because no segments were missed just that they got updated during query execution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
