Jackie-Jiang commented on PR #10702: URL: https://github.com/apache/pinot/pull/10702#issuecomment-1529149862
> @Jackie-Jiang and myself discussed offline, and we have agreed to iterate on this PR -- and look on solving the snapshot of validDocID map issues. > > The main issue that I wish to avoid is the separate map for queryable docIDs. This is not what I said.. I said not to directly close this PR and see if it is possible to solve the problems, and if @navina cannot get an implementation sticking with your timeline we can go with this one. Since Navina already gets an implementation #10703 and I feel that can solve all the existing problem with the overhead of keeping 2 bitmaps per segment, I'm leaning towards that solution as I don't see a good way to solve the snapshot issue with one single bitmap. The main problem with one single bitmap is that it breaks the contract that all the keys in the upsert metadata should point to a valid doc in the segment, and if we break that contract a lot of things could go wrong. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
