tibrewalpratik17 commented on issue #11736: URL: https://github.com/apache/pinot/issues/11736#issuecomment-1758644725
@Jackie-Jiang @deemoliu referring to #11779, we do not intend to leverage the watermark feature of upsertTTL to address this use case. We would require loading all segments' validDocIds (not just after the watermark window). However, given that we mark deletedKeys validDocIds as invalid, these keys will not be recreated in the HashMap, minimizing the risk of server crashes during restarts or replacements. There is a specific scenario to consider: only the deletedKeys from the last purge may get loaded again since the invalidation operation of those validDocIds may not have been snapshotted. Nevertheless, these keys will be purged in the subsequent run. But do point out if i am missing something trivial here which will not fulfill to what we need. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
