tarun11Mavani opened a new pull request, #16034: URL: https://github.com/apache/pinot/pull/16034
### Problem Fixes data inconsistency issue discovered in #15846, where different segment creation times across replicas lead to inconsistent shouldReplaceOnComparisonTie() behavior during merge compaction operations. This results in: - COUNT() queries returning different results depending on the server - Primary key lookups sometimes returning records, sometimes none - Data inconsistency after segment deletions during compaction ### Root Cause When realtime segments are committed, slower replicas can have creation times that differ from faster replicas by seconds. During upsert operations, shouldReplaceOnComparisonTie() uses creation time as a tiebreaker, causing different replicas to make different replacement decisions for the same records. ### Solution Implements @klsince's suggestion from issue #15846 to fix creation time consistency during segment commit process: - Added creation time consistency check in RealtimeTableDataManager.replaceConsumingSegment() - Fixes creation.meta before segment loading - ensures SegmentMetadata objects get initialized with consistent creation times ### Testing [Still in progress] Will test this in a test cluster with multiple replica and share the results soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
