deepthi912 opened a new pull request, #17324: URL: https://github.com/apache/pinot/pull/17324
Continuation to: https://github.com/apache/pinot/pull/17144 This PR is a long term fix without having to disable reload/force commit for Partial Upserts and Tables with dropOutOfOrder=true. In this PR, the following solution is implemented to revert the metadata of the current segment incases of inconsistencies. Note that, we do not fix any inconsistencies that has happened in the past, the inconsistencies will only be fixed starting from now. **During Consumption:** - We keep track of the current segment's pk -> RecordLocation of the previous location that is existing only in the other segment During segment replacement when the metadata inconsistencies are detected: - Update the existing pk -> Record Location map with the values in the temporary previous record location map being stored for each consuming segment. - This map is erased once the segment is added/ replaced. The map will be started empty for each new consuming segment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
