KKcorps opened a new pull request, #11027: URL: https://github.com/apache/pinot/pull/11027
This PR completes the bug fix introduced with https://github.com/apache/pinot/pull/10704 With the fix introduced previously, although we merge the comparison column values, we still mark the original columns as null in the bitmap so that it can help during restart. However, during the segment build, we simply filter out all the columns which have null set in the bitmap and thus we lose all these values. ## Reproducing the bug * create a partital upsert table with two comparison columns `mtime` and `mtime_2` * publish the following two records to the table ```jsonl { "rsvp_count": 23, "venue_name": "Venue A", "event_id": "E12345", "event_time": 1688372645422, "group_city": "San Francisco", "group_country": "USA", "group_id": 1234567890, "group_name": "OpenAI enthusiasts", "group_lat": 37.7749, "group_lon": -122.4194, "mtime_2": 1687341314322 } { "rsvp_count": 23, "venue_name": "Venue A", "event_id": "E12345", "event_time": 1688372645422, "group_city": "San Francisco", "group_country": "USA", "group_id": 1234567890, "group_name": "OpenAI enthusiasts", "group_lat": 37.7749, "group_lon": -122.4194, "mtime": 1687341314100 } ``` * verify the records are showing up correctly in the table. Then do either reload or force commit. * You will see that no records show up in the table now. If you use `skipUpsert(true)` though, you will be able to see everything -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
