sajjad-moradi opened a new pull request, #11476: URL: https://github.com/apache/pinot/pull/11476
## Description This PR handles the bug described in issue #11475 . ## Testing Done Verified the correct behavior by changing `LLCRealtimeClusterIntegrationTest`: - created a Kafka topic with 2 partitions - created a table with partitioning enabled (partition count was set as 6 to not match the stream partition count) - published 4 events with id's [0, 1, 2, 3] to the topic - verified that partitioning info in segment ZK metadata is correctly set based on 2 partitions, not 6 - increased the Kafka topic partition count to 4 - published 4 more events with id's [4, 5, 6, 7] - at this point, there's only two consuming segments. Some of the new events went to the existing consuming segments, for some there's no consuming segment available. - kicked off `RealtimeSegmentValidationManager` periodic job to create new consuming segments - verified that for the new consuming segments, `numPartition` field in segment ZK metadata is correctly set as 4, while for the existing consuming, the value is 2. - queried the table with different id's in where clause to make sure that partition pruning / routing works as expected - issued a force-commit - verified that partition info for committed segments as well as new consuming segments are correctly set Note: I think it's best if later we put all the above steps in a separate integration test, so we make sure the expected behavior doesn't change with future refactoring. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
