sajjad-moradi opened a new pull request, #9197: URL: https://github.com/apache/pinot/pull/9197
## Description For realtime segments, if there's any changes to stream configs in the table config, the changes don't get picked up immediately in the existing consuming segments. After the existing consuming segments complete - which may take hours depending on flush thresholds defined in table config - the new consuming segments will pick up the stream config changes. In this PR, the force commit functionality that was introduced in pause/resume feature (#8986) is used in a controller endpoint to reset the consumption of a realtime table. After "consumption reset" is issued, the current consuming segments will be forced to commit immediately and then the new consuming segments will pick up any changes in the table config. ## Testing Done Verified the expected behavior using `LLCRealtimeClusterIntegrationTest`. During ingestion, I added a consumption rate limit parameter to the table config and then used `resetConsumption` endpoint and verified that the existing consuming segments completed and then the new consuming segments consumed according to the specified consumption rate limit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
