Jackie-Jiang opened a new pull request, #15404: URL: https://github.com/apache/pinot/pull/15404
Currently the consumer lock is acquired during the `OFFLINE -> CONSUMING` state transition. If the lock cannot be acquired, helix thread will be blocked, and end up causing starvation. With #15261 introducing dependency among segments, the chance of starvation increases. This PR moves the lock acquisition into the consumer thread so that it won't block the helix state transition. This PR also modifies the consumer coordinator logic: - Only use ideal state to calculate the previous sequence number, always use watermark to determine whether previous segment is registered - Fix a bug where deleted segment is not properly handled -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
