noob-se7en opened a new issue, #17209:
URL: https://github.com/apache/pinot/issues/17209
**Problem**
When shards are merged or reduced in Kinesis, Servers send segment consumed
message to Controller with reason: endOfPartition reached.
Controller during commit process can incorrectly determine partition has not
reached its end and creates new consuming segment. The servers immediately
commit that new segment. This process keeps on repeating.
Error log:
```
2025/11/11 18:41:00.041 WARN [KinesisStreamMetadataProvider]
[grizzly-http-server-0] Reached max attempts to check if end of shard reached
from checkpoint {"shardId-000000000000":"<>"}. Assuming we have not consumed
till end of shard.
```
Code to be re-visited:
https://github.com/apache/pinot/blob/844d4ab8f9fcdc8fa71ffd750ac31ee858a815d0/pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisStreamMetadataProvider.java#L228-L257
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
