avshenuk commented on issue #12390:
URL: https://github.com/apache/pinot/issues/12390#issuecomment-3178162959
Got the same problem on one of the tables after running the cluster not more
than for 24 hours.
```
2025/08/12 01:01:44.443 ERROR
[RealtimeSegmentDataManager_recs_events__39__3__20250811T1700Z]
[HelixTaskExecutor-message_handle_thread_6] InterruptedException when acquiring
the partitionConsumerSemaphore
2025/08/12 01:01:48.743 INFO [com_8098 - SegmentOnlineOfflineStateModel]
[HelixTaskExecutor-message_handle_STATE_TRANSITION]
SegmentOnlineOfflineStateModel.onBecomeConsumingFromOffline() :
ZnRecord=4fa5dce0-8eed-4f37-8393-5a82ffdf7166, {CREATE_TIMESTAMP=1754960507883,
ClusterEventName=CurrentStateChange, EXECUTE_START_TIMESTAMP=1754960508743,
EXE_SESSION_ID=1018c12e46451d1, FROM_STATE=OFFLINE,
MSG_ID=4fa5dce0-8eed-4f37-8393-5a82ffdf7166, MSG_STATE=read,
MSG_TYPE=STATE_TRANSITION, PARTITION_NAME=recs_events__39__3__20250811T1700Z,
READ_TIMESTAMP=1754960508031, RESOURCE_NAME=recs_events_REALTIME,
RESOURCE_TAG=recs_events_REALTIME, RETRY_COUNT=3,
SRC_NAME=pinocube-pinot-controller-0.pinocube-pinot-controller-headless.sandbox.svc.de-dev.cxense.com_9000,
SRC_SESSION_ID=1018c12e46401e6,
STATE_MODEL_DEF=SegmentOnlineOfflineStateModel,
STATE_MODEL_FACTORY_NAME=DEFAULT,
TGT_NAME=Server_pinocube-pinot-server-0.pinocube-pinot-server-headless.sandbox.svc.de-dev.cxense.com_8098,
TGT_SESSION_ID
=1018c12e46451d1, TO_STATE=CONSUMING}{}{}, Stat=Stat {_version=0,
_creationTime=1754960507907, _modifiedTime=1754960507907, _ephemeralOwner=0}
2025/08/12 01:01:48.744 INFO [recs_events_REALTIME-RealtimeTableDataManager]
[HelixTaskExecutor-message_handle_STATE_TRANSITION] Adding CONSUMING segment:
recs_events__39__3__20250811T1700Z
2025/08/12 01:01:48.762 INFO [recs_events_REALTIME-RealtimeTableDataManager]
[HelixTaskExecutor-message_handle_STATE_TRANSITION] Adding new CONSUMING
segment: recs_events__39__3__20250811T1700Z
```
in the logs.
And segment is permanently stuck (reload/reset) doesn't fix it:
<img width="1512" height="818" alt="Image"
src="https://github.com/user-attachments/assets/0af61c6c-f856-419c-9da0-7a94999be981";
/>
The consumption is stopped because of this too as no consuming segment was
actually added:
<img width="617" height="213" alt="Image"
src="https://github.com/user-attachments/assets/f35c5e41-1586-4714-9906-5c4492514011";
/>
Sounds like a quite serious issue that could cause problems on production
for many folks?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
