tarun11Mavani commented on issue #15846:
URL: https://github.com/apache/pinot/issues/15846#issuecomment-2903300185
> What is the difference between the proposed solution vs the existing
behavior?
>
> Can we use the creation time set by the controller when the CONSUMING
segment is created? That should be consistent across all replicas.
@Jackie-Jiang the proposed solution is to fetch creation time of each
segment replica from server and use that instead of using the creation time
from segment metadata. During my testing, I noticed that the creation time of
the segment replica differed by ~1 or 2 seconds.
Example logs:
Server 1:
`Resolving comparison tie for with primary key:
[06321d2b-1c14-42a8-b459-c130b4bcb6d6], new segment:
compacted__rta_tarunm_hcv_merge_task_3__2__1747647004707__0 and current
segment: rta_tarunm_hcv_merge_task_3__2__4__20250519T0907Z with creation time:
1747646240006 and 1747646240006, creationTimeComparisonRes: 0,
UploadedRealtimeSegmentName.of(currentSegmentName): null,
UploadedRealtimeSegmentName.of(segmentName):
compacted__rta_tarunm_hcv_merge_task_3__2__1747647004707__0`
New Segment: compacted__rta_tarunm_hcv_merge_task_3__2__1747647004707__0:
1747646240006
Current segment: rta_tarunm_hcv_merge_task_3__2__4__20250519T0907Z:
1747646240006
Server 2:
`Resolving comparison tie for with primary key:
[06321d2b-1c14-42a8-b459-c130b4bcb6d6], new segment:
compacted__rta_tarunm_hcv_merge_task_3__2__1747647004707__0 and current
segment: rta_tarunm_hcv_merge_task_3__2__4__20250519T0907Z with creation time:
1747646240006 and 1747646241130, creationTimeComparisonRes: -1,
UploadedRealtimeSegmentName.of(currentSegmentName): null,
UploadedRealtimeSegmentName.of(segmentName):
compacted__rta_tarunm_hcv_merge_task_3__2__1747647004707__0`
New Segment: compacted__rta_tarunm_hcv_merge_task_3__2__1747647004707__0:
1747646240006
Current segment: rta_tarunm_hcv_merge_task_3__2__4__20250519T0907Z:
1747646241130
As you can see, the segment
`rta_tarunm_hcv_merge_task_3__2__4__20250519T0907Z` has different creation time
on both server.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
