tibrewalpratik17 opened a new pull request, #12353: URL: https://github.com/apache/pinot/pull/12353
label: `bugfix` During downloadSegmentFromPeer : https://github.com/apache/pinot/blob/041e04078f5a94fca92c805a8db8fdf1f904a985/pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java#L596-L598 We saw that in some scenarios, the download request went to the same peer instead of having multiple replicas. Logs: ``` host1 WARN Download and move segment tableName__3__7940__20240201T1322Z from peer with scheme http failed. 2024-02-02T04:40:43.510+00:00 host1 WARN Caught exception while fetching segment from: http://host2:25602/segments/tableName_REALTIME/tableName__3__7940__20240201T1322Z to: /data/upinot/stream-pinot-server/dataDir/tableName_REALTIME/tmp/tmp-tableName__3__7940__20240201T1322Z.1706848841111/tableName__3__7940__20240201T1322Z.tar.gz 2024-02-02T04:40:43.504+00:00 host1 WARN Caught exception while downloading segment from: http://host2:25602/segments/tableName_REALTIME/tableName__3__7940__20240201T1322Z to: /data/upinot/stream-pinot-server/dataDir/tableName_REALTIME/tmp/tmp-tableName__3__7940__20240201T1322Z.1706848841111/tableName__3__7940__20240201T1322Z.tar.gz 2024-02-02T04:40:43.503+00:00 host1 INFO Received a request to download segment tableName__3__7940__20240201T1322Z for table tableName_REALTIME 2024-02-02T04:40:43.502+00:00 host1 WARN Caught exception while downloading segment from: http://host2:25602/segments/tableName_REALTIME/tableName__3__7940__20240201T1322Z to: /data/upinot/stream-pinot-server/dataDir/tableName_REALTIME/tmp/tmp-tableName__3__7940__20240201T1322Z.1706848841111/tableName__3__7940__20240201T1322Z.tar.gz 2024-02-02T04:40:41.583+00:00 host1 WARN Caught exception while fetching segment from: http://host2:25602/segments/tableName_REALTIME/tableName__3__7940__20240201T1322Z to: /data/upinot/stream-pinot-server/dataDir/tableName_REALTIME/tmp/tmp-tableName__3__7940__20240201T1322Z.1706848841111/tableName__3__7940__20240201T1322Z.tar.gz 2024-02-02T04:40:41.583+00:00 host2 INFO Received a request to download segment tableName__3__7940__20240201T1322Z for table tableName_REALTIME 2024-02-02T04:40:41.581+00:00 host1 WARN Caught exception while downloading segment from: http://host2:25602/segments/tableName_REALTIME/tableName__3__7940__20240201T1322Z to: /data/upinot/stream-pinot-server/dataDir/tableName_REALTIME/tmp/tmp-tableName__3__7940__20240201T1322Z.1706848841111/tableName__3__7940__20240201T1322Z.tar.gz 2024-02-02T04:40:41.273+00:00 host1 WARN Caught exception while fetching segment from: http://host2:25602/segments/tableName_REALTIME/tableName__3__7940__20240201T1322Z to: /data/upinot/stream-pinot-server/dataDir/tableName_REALTIME/tmp/tmp-tableName__3__7940__20240201T1322Z.1706848841111/tableName__3__7940__20240201T1322Z.tar.gz 2024-02-02T04:40:41.273+00:00 ``` This patch adds a round-robin logic to ensure we hit other peers as well to download replica. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
