swaminathanmanish commented on code in PR #13036:
URL: https://github.com/apache/pinot/pull/13036#discussion_r1585479559
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java:
##########
@@ -1504,14 +1510,24 @@ public RealtimeSegmentDataManager(SegmentZKMetadata
segmentZKMetadata, TableConf
// Create message decoder
Set<String> fieldsToRead =
IngestionUtils.getFieldsForRecordExtractor(_tableConfig.getIngestionConfig(),
_schema);
+ RetryPolicy retryPolicy = RetryPolicies.exponentialBackoffRetryPolicy(5,
1000L, 1.2f);
Review Comment:
Curious how we picked these values ?
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java:
##########
@@ -1504,14 +1510,24 @@ public RealtimeSegmentDataManager(SegmentZKMetadata
segmentZKMetadata, TableConf
// Create message decoder
Set<String> fieldsToRead =
IngestionUtils.getFieldsForRecordExtractor(_tableConfig.getIngestionConfig(),
_schema);
+ RetryPolicy retryPolicy = RetryPolicies.exponentialBackoffRetryPolicy(5,
1000L, 1.2f);
+ AtomicReference<StreamDataDecoder> localStreamDataDecoder = new
AtomicReference<>();
Review Comment:
is this done because only a reference is allowed in lambda?
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java:
##########
@@ -1504,14 +1510,24 @@ public RealtimeSegmentDataManager(SegmentZKMetadata
segmentZKMetadata, TableConf
// Create message decoder
Set<String> fieldsToRead =
IngestionUtils.getFieldsForRecordExtractor(_tableConfig.getIngestionConfig(),
_schema);
+ RetryPolicy retryPolicy = RetryPolicies.exponentialBackoffRetryPolicy(5,
1000L, 1.2f);
+ AtomicReference<StreamDataDecoder> localStreamDataDecoder = new
AtomicReference<>();
try {
- StreamMessageDecoder streamMessageDecoder =
createMessageDecoder(fieldsToRead);
- _streamDataDecoder = new StreamDataDecoderImpl(streamMessageDecoder);
+ retryPolicy.attempt(() -> {
+ try {
+ StreamMessageDecoder streamMessageDecoder =
createMessageDecoder(fieldsToRead);
+ localStreamDataDecoder.set(new
StreamDataDecoderImpl(streamMessageDecoder));
+ return true;
+ } catch (Exception e) {
+ return false;
Review Comment:
Could you log the exception as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
