navina commented on code in PR #9224:
URL: https://github.com/apache/pinot/pull/9224#discussion_r980752230
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java:
##########
@@ -544,16 +547,16 @@ private boolean processStreamEvents(MessageBatch
messagesAndOffsets, long idlePi
// Index each message
reuse.clear();
- // retrieve metadata from the message batch if available
- // this can be overridden by the decoder if there is a better indicator
in the message payload
- RowMetadata msgMetadata = messagesAndOffsets.getMetadataAtIndex(index);
-
- GenericRow decodedRow = _messageDecoder
- .decode(messagesAndOffsets.getMessageAtIndex(index),
messagesAndOffsets.getMessageOffsetAtIndex(index),
- messagesAndOffsets.getMessageLengthAtIndex(index), reuse);
- if (decodedRow != null) {
+ StreamDataDecoderResult decodedRow =
_streamDataDecoder.decode(messagesAndOffsets.getStreamMessage(index));
+ RowMetadata msgMetadata =
messagesAndOffsets.getStreamMessage(index).getMetadata();
+ if (decodedRow.getException() != null) {
+ // TODO: handle exception as we do today - do we silently drop the
record or throw exception?
Review Comment:
I am open to introducing a class of retriable and non-retriable exceptions
if we can clearly articulate them in a manner that is common to all decoders.
As of today, I only see this to be the case in avro decoder. Hence, for now, I
would leave this retry capability within the individual decoders and not
surface it in `LLRealtimeSegmentManager`.
We can iterate on it if the need arises in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
