navina commented on code in PR #8897:
URL: https://github.com/apache/pinot/pull/8897#discussion_r898716958
##########
pinot-plugins/pinot-stream-ingestion/pinot-pulsar/src/main/java/org/apache/pinot/plugin/stream/pulsar/PulsarMessageBatch.java:
##########
@@ -105,4 +119,26 @@ public StreamPartitionMsgOffset
getNextStreamPartitionMsgOffsetAtIndex(int index
public long getNextStreamMessageOffsetAtIndex(int index) {
throw new UnsupportedOperationException("Pulsar does not support long
stream offsets");
}
+
+ /**
+ * Stitch key and value bytes together using a simple format:
+ * 4 bytes for key length + key bytes + 4 bytes for value length + value
bytes
+ */
+ private byte[] stitchKeyValue(byte[] keyBytes, byte[] valueBytes) {
Review Comment:
I feel like this is the case for other stream connectors too. I don't think
the decoders have access the message key or message headers today.
A more elegant approach maybe to use `MessageBatch<StreamMessageType>`,
where StreamMessageType can contain payload and metadata. But I suspect you
want to avoid a more invasive code change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
