Re: [PR] byte-based splitting for mse query blocks [pinot]

albertobastos commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2071398844 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/GrpcSendingMailbox.java: ## @@ -156,19 +176,30 @@ private StreamObserver getContentObserver()

Re: [PR] byte-based splitting for mse query blocks [pinot]

Jackie-Jiang commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2069226305 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/GrpcSendingMailbox.java: ## @@ -156,19 +176,30 @@ private StreamObserver getContentObserver()

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz merged PR #15571: URL: https://github.com/apache/pinot/pull/15571 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apach

Re: [PR] byte-based splitting for mse query blocks [pinot]

albertobastos commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2068400368 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java: ## @@ -1508,6 +1508,13 @@ public static class MultiStageQueryRunner { public stat

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2068389246 ## pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java: ## @@ -1508,6 +1508,13 @@ public static class MultiStageQueryRunner { public static fina

Re: [PR] byte-based splitting for mse query blocks [pinot]

albertobastos commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2068026931 ## pinot-query-runtime/src/test/java/org/apache/pinot/query/mailbox/MailboxServiceTest.java: ## @@ -87,7 +87,7 @@ public void testLocalHappyPathSendFirst() S

Re: [PR] byte-based splitting for mse query blocks [pinot]

albertobastos commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2067999120 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/GrpcSendingMailbox.java: ## @@ -52,24 +58,32 @@ public class GrpcSendingMailbox implements Se

Re: [PR] byte-based splitting for mse query blocks [pinot]

yashmayya commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2067962738 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java: ## @@ -252,7 +252,7 @@ public static DataBlock readFrom(ByteBuffer buffer) pub

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on PR #15571: URL: https://github.com/apache/pinot/pull/15571#issuecomment-2840947479 > the BlockSplitter used in BlockExchange to split blocks based on rows doesn't serve any other purpose than trying to keep the gRPC message size under the max limit right? Any reason we

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2067940510 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/channel/MailboxContentObserver.java: ## @@ -60,10 +63,15 @@ public void onNext(MailboxContent mailbox

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2067934786 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java: ## @@ -252,7 +252,7 @@ public static DataBlock readFrom(ByteBuffer buffer) public

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2067925293 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/GrpcSendingMailbox.java: ## @@ -229,4 +254,64 @@ public DataBlock visit(ErrorMseBlock block, List se

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2067918455 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/BaseDataBlock.java: ## @@ -402,7 +403,14 @@ public List serialize() if (_serialized == null) {

Re: [PR] byte-based splitting for mse query blocks [pinot]

yashmayya commented on PR #15571: URL: https://github.com/apache/pinot/pull/15571#issuecomment-2840906511 > This PR creates a new splitting strategy on top of that based purely on the block bytes payload. Both strategies can live simultaneously, although it is expected that byte-based split

Re: [PR] byte-based splitting for mse query blocks [pinot]

yashmayya commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2067868126 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/BaseDataBlock.java: ## @@ -402,7 +403,14 @@ public List serialize() if (_serialized == null) {

Re: [PR] byte-based splitting for mse query blocks [pinot]

albertobastos commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2066929427 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/GrpcSendingMailbox.java: ## @@ -52,24 +58,35 @@ public class GrpcSendingMailbox implements Se

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on PR #15571: URL: https://github.com/apache/pinot/pull/15571#issuecomment-2838694209 Given I've added some code, I would request another committer to approve this PR as well. -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2065993917 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/GrpcSendingMailbox.java: ## @@ -52,24 +58,35 @@ public class GrpcSendingMailbox implements SendingMa

Re: [PR] byte-based splitting for mse query blocks [pinot]

albertobastos commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2063163586 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/GrpcSendingMailbox.java: ## @@ -229,4 +254,64 @@ public DataBlock visit(ErrorMseBlock block,

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2063125983 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/GrpcSendingMailbox.java: ## @@ -229,4 +254,64 @@ public DataBlock visit(ErrorMseBlock block, List se

Re: [PR] byte-based splitting for mse query blocks [pinot]

albertobastos commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2060203202 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/channel/MailboxContentObserver.java: ## @@ -58,10 +61,18 @@ public void onNext(MailboxContent

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2058307782 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java: ## @@ -230,6 +230,28 @@ public static ByteString toByteString(DataBlock dataBlock)

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2058295221 ## pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/channel/MailboxContentObserver.java: ## @@ -58,10 +61,18 @@ public void onNext(MailboxContent mailbox

Re: [PR] byte-based splitting for mse query blocks [pinot]

albertobastos commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2055538832 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java: ## @@ -230,6 +230,28 @@ public static ByteString toByteString(DataBlock dataB

Re: [PR] byte-based splitting for mse query blocks [pinot]

codecov-commenter commented on PR #15571: URL: https://github.com/apache/pinot/pull/15571#issuecomment-2823434303 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15571?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2052080756 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java: ## @@ -230,6 +230,28 @@ public static ByteString toByteString(DataBlock dataBlock)

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2052078798 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java: ## @@ -230,6 +230,28 @@ public static ByteString toByteString(DataBlock dataBlock)

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2052078798 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java: ## @@ -230,6 +230,28 @@ public static ByteString toByteString(DataBlock dataBlock)

Re: [PR] byte-based splitting for mse query blocks [pinot]

gortiz commented on code in PR #15571: URL: https://github.com/apache/pinot/pull/15571#discussion_r2052075968 ## pinot-common/src/main/java/org/apache/pinot/common/datablock/DataBlockUtils.java: ## @@ -230,6 +230,28 @@ public static ByteString toByteString(DataBlock dataBlock)