walterddr commented on code in PR #9906:
URL: https://github.com/apache/pinot/pull/9906#discussion_r1040002426
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/blocks/TransferableBlockUtils.java:
##########
@@ -76,9 +76,13 @@ public static boolean isNoOpBlock(TransferableBlock
transferableBlock) {
public static Iterator<TransferableBlock> splitBlock(TransferableBlock
block, DataBlock.Type type, int maxBlockSize) {
List<TransferableBlock> blockChunks = new ArrayList<>();
if (type == DataBlock.Type.ROW) {
- // Use estimated row size, this estimate is not accurate and is used to
estimate numRowsPerChunk only.
- int estimatedRowSizeInBytes =
block.getDataSchema().getColumnNames().length * MEDIAN_COLUMN_SIZE_BYTES;
- int numRowsPerChunk = maxBlockSize / estimatedRowSizeInBytes;
+ // Use 1 row per chunk when block.getDataSchema().size() is 0, we may
want to fine tune it.
+ int numRowsPerChunk = 1;
+ if (block.getDataSchema().size() != 0) {
+ // Use estimated row size, this estimate is not accurate and is used
to estimate numRowsPerChunk only.
+ int estimatedRowSizeInBytes = block.getDataSchema().size() *
MEDIAN_COLUMN_SIZE_BYTES;
+ numRowsPerChunk = maxBlockSize / estimatedRowSizeInBytes;
+ }
Preconditions.checkState(numRowsPerChunk > 0, "row size too large for
query engine to handle, abort!");
Review Comment:
yeah dataSchema should not be zero once reach this far down. it looks like a
bug somewhere else in the planner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
