walterddr commented on code in PR #9782:
URL: https://github.com/apache/pinot/pull/9782#discussion_r1023401183
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/LeafStageTransferableBlockOperator.java:
##########
@@ -0,0 +1,120 @@
+package org.apache.pinot.query.runtime.operator;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import java.util.PriorityQueue;
+import javax.annotation.Nullable;
+import org.apache.pinot.common.datablock.DataBlock;
+import org.apache.pinot.common.datablock.DataBlockUtils;
+import org.apache.pinot.common.datablock.MetadataBlock;
+import org.apache.pinot.common.datatable.DataTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.core.common.Operator;
+import org.apache.pinot.core.operator.BaseOperator;
+import org.apache.pinot.core.operator.blocks.InstanceResponseBlock;
+import org.apache.pinot.core.operator.blocks.results.BaseResultsBlock;
+import org.apache.pinot.query.runtime.blocks.TransferableBlock;
+
+
+/**
+ * Leaf-stage transfer block operator is used to wrap around the leaf stage
process results. They are passed to the
+ * Pinot server to execute query thus only one {@link DataTable} were
returned. However, to conform with the
+ * intermediate stage operators. an additional {@link MetadataBlock} needs to
be transfer after the data block.
+ *
+ * <p>In order to achieve this:
+ * <ul>
+ * <li>The leaf-stage result is split into data payload block and metadata
payload block.</li>
+ * <li>In case the leaf-stage result contains error or only metadata, we
skip the data payload block.</li>
+ * <li>Leaf-stage result blocks are in the {@link
DataSchema.ColumnDataType#getStoredColumnDataTypes()} format
+ * thus requires canonicalization.</li>
Review Comment:
It's a bit hard to refactor out but I will try.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
