flyrain commented on code in PR #11551:
URL: https://github.com/apache/iceberg/pull/11551#discussion_r1865274246
##########
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/ColumnarBatchReader.java:
##########
@@ -245,5 +247,16 @@ void applyEqDelete(ColumnarBatch columnarBatch) {
columnarBatch.setNumRows(currentRowId);
}
+
+ ColumnarBatch removeExtraColumns(
Review Comment:
It would be helpful to keep the following comments as part of the JavaDoc
for the method to provide clarity and context:
```
/**
* Removes extra columns added for processing equality delete filters
that are not part of the
* final query output.
* <p>
* During query execution, additional columns may be included in the
schema to evaluate equality
* delete filters. For example, if the table schema contains columns C1,
C2, C3, C4, and C5, and
* the query is 'SELECT C5 FROM table' while equality delete filters are
applied on C3 and C4,
* the processing schema includes C5, C3, and C4. These extra columns
(C3 and C4) are needed
* to identify rows to delete but are not included in the final result.
* <p>
* This method removes these extra columns from the end of {@code
arrowColumnVectors}, ensuring
* only the expected columns remain.
*
* @param arrowColumnVectors the array of column vectors representing
query result data
* @param columnarBatch the original {@code ColumnarBatch} containing
query results
* @return a new {@code ColumnarBatch} with extra columns removed, or
the original batch if no
* extra columns were found
*/
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
