joyhaldar opened a new pull request, #17635: URL: https://github.com/apache/iceberg/pull/17635
[VectorizedSparkOrcReaders.StructConverter](https://github.com/apache/iceberg/blob/f610fab08773733dda8e4919d715ae25986cfb51/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/data/vectorized/VectorizedSparkOrcReaders.java#L433) checked `idToConstant` before any metadata column, so `_row_id` matched the generic constant branch and every row received `first_row_id`. Stored per-row values were ignored and the position offset was never applied. `_last_updated_sequence_number` had the same problem. | row | expected | before | |-----|----------|--------| | 0 | 555 | 100 | | 1 | 101 | 100 | | 2 | 557 | 100 | | 3 | 103 | 100 | - Adds `RowIdColumnVector` and `LastUpdatedSeqColumnVector`, mirroring [RowIdReader](https://github.com/apache/iceberg/blob/f610fab08773733dda8e4919d715ae25986cfb51/orc/src/main/java/org/apache/iceberg/orc/OrcValueReaders.java#L375) and [LastUpdatedSeqReader](https://github.com/apache/iceberg/blob/f610fab08773733dda8e4919d715ae25986cfb51/orc/src/main/java/org/apache/iceberg/orc/OrcValueReaders.java#L413) in `OrcValueReaders`. - Moves the metadata-column checks ahead of `idToConstant`, matching the row-based reader. - Passes the ORC schema into `StructConverter` to look up stored `_row_id` and `_last_updated_sequence_number` columns by field id. Row based `ORC` got lineage support in #15776 and #16534, but the vectorized reader was not added. Found while adding vectorized coverage to the format model TCK (#17610). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
