geruh opened a new pull request, #14081:
URL: https://github.com/apache/iceberg/pull/14081
Fixes #14071
Variant types were not handled correctly in Parquet row group filtering. The
`ParquetMetricsRowGroupFilter.notNull()` needs to account for variant types
which require post scan evaluation to access shredded statistics for nested
field filtering, similar to structs are handled.
This change always returns `ROWS_MIGHT_MATCH` for variant types.
### Testing
- testVariantFilterNotNull(): test with mixed variant/null data
- testAllNullsVariantNotNull(): Edge case with all-null variant columns
- TestInclusiveMetricsEvaluatorWithExtract.java already provides extensive
coverage of variant extract expressions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
