nssalian commented on PR #17424: URL: https://github.com/apache/iceberg/pull/17424#issuecomment-5126677729
@Guosmilesmile thanks for taking a look. The raw column min/max you see (10000000000 for both) is because integer widening (INT8+INT64 → INT64 schema, preserved in admittedType() from the old getMostCommonType) combined with ShreddedVariantWriter.write doing an exact-type check means INT8/INT16/INT32 rows fall to residual value. But Iceberg's variant bounds handle this correctly: ParquetMetrics.value() invalidates the typed bounds whenever the value residual has any non-null entry, so filter pushdown doesn't use those raw stats. I recently fixed iceberg-go in (https://github.com/apache/iceberg-go/pull/1555) to match this behavior based on the spec. The spec permits either widening on write or falling to residual for out-of-schema types; Iceberg went with residual (see testMixedShredding in TestVariantWriters which relies on this to preserve exact width). This PR only adds the uniformity check above the widening logic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
