andygrove commented on issue #6659: URL: https://github.com/apache/datafusion-comet/issues/6659#issuecomment-5998474833
Closing as a duplicate of an accepted divergence the user guide already documents. Under "Manifest metadata visible to later readers", `iceberg-writes.md` says that manifest `value_counts` / `null_value_counts` for float/double columns nested under a nullable struct count the rows whose parent struct is null, while iceberg-java's writer-tracked counts do not. Both counts grow by the same amount, so null ratios and `IS NULL` / `IS NOT NULL` pruning are unaffected, which matches the "never prunes more" analysis above. That entry dates the difference to Iceberg 1.10, but it starts in 1.9, when `ParquetMetrics` arrived. #6681 corrects the version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
