pvary commented on PR #12479: URL: https://github.com/apache/iceberg/pull/12479#issuecomment-2713693155
> I checked out this change and ran the newly added test with and without the fix. The behavior is as expected, except that even without the fix, two of the six cases in the parameterized test pass, namely when the file format is Avro. For my edification, can you explain why the bug doesn't affect Avro? We have metrics based filtering set up for the delete files as well (so we don't read deletes which are not needed). See: https://github.com/apache/iceberg/blob/6e8718113c08aebf76d8e79a9e2534c89c73407a/core/src/main/java/org/apache/iceberg/ManifestReader.java#L229 For the Parquet equality delete files we have metrics available, for Avro we don't have metrics. When we have metrics available the `metricsEvaluator` checks the pushed down filter, and decides to skip the equality delete file based on the filter, so for Parquet the delete file is ignored. For Avro we don't have metrics, so the `metricsEvaluator` doesn't filter out the equality delete file, and the equality delete is applied. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
