RussellSpitzer commented on issue #12467: URL: https://github.com/apache/iceberg/issues/12467#issuecomment-2704525496
Figured it out. The equality delete files are eliminated from scan planning because we assume that the filters exclude any matching rows from being evaluated. This is incorrect because in COW mode we *do need all rows to be evaluated even if they are not part of the current operation.* Example Table has an equality delete removing rows where X = 3 We issue a command "DELETE WHERE X = 2" When planning we currently incorrectly assume we don't need to look at the equality delete since we aren't touching rows where X = 3 The file is read, rows where X = 3 are materialized and copied into the new data file. CC @aokolnychyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
