zhengxiang378928908-code commented on PR #18414: URL: https://github.com/apache/iceberg/pull/18414#issuecomment-6336745465
I reproduced a related Spark SQL problem on Spark 4.2. With `identity(data)` partitioning and rows `'a1'`, `'b'` and NULL, `DELETE FROM t WHERE data NOT LIKE 'a%'` currently fails with this NPE. With this change it succeeds, but removes both `'b'` and the NULL row, which SQL `NOT LIKE` should keep. The unpartitioned table already removes the NULL row in the same test. I filed #18605 with the reproduction and the related results for `<>` and single-value `NOT IN`. This change follows Iceberg's expression semantics; the mismatch is in Spark's predicate conversion for metadata-only DELETE. Could we coordinate the Spark fix (or a fallback that avoids metadata-only DELETE for these predicates), so that fixing the NPE doesn't turn the partitioned case into incorrect deletion? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
