rdblue commented on PR #6402: URL: https://github.com/apache/iceberg/pull/6402#issuecomment-1354038557
@hililiwei, I flagged the test cases in my review, but I now see that @stevenzwu did as well. The problem is that NaN comparison should always result in `false`. That's why Iceberg doesn't allow `NaN` as a literal value in expressions. If you try to create an expression with a [`NaN` literal, it will fail](https://github.com/apache/iceberg/blob/master/api/src/main/java/org/apache/iceberg/expressions/Literals.java#L61). That forces callers to be specific. If the caller intends to compare with `NaN`, then the comparison should be `false` and there's no need to pass it to Iceberg. If the caller intends to check whether a value is `NaN`, then there is the [`isNaN` unary predicate](https://github.com/apache/iceberg/blob/master/api/src/main/java/org/apache/iceberg/expressions/Expressions.java#LL125C28-L125C28). It is up to Flink how to map expressions. In Spark, `NaN` is a valid comparison, so Spark filter translation converts `x = NaN` to `isNaN(x)`. Flink could do something similar. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
