sunchao opened a new pull request, #5472:
URL: https://github.com/apache/datafusion-comet/pull/5472
## Which issue does this PR close?
No issue is automatically closed. This is a standalone scalar-comparison
normalization fix.
## Rationale for this change
Comet normalizes floating-point comparison operands to match Spark's NaN and
signed-zero semantics, but the literal fast path skips every float/double
literal except negative zero. That also skips NaNs with a different sign or
payload. Such literals can therefore retain a different ordering/equality
representation from a normalized input column.
## What changes are included in this PR?
- Add `!isNaN` to the two literal fast-path guards so NaN literals use the
existing normalization path.
- Add float and double regressions using programmatically constructed
positive-payload and negative-payload NaNs. They compare Boolean results with
Spark, require native projections and filters, and cover both operand orders,
nulls, finite values, infinities, and signed zero.
The production change is only the two guards. No normalization refactor,
join changes, configuration changes, or new dependency is included. The tests
use the default floating-point mode.
## How are these changes tested?
**Draft: JVM validation is blocked, so no passing JVM test result is
claimed.**
- Built a matching native library from public main `5baa6b03`.
- Spotless checks on the final public patch and `git diff --check` passed.
- The attempted public Spark 4.0 run stopped during Maven dependency
resolution before any tests ran. No repository configuration was changed.
```sh
./mvnw test -Pspark-4.0 -Dtest=none \
-Dsuites=org.apache.comet.CometExpressionSuite \
'-Dtests=noncanonical NaN literals'
```
Red/green confirmation of the two new regressions and the existing
signed-zero/NaN checks remains required before this draft is ready to merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
