sam-1112 opened a new pull request, #5359:
URL: https://github.com/apache/datafusion-comet/pull/5359
## Which issue does this PR close?
Closes #5074
## Rationale for this change
Registry-resolved UDFs (including datafusion-spark) previously ignored
`fail_on_error=true`, which could silently change ANSI semantics. Plain
`CometScalarFunction` also always serialized `fail_on_error=false`, so
ANSI-sensitive Spark expressions could be miswired without failing at planning
time.
## What changes are included in this PR?
- **Native**: `create_comet_physical_fun_with_eval_mode` fails closed when
`fail_on_error=true` would fall through to registry lookup.
- **Scala serde**: `CometScalarFunction.convert` rejects expressions with
`failOnError`, `evalMode`, or `nullOnOverflow` fields; adds
`isAnsiSensitive` helpers for registration audits.
- **Docs**: document ANSI / `fail_on_error` wiring constraints in the
contributor guide.
- **Tests**: Rust integration tests in `test_udf_registration`; Scala
`CometScalarFunctionSuite` for serde guards and audit.
## How are these changes tested?
- `cargo test -p datafusion-comet-spark-expr --test test_udf_registration --
--nocapture`
- `./mvnw test -Dtest=none
-Dsuites="org.apache.comet.serde.CometScalarFunctionSuite" -Pspark-4.1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
