Ersa-tech opened a new pull request, #6969:
URL: https://github.com/apache/datafusion-comet/pull/6969
## Which issue does this PR close?
Closes #4942.
## Rationale for this change
`cast_int_to_timestamp`, `cast_decimal_to_timestamp`,
`cast_float_to_timestamp`, and `cast_boolean_to_timestamp` still built the
output one row at a time with `TimestampMicrosecondBuilder`. These are the
remaining per-element casts to timestamp in #4942.
## What changes are included in this PR?
- Int casts use `unary` and `saturating_mul`. The multiply is infallible on
a null slot.
- Boolean casts use `from_unary`. `true` is still 1 microsecond and `false`
is 0.
- Float casts use `try_unary` in ANSI mode and `unary_opt` in Legacy and
Try. The error string is built only when ANSI fails.
- Decimal casts use one `unary_opt` pass. The 256-bit division does not
vectorize, and evaluating null slots was slower on every measured shape,
including a batch with no nulls. Spark still truncates toward zero and does not
raise.
- A batch with no nulls still has `nulls() == None`.
`unary` and `from_unary` transform the value stored under a null. The
builder wrote 0 there. Arrow equality ignores those slots. A negative decimal
scale still panics in `10_i128.pow`, same as before.
This does not edit the float-to-int or decimal-to-int macros, or the test
before `test_spark_cast_float_min_value_to_string` (open PR #6953).
Prepared with Cursor Grok 4.7. The semantics, tests, and benchmarks were
reviewed.
## How are these changes tested?
Rust tests pin saturation, decimal truncation and the low 64 bits, the ANSI
error variant and the first offending value, Legacy and Try agreeing, and a NaN
hidden under a null. `cargo test -p datafusion-comet-spark-expr --lib
conversion_funcs::` (119 tests). `cargo fmt`. `cargo clippy --all-targets
--workspace -- -D warnings` from `native/`.
`CometNativeCastSuite` tests whose names contain `to TimestampType` (20
tests, Spark 4.1, release `libcomet.so`) all passed, including boolean, byte,
short, int, long, float, double, and decimal to timestamp.
Criterion, 8192 rows, baseline `main` on the same machine (4 vCPU Intel
Xeon, family 6 model 207, AVX-512, 15 GiB RAM, rustc 1.99.0). Bench profile:
`opt-level=3`, `lto=thin`, `codegen-units=1`, `overflow-checks=false`. No shape
regressed. Decimal with no nulls is under 10%.
```
cd native
export CARGO_PROFILE_BENCH_OPT_LEVEL=3 CARGO_PROFILE_BENCH_LTO=thin \
CARGO_PROFILE_BENCH_CODEGEN_UNITS=1
CARGO_PROFILE_BENCH_OVERFLOW_CHECKS=false
cargo bench -p datafusion-comet-spark-expr \
--bench cast_int_to_timestamp --bench cast_non_int_numeric_timestamp --
--save-baseline main
cargo bench -p datafusion-comet-spark-expr \
--bench cast_int_to_timestamp --bench cast_non_int_numeric_timestamp --
--baseline main
```
| Cast | No nulls | Sparse nulls | Dense nulls |
| --- | --- | --- | --- |
| i64 | 20.2 → 2.3 µs (−89%) | 27.6 → 2.4 µs (−91%) | 21.8 → 2.3 µs (−89%) |
| f64 legacy | 42.3 → 34.4 µs (−19%) | 50.2 → 33.0 µs (−34%) | 23.2 → 5.5 µs
(−76%) |
| f64 ANSI | 42.2 → 35.5 µs (−14%) | 50.3 → 32.8 µs (−35%) | 23.4 → 5.6 µs
(−76%) |
| decimal(38, 6) | 92.8 → 84.2 µs (−9%) | 94.7 → 76.4 µs (−19%) | 31.2 →
11.6 µs (−63%) |
| bool | 20.5 → 3.8 µs (−81%) | 27.6 → 4.0 µs (−86%) | 20.9 → 3.8 µs (−81%) |
All-null batches: i64 −89%, f64 legacy −93%, f64 ANSI −94%, decimal −94%,
bool −80%. A legacy f64 batch of NaN and infinities is −34%. f32 and the
narrower integers use the same kernels. Decimal scales 0 and 18 with no nulls,
measured on this same `unary_opt` kernel, were −9.6% and −9.8%.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
