sam-1112 opened a new pull request, #5967: URL: https://github.com/apache/datafusion-comet/pull/5967
## Which issue does this PR close? Part of #5103. This PR only adds differential coverage. It does not change production routing and does not close the issue. ## Rationale for this change Before changing how Comet routes `xxhash64`, we need a small, reviewable proof that the native kernel already matches `datafusion_spark::SparkXxhash64` on flat Arrow types at Spark's fixed seed of 42. ## What changes are included in this PR? Adds `native/spark-expr/src/hash_funcs/xxhash64_diff.rs` and registers it from `mod.rs` under `#[cfg(test)]`. Helpers: - `comet_kernel` calls `create_xxhash64_hashes` with a caller-supplied starting seed - `spark_xxhash64_upstream` calls `SparkXxhash64::invoke_with_args` - `columnar_u64s` normalizes scalar/array results to `Vec<u64>` - `assert_compatible` compares Comet and upstream at seed 42 Coverage: - Primitives: Boolean, Int8/16/32/64, Float32/64 (sign, zero, `-0.0`, type bounds, null; floats also include NaN/Inf) - Variable-width: Utf8, LargeUtf8, Binary, LargeBinary, FixedSizeBinary(1/4/8) (empty/all-zero, ASCII, Unicode, null, different lengths) - Temporal: Date32, Date64, TimestampMicrosecond without timezone and with UTC, including a pre-epoch value; Time64 is intentionally omitted - Decimal128: precision 10/18/20/38 with different scales, including unscaled values outside `i64` - Multi-column chaining: `xxhash64(a, b, c)` matches sequential per-column seed updates and upstream - Kernel-level custom seeds: 0, 1, 7, 42, -1, `i64::MIN`, and a different starting seed per row This PR does **not** include Dictionary, List, Map, Struct, Time64, production routing, Scala tests, or audit-doc routing notes. ## How are these changes tested? ```bash cd native cargo test -p datafusion-comet-spark-expr hash_funcs::xxhash64_diff -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
