peterxcli commented on code in PR #5684:
URL: https://github.com/apache/datafusion-comet/pull/5684#discussion_r3937522721
##########
native/spark-expr/src/conversion_funcs/numeric.rs:
##########
@@ -863,6 +865,103 @@ pub(crate) fn spark_cast_decimal_to_boolean(array: &dyn
Array) -> SparkResult<Ar
Ok(Arc::new(result.finish()))
}
+/// Powers of ten that are exactly representable as `f64` (`10^n = 2^n * 5^n`
and `5^22 < 2^53`);
+/// the same table as Java's `BigDecimal.DOUBLE_10_POW`.
+const F64_EXACT_POW10: [f64; 23] = [
+ 1e0, 1e1, 1e2, 1e3, 1e4, 1e5, 1e6, 1e7, 1e8, 1e9, 1e10, 1e11, 1e12, 1e13,
1e14, 1e15, 1e16,
+ 1e17, 1e18, 1e19, 1e20, 1e21, 1e22,
+];
+
+/// Powers of ten that are exactly representable as `f32` (`5^10 < 2^24`); the
same table as
+/// Java's `BigDecimal.FLOAT_10_POW`.
+const F32_EXACT_POW10: [f32; 11] = [1e0, 1e1, 1e2, 1e3, 1e4, 1e5, 1e6, 1e7,
1e8, 1e9, 1e10];
+
+/// Largest integer magnitude that converts to `f64` without rounding.
+const F64_EXACT_INT_LIMIT: u128 = 1 << 53;
+
+/// Largest integer magnitude that converts to `f32` without rounding.
+const F32_EXACT_INT_LIMIT: u128 = 1 << 24;
+
+/// Converts a Decimal128 value to the `f64` nearest to its exact decimal
value (ties to even),
+/// matching Spark's `Decimal.toDouble`, i.e.
`java.math.BigDecimal.doubleValue()`.
+///
+/// DataFusion's kernel computes `(unscaled as f64) / 10^scale`, which rounds
the unscaled value
+/// first and the quotient second; once `|unscaled| > 2^53` (any
`DECIMAL(38,18)` value of
+/// magnitude `>= 0.01`) the two roundings can land one ulp away from the
correctly rounded
+/// result, e.g. `12345.6789` becomes `12345.678899999999`.
+pub(crate) fn decimal128_to_f64(unscaled: i128, scale: i8) -> f64 {
+ // Fast path (also Java's): when the unscaled value and the power of ten
are both exact
+ // doubles, a single IEEE division or multiplication is correctly rounded.
+ if unscaled.unsigned_abs() <= F64_EXACT_INT_LIMIT {
Review Comment:
Addressed in d410f61d8. Both helpers now handle scale zero before the
magnitude guard with direct target-width casts (`i128 as f64` and `i128 as
f32`). I also added `cast_numeric` Criterion cases for DECIMAL(18,0) to DOUBLE,
DECIMAL(38,0) to DOUBLE, and DECIMAL(12,0) to FLOAT, each with and without
nulls. A local release run measured 1.71-1.99 ns/input slot across the six
cases. `cargo test -p datafusion-comet-spark-expr --lib` passed 663 tests, and
package Clippy with all targets passed with warnings denied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
