0lai0 commented on code in PR #5367:
URL: https://github.com/apache/datafusion-comet/pull/5367#discussion_r3881965911
##########
native/spark-expr/src/conversion_funcs/numeric.rs:
##########
@@ -852,15 +853,13 @@ pub(crate) fn spark_cast_int_to_int(
pub(crate) fn spark_cast_decimal_to_boolean(array: &dyn Array) ->
SparkResult<ArrayRef> {
let decimal_array = array.as_primitive::<Decimal128Type>();
- let mut result = BooleanBuilder::with_capacity(decimal_array.len());
- for i in 0..decimal_array.len() {
- if decimal_array.is_null(i) {
- result.append_null()
- } else {
- result.append_value(!decimal_array.value(i).is_zero());
- }
- }
- Ok(Arc::new(result.finish()))
+ // Arrow has no Decimal-to-Boolean cast. `neq` against a zero of the same
+ // precision/scale is exactly `!value.is_zero()`, including null handling.
+ let zero = Scalar::new(
+ Decimal128Array::from(vec![0i128])
+ .with_precision_and_scale(decimal_array.precision(),
decimal_array.scale())?,
Review Comment:
Thanks for follow-up, I added a regression test in
`CometExpressionSuite.scala` with checkSparkAnswerAndOperator so a native-plan
fallback fails loudly.
Covered by:
- all-null fast path (skips the zero-scalar construction Arrow rejects for
precision == 0)
- precision-zero fast path (v != 0 on the raw i128 payload, so an
out-of-contract non-zero valid slot still round-trips)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
