rich7420 opened a new pull request, #6076: URL: https://github.com/apache/datafusion-comet/pull/6076
## Which issue does this PR close? Closes #6044. ## Rationale for this change For nearby DOUBLE values around 1e16, native `var_pop` returns 2 instead of Spark's 1. Subtracting the rounded mean loses precision and also affects standard deviation. ## What changes are included in this PR? Use Spark's `CentralMomentAgg` update for variance and standard deviation, while preserving the Pearson update used by `corr` and `regr_r2`. Add scalar and grouped regression coverage, including the regression aggregates that share these accumulators. ## How are these changes tested? The new Rust regression fails before the fix. All 128 aggregate Rust tests and 8 targeted Scala/SQL tests pass locally; the regression asserts native Partial and Final execution. [Fork CI](https://github.com/rich7420/datafusion-comet/pull/31/checks) passes across Spark 3.4–4.2, including the Spark 4.1 SQL suites. Clippy and formatting checks pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
