sunchao commented on PR #5420: URL: https://github.com/apache/datafusion-comet/pull/5420#issuecomment-5447295543
Fixed the grouped decimal AVG evaluation-order issue in [728c55e8d](https://github.com/apache/datafusion-comet/pull/5420/commits/728c55e8d1fb92e3ebb59e21d9ed5c393cbc17f2). Grouped decimal `AVG` in ANSI mode now keeps every stage evaluating that AVG in Spark. This prevents native batch emission from throwing for a group that Spark's LIMIT does not consume, while preserving errors when the overflowing group is consumed. The native empty/count-state and overflow repairs are unchanged. The guard applies to every decimal precision and to the whole affected aggregate operator, including colocated aggregates, even without LIMIT. This intentionally gives up native aggregation for those ANSI queries. Grouped `TRY_AVG`, legacy decimal AVG, non-decimal AVG, and unrelated native work retain their existing eligibility. The description and generated-documentation source now state that scope; existing benchmark timings use legacy mode and do not measure this new fallback. The full Spark 4.1.3 JVM reactor passed **139 tests** (107 aggregate, 31 planner, and the AVG SQL fixture), with the loaded native library verified against the unchanged native tree. Both new LIMIT regressions fail against the unchanged planner and pass with the fix, with AQE off/on. Consumed-overflow and native-control checks also pass, as do Spotless, Scalastyle, and CI's syntactic Scalafix check. CI for the new commit is pending. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
