andygrove opened a new issue, #5931:
URL: https://github.com/apache/datafusion-comet/issues/5931
### Describe the bug
With Comet enabled, `regr_r2` returns NULL where Spark returns 1.0 when the
x argument is constant (a literal, or a column with a single value). Spark
alone returns the result shown first. The executed plan runs it through
CometColumnarToRow, CometExchange, CometHashAggregate, CometNativeScan.
### Steps to reproduce
```sql
CREATE TABLE test_regr(y double, x double, grp string) USING parquet;
INSERT INTO test_regr VALUES
(1.0, 2.0, 'a'),
(3.0, 4.0, 'a'),
(5.0, 6.0, 'a'),
(7.0, 8.0, 'a'),
(9.0, 10.0, 'a'),
(2.0, 1.0, 'b'),
(4.0, 3.0, 'b'),
(6.0, 5.0, 'b'),
(NULL, 3.0, 'b'),
(4.0, NULL, 'b'),
(NULL, NULL, 'b');
CREATE TABLE test_regr_const_x(y double, x double) USING parquet;
INSERT INTO test_regr_const_x VALUES (1.0, 5.0), (2.0, 5.0), (3.0, 5.0),
(4.0, 5.0);
CREATE TABLE test_regr_const_y(y double, x double) USING parquet;
INSERT INTO test_regr_const_y VALUES (7.0, 1.0), (7.0, 2.0), (7.0, 3.0),
(7.0, 4.0);
SELECT regr_slope(y, 3.0), regr_sxx(y, 3.0), regr_r2(y, 3.0) FROM test_regr;;
```
Spark:
| regr_slope(y, 3.0) | regr_sxx(y, 3.0) | regr_r2(y, 3.0) |
|---|---|---|
| NULL | 0.0 | 1.0 |
Comet:
| regr_slope(y, 3.0) | regr_sxx(y, 3.0) | regr_r2(y, 3.0) |
|---|---|---|
| NULL | 0.0 | NULL |
### Expected behavior
The same rows, schema and nullability as Spark.
### Additional context
- Comet built from `apache/main` at commit
`481aefea9c60592612650de73bd2e1c7aa173979` (2026-09-13T19:58:05+00:00)
- Spark 3.5.3, JDK 17, Linux x86_64, `local[4]`
- Configuration:
- `spark.comet.enabled=true`
- `spark.comet.exec.enabled=true`
- `spark.comet.exec.shuffle.enabled=true`
- `spark.memory.offHeap.enabled=true`
- `spark.memory.offHeap.size=8g`
- `spark.plugins=org.apache.spark.CometPlugin`
-
`spark.shuffle.manager=org.apache.spark.sql.comet.execution.shuffle.CometShuffleManager`
Further failing variants of the same expression (same fixture, different
arguments):
- `SELECT regr_slope(y, x), regr_intercept(y, x), regr_r2(y, x) FROM
test_regr_const_x`
- `SELECT regr_slope(y, x), regr_intercept(y, x), regr_r2(y, x) FROM
test_regr_const_y`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
