andygrove commented on issue #5701:
URL:
https://github.com/apache/datafusion-comet/issues/5701#issuecomment-5667180334
Checked against `apache/main` at commit
`481aefea9c60592612650de73bd2e1c7aa173979` (2026-09-13), Spark 3.5.3, JDK 17,
Linux x86_64, stock config (`spark.comet.enabled=true`,
`spark.comet.exec.enabled=true`, `spark.comet.exec.shuffle.enabled=true`,
`spark.shuffle.manager=org.apache.spark.sql.comet.execution.shuffle.CometShuffleManager`,
off-heap 8g).
`array_distinct` and `array_union` still reproduce as described here. The
same `-0.0` handling also shows up in three more functions that don't seem to
be tracked yet; all three run natively (`CometProject` over `CometNativeScan`).
Values come from parquet tables so the queries are not constant-folded away:
```sql
CREATE TABLE t_except(a array<double>, b array<double>) USING parquet;
INSERT INTO t_except VALUES (array(0.0D, -0.0D, 1.0D), array(0.0D)),
(array(0.0D, 1.0D), array(-0.0D));
SELECT a, b, array_except(a, b) FROM t_except;
-- Spark: [0.0, -0.0, 1.0] | [0.0] -> [-0.0, 1.0] Comet: [1.0]
-- Spark: [0.0, 1.0] | [-0.0] -> [0.0, 1.0] Comet: [1.0]
CREATE TABLE t_intersect(a array<double>, b array<double>) USING parquet;
INSERT INTO t_intersect VALUES (array(0.0D), array(-0.0D)), (array(-0.0D),
array(0.0D)), (array(-0.0D), array(-0.0D));
SELECT a, b, array_intersect(a, b) FROM t_intersect;
-- Spark: [0.0] | [-0.0] -> [] Comet: [0.0]
-- Spark: [-0.0] | [0.0] -> [] Comet: [0.0]
-- Spark: [-0.0] | [-0.0] -> [-0.0] Comet: [0.0]
CREATE TABLE t_sort(arr array<double>) USING parquet;
INSERT INTO t_sort VALUES (array(CAST('Infinity' AS DOUBLE),
CAST('-Infinity' AS DOUBLE), CAST('NaN' AS DOUBLE), 3.0D, 1.0D, -0.0D, 0.0D,
NULL));
SELECT sort_array(arr, false) FROM t_sort;
-- Spark: [NaN, Infinity, 3.0, 1.0, -0.0, 0.0, -Infinity, NULL]
-- Comet: [NaN, Infinity, 3.0, 1.0, 0.0, -0.0, -Infinity, NULL]
```
`array_min` on the same kind of input is covered by #5401.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
