andygrove opened a new pull request, #4100:
URL: https://github.com/apache/datafusion-comet/pull/4100
## Which issue does this PR close?
Partially addresses #4045.
## Rationale for this change
The SPARK-32509 test in `DynamicPartitionPruningSuite` was disabled across
all Spark versions under issue #4045. Re-running it on Spark 4.0.1 with Comet
enabled now passes in all 6 concrete suite variants (V1/V2/V2Filter ×
AEOff/AEOn).
## What changes are included in this PR?
Removes the `IgnoreComet` annotation from the SPARK-32509 test in
`dev/diffs/4.0.1.diff`. The companion SPARK-34637 test referenced by the same
`IgnoreComet` annotation remains ignored under #4045 since it still fails on
Spark 4.0.1 with AQE enabled.
The test was checked on Spark 3.5.8 as well but still fails there in
`DynamicPartitionPruningV1SuiteAEOn`, so the change is scoped to Spark 4.0 only.
## How are these changes tested?
Locally ran the test against the patched Spark 4.0.1 source with
`ENABLE_COMET=true ENABLE_COMET_ONHEAP=true`:
```
sql/testOnly org.apache.spark.sql.DynamicPartitionPruningV1SuiteAEOff \
org.apache.spark.sql.DynamicPartitionPruningV1SuiteAEOn \
org.apache.spark.sql.DynamicPartitionPruningV2SuiteAEOff \
org.apache.spark.sql.DynamicPartitionPruningV2SuiteAEOn \
org.apache.spark.sql.DynamicPartitionPruningV2FilterSuiteAEOff \
org.apache.spark.sql.DynamicPartitionPruningV2FilterSuiteAEOn \
-- -z "SPARK-32509"
```
All 6 variants pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
