andygrove opened a new pull request, #4106: URL: https://github.com/apache/datafusion-comet/pull/4106
## Which issue does this PR close? Closes #. ## Rationale for this change Stacks on top of #4097 (spark-4.1 Maven profile and shim sources). Now that the profile compiles, run the existing Comet test matrix against Spark 4.1 in PR CI so regressions are caught before merge. Note: until #4097 lands, the diff here includes the prep branch changes. Review only the additional commit; rebase to main once #4097 is merged. ## What changes are included in this PR? - Add `Spark 4.1, JDK 17` entry to `linux-test` matrix in `pr_build_linux.yml`. - Add `Spark 4.1, JDK 17, Scala 2.13` entry to `macos-aarch64-test` in `pr_build_macos.yml`. The spark-4.1 profile already pins Scala to 2.13.17 (matching Spark 4.1.1's runtime); activating `-Pscala-2.13` would override that back to 2.13.16 and break, so the entry omits it. - Remove the placeholder compile-only `build-spark-4-1` job from `pr_build_linux.yml`. The new matrix entry both compiles and tests, so the placeholder is redundant. - Document in `lint-java` why Spark 4.1 is intentionally absent: `semanticdb-scalac_2.13.17` is not yet published, so scalafix cannot run against the 4.1 profile. - `CometNativeWriteExec`: switch `newTaskTempFile` to the `FileNameSpec` overload. Spark 4.1 made the `(taskContext, dir, ext: String)` overload throw `mustOverrideOneMethodError` by default; `FileNameSpec` exists in 3.4 onward so this works across all profiles. - `CometExpressionSuite` remainder test: branch the expected error message on `isSpark41Plus`. Spark 4.1 introduced `REMAINDER_BY_ZERO` for `%` instead of reusing `DIVIDE_BY_ZERO`. ## How are these changes tested? The whole point of the PR is to get the existing Comet test suite running against Spark 4.1 in CI. The two source-level fixes are exercised by `CometNativeWriteExec` paths and the `CometExpressionSuite` remainder test under the 4.1 matrix entry. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
