nssalian opened a new pull request, #1140: URL: https://github.com/apache/iceberg-go/pull/1140
First part of #1139 ## Summary Adds Spark 4.0 alongside Spark 3.5 in the integration test matrix so v3 features can be validated cross-engine in CI. ## What's in - `Dockerfile.spark4` (apache/spark:4.0.2 + iceberg-spark-runtime-4.0 + aws-bundle), with companion `spark-defaults.conf` and `entrypoint.sh` - docker-compose parameterized via `SPARK_IMAGE` and `SPARK_DOCKERFILE`; defaults preserve 3.5 behavior - `make integration-setup-spark4` - Workflow matrix entry for 4.0 (`fail-fast: false`) - 3 CTAS statements in `provision.py` rewritten as `CREATE TABLE` + `INSERT INTO` so the same provisioning runs on both versions ## What's not in No new tests, no cross-engine v3 feature validation. Follow-up PR. ## Validation Locally ran on both Spark versions: - `make integration-setup && make integration-test` - `make integration-setup-spark4 && make integration-test` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
