andygrove opened a new pull request, #7444: URL: https://github.com/apache/datafusion-comet/pull/7444
## Which issue does this PR close? N/A ## Rationale for this change The published TPC-DS results are for Comet 1.1.0 on Spark 4.2.0. This updates them for Comet 1.2.0. ## What changes are included in this PR? - New charts in `docs/source/_static/images/benchmark-results/1.2.0/`, generated with `benchmarks/tpc/generate-comparison.py`. - `README.md` points at the new charts. The headline is now "~1.8x speedup over Spark 4.1, ~43% cost savings" (1,460 s for Spark vs 827 s for Comet, so 1.77x and 43% less time). - `benchmark-results/tpc-ds.md` and the benchmarking guide list the versions and the non-default settings used. Setup: - Spark 4.1.3 (Scala 2.13), Comet 1.2.0, Apache Iceberg 1.11.0 (zstd Parquet, Hadoop catalog), TPC-DS SF 1000, all 103 queries. - Both engines read the same Iceberg tables. Comet uses its native Iceberg scan. - EKS cluster with 32 executors of 16 cores each (8 CPUs per pod), data in S3. Each query was run twice and the mean is reported. - The settings that differ from the defaults are in the Configuration section of `tpc-ds.md`. For Comet 1.2.0 these are `spark.comet.scan.icebergNative.dataFileConcurrencyLimit=4` (the default is 2) and `spark.comet.exec.memoryPool.fraction=0.8` (the default is 1.0). The Spark 4.1.3 baseline replaces Spark 4.2.0 because the Iceberg data and the Comet 1.2.0 Iceberg scan were benchmarked on 4.1. The numbers are not comparable with the 1.1.0 charts, which read Parquet on Spark 4.2.0. ## How are these changes tested? Docs and images only. I rendered the charts and checked the totals against the per-query results. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
