This is an automated email from the ASF dual-hosted git repository.
gengliangwang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new d3c6a5a91694 [SPARK-57973][SQL][TEST] Drop the unreliable compile-time
metric from WholeStageCodegenSizeBenchmark
d3c6a5a91694 is described below
commit d3c6a5a91694e867ae432d1c759a7664ee715469
Author: Gengliang Wang <[email protected]>
AuthorDate: Fri Jul 10 11:50:18 2026 -0700
[SPARK-57973][SQL][TEST] Drop the unreliable compile-time metric from
WholeStageCodegenSizeBenchmark
### What changes were proposed in this pull request?
Drop the `compile time (ms)` metric from `WholeStageCodegenSizeBenchmark`
(added in SPARK-57915),
and regenerate its results file. This removes the metric's docstring
bullet, the
`CodeGenerator.resetCompileTime()` / `compileTime` measurement, the
reported row, and the
now-unused `CodeGenerator` import.
### Why are the changes needed?
The compile-time number is not a reliable or reproducible metric, so it can
mislead when comparing
before/after:
- `CodeGenerator.compile` is wrapped in a cache keyed on the generated
source string, and
`compileTime` only accrues on cache misses. Across the TPC-DS suite a
large fraction of stages
generate identical source (measured: ~2247 stages but only ~1214 distinct
sources), so the metric
reflects the time to compile the *distinct* subset -- and that subset
differs between two versions,
so the totals are not comparable.
- The size-bounded cache (`spark.sql.codegen.cache.maxEntries`, default
100) evicts and recompiles,
so the number also depends on eviction order, not just the code.
- It is wall-clock time and therefore JIT/GC/machine-load sensitive: the
same tree measured ~10%
apart across two runs, larger than the differences the benchmark is meant
to detect.
The remaining metrics (generated source size, summed/max method bytecode,
inner-class count,
constant-pool entries) are deterministic and machine-independent, and
source size is itself a stable
proxy for compile cost (roughly linear in input), so nothing actionable is
lost.
### Does this PR introduce _any_ user-facing change?
No (test-only benchmark).
### How was this patch tested?
Re-ran `build/sbt "sql/Test/runMain
org.apache.spark.sql.execution.benchmark.WholeStageCodegenSizeBenchmark"`
and regenerated
`sql/core/benchmarks/WholeStageCodegenSizeBenchmark-results.txt` with
`SPARK_GENERATE_BENCHMARK_FILES=1`; the compile-time row is gone and the
deterministic metrics are
unchanged.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Closes #57049 from gengliangwang/SPARK-57973-drop-compiletime-metric.
Authored-by: Gengliang Wang <[email protected]>
Signed-off-by: Gengliang Wang <[email protected]>
---
.../benchmarks/WholeStageCodegenSizeBenchmark-results.txt | 13 ++++++-------
.../benchmark/WholeStageCodegenSizeBenchmark.scala | 10 +---------
2 files changed, 7 insertions(+), 16 deletions(-)
diff --git a/sql/core/benchmarks/WholeStageCodegenSizeBenchmark-results.txt
b/sql/core/benchmarks/WholeStageCodegenSizeBenchmark-results.txt
index 7dc8f3ae5e46..36304202f8a7 100644
--- a/sql/core/benchmarks/WholeStageCodegenSizeBenchmark-results.txt
+++ b/sql/core/benchmarks/WholeStageCodegenSizeBenchmark-results.txt
@@ -4,13 +4,12 @@ TPCDS codegen size
queries measured 135
codegen stages 2247
-source code size (chars) 23794382
-max method bytecode (sum) 925512
-max method bytecode (max) 5843
-inner classes 516
-constant pool (sum) 440540
-constant pool (max) 770
+source code size (chars) 21846001
+max method bytecode (sum) 841185
+max method bytecode (max) 4962
+inner classes 258
+constant pool (sum) 432881
+constant pool (max) 765
codegen fallbacks 0
-compile time (ms) 10618.3
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/WholeStageCodegenSizeBenchmark.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/WholeStageCodegenSizeBenchmark.scala
index 0a9fa43ea397..5ec19ecbdfd4 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/WholeStageCodegenSizeBenchmark.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/WholeStageCodegenSizeBenchmark.scala
@@ -24,7 +24,7 @@ import org.apache.spark.benchmark.Benchmark
import org.apache.spark.internal.Logging
import org.apache.spark.sql.{SparkSession, TPCDSSchema, TPCDSTableStats}
import org.apache.spark.sql.catalyst.TableIdentifier
-import org.apache.spark.sql.catalyst.expressions.codegen.{ByteCodeStats,
CodeGenerator}
+import org.apache.spark.sql.catalyst.expressions.codegen.ByteCodeStats
import org.apache.spark.sql.catalyst.util._
import org.apache.spark.sql.execution.debug._
import org.apache.spark.sql.internal.SQLConf
@@ -43,8 +43,6 @@ import org.apache.spark.sql.internal.SQLConf
* - max method bytecode (max): the single largest compiled method across
all stages;
* - inner classes: number of generated inner classes summed over all stages;
* - constant pool (sum / max): compiled constant-pool entries, summed and
the per-stage max;
- * - compile time: total Janino compilation time (ms), measured cold via
- * [[CodeGenerator.resetCompileTime()]];
* - codegen fallbacks: stages whose generated code failed to compile (fell
back to interpreted);
* - stages / queries: the whole-stage-codegen stage count and the number of
queries measured.
*
@@ -161,14 +159,9 @@ object WholeStageCodegenSizeBenchmark extends
SqlBasedBenchmark with Logging {
val queriesV1_4 = CodegenSizeTpcdsSchema.tpcdsQueries
val queriesV2_7 = CodegenSizeTpcdsSchema.tpcdsQueriesV2_7_0
- // Measure compile time cold: reset the accumulator, let
codegenStringSeq trigger the Janino
- // compiles (the compile cache is keyed on the source string, so
distinct queries miss), then
- // read the total below.
- CodeGenerator.resetCompileTime()
val totals = Totals()
queriesV1_4.foreach(measureQuery("tpcds", _, totals))
queriesV2_7.foreach(measureQuery("tpcds-v2.7.0", _, totals))
- val compileTimeMs = CodeGenerator.compileTime / 1000000.0
runBenchmark("TPCDS codegen size") {
// Use a Benchmark instance only for its `out` PrintStream, which tees
to stdout and the
@@ -188,7 +181,6 @@ object WholeStageCodegenSizeBenchmark extends
SqlBasedBenchmark with Logging {
row("constant pool (sum)", totals.constPoolSum)
row("constant pool (max)", totals.constPoolMax)
row("codegen fallbacks", totals.fallbacks)
- row("compile time (ms)", "%.1f".format(compileTimeMs))
out.println()
// scalastyle:on println
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
