gerashegalov opened a new pull request, #19000: URL: https://github.com/apache/iceberg/pull/19000
This backports the `effectiveSplitTargetSize` driver metric from #18709 to Spark 3.5, 4.0 and 4.1. The change is the same as the merged Spark 4.2 commit. Spark 3.5 and 4.0 also need a `lastExecutedMetricValue` import in `TestSparkReadMetrics`. Issue: [apache/iceberg#18708](https://github.com/apache/iceberg/issues/18708) Test plan: the same cases as #18709 on each Spark version. - Adaptive sizing, adaptation disabled, and an explicit read option each report the target actually used. - A runtime filter that prunes large files changes the reported target to that of the re-planned task groups. - Staged scans report the split size of each read. - Scans that plan no task groups report 0. - The SQL UI renders the reported value. For each of Spark 3.5, 4.0 and 4.1, formatting, checkstyle, `TestSparkReadMetrics`, `TestSparkStagedScan`, `TestSparkScan` and `TestRuntimeFiltering` pass. --- **AI Disclosure** - Model: Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: [unknown - human to fill in] - Prompt Summary: Backport the merged Spark 4.2 `effectiveSplitTargetSize` metric and its tests from #18709 to Spark 3.5, 4.0 and 4.1. The ported code and the import conflict resolution are generated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
