NoahKusaba opened a new pull request, #2479: URL: https://github.com/apache/datafusion-ballista/pull/2479
# Rationale for this change The tuning guide's Join Strategy section only described the static planner. It told users to `SET datafusion.optimizer.prefer_hash_join = true` to get hash joins, but under AQE, the default, that setting is not consulted. # What changes are included in this PR? - Split the section into **With AQE (the default)** and **With AQE turned off**. - Under AQE, the join is chosen at runtime from measured sizes: broadcast below `ballista.optimizer.broadcast_join_threshold_bytes`, a partitioned hash join while every build partition is under `ballista.optimizer.hash_join_max_build_partition_bytes`, and a sort-merge join otherwise. - With AQE off, keep the existing advice, now scoped to that case. It also notes that the static planner only promotes hash joins to broadcast, so the sort-merge default means no broadcast joins. - The opt-in example now also sets `ballista.planner.adaptive.enabled = false`, since `prefer_hash_join` does nothing otherwise. - Join reordering compares byte sizes first and falls back to row counts, not row counts alone. # Are there any user-facing changes? Documentation only, but **this changes tuning advice users may have acted on**, which is why it is a PR of its own. Worth a review from someone who knows the AQE join-selection rules. Split out of #2438 to make it easier to review. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
