milenkovicm opened a new issue, #1770: URL: https://github.com/apache/datafusion-ballista/issues/1770
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** static job planner uses `ballista.optimizer.broadcast_join_threshold_bytes` configuration value to decide if shuffle should be broadcast (`CollectLeft`), this configuration value diverges from datafusion configurations: - `datafusion.optimizer.hash_join_single_partition_threshold_rows` - `datafusion.optimizer.hash_join_single_partition_threshold` which serve same purpose. **Describe the solution you'd like** I would propose to change static planner to to respect same configuration values like datafusion `datafusion.optimizer.hash_join_single_partition_threshold_rows` & `datafusion.optimizer.hash_join_single_partition_threshold` AQE implementation (#1752) uses the same values as datafusion so it would be good if we can have both planners (and datafusion) aligned **Describe alternatives you've considered** align both planners to `ballista.optimizer.broadcast_join_threshold_bytes` but that would diverge from datafusion configuration, which is not ideal as ballista tends to align with datafusion as much as possible **Additional context** Add any other context or screenshots about the feature request here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
