924060929 commented on code in PR #67177:
URL: https://github.com/apache/doris/pull/67177#discussion_r3877932016
##########
be/src/exec/pipeline/pipeline_fragment_context.cpp:
##########
@@ -1073,8 +1084,8 @@ Status PipelineFragmentContext::_add_local_exchange_impl(
: 0);
break;
case TLocalPartitionType::PASS_TO_ONE:
- if (_runtime_state->enable_share_hash_table_for_broadcast_join()) {
- // If shared hash table is enabled for BJ, hash table will be
built by only one task
+ if (_runtime_state->be_exec_version() >=
SUPPORT_UNCONDITIONAL_PASS_TO_ONE_VERSION ||
Review Comment:
Fixed in b5d5e1246fb. I separated FE-deferred and BE-native PASS_TO_ONE
semantics. `_add_local_exchange_impl()` is the BE-native path and now preserves
the historical private-build fallback: shared broadcast hash tables use
PASS_TO_ONE, while `enable_share_hash_table_for_broadcast_join=false` uses
BroadcastExchanger so every task receives the full build. Unconditional v14
gather remains limited to `_create_deferred_local_exchangers()`. I also added a
3-BE FE plan-shape test that pins a private broadcast build to `LE(BROADCAST)`,
with the existing shared-build `LE(PASS_TO_ONE)` case as its counterpart. The
full `nereids_p0/local_shuffle` directory passed (8 suites, 0 failures).
##########
fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java:
##########
@@ -1100,6 +1103,18 @@ protected Pair<PlanNode, LocalExchangeType>
enforceRequire(
Pair<PlanNode, LocalExchangeType> childOutput =
child.enforceAndDeriveLocalExchange(translatorContext, this,
require);
+ // A serial consumer must not implicitly reduce a non-serial subtree
to one pipeline
+ // task. Besides losing parallelism, that can make a remote Exchange
expose fewer
+ // receiver tasks than FE addresses. Keep the subtree parallel and
make the N-to-one
+ // transition explicit. PASS_TO_ONE keeps every upstream receiver task
alive and
+ // funnels their output into the serial downstream pipeline's only
task.
+ if (passToOneAtSerialBoundary && childOutput.second !=
LocalExchangeType.PASS_TO_ONE) {
Review Comment:
Fixed in b5d5e1246fb. A new FE no longer serializes FE-planned local
exchanges to an execution-version-13 BE. The planner snapshots the BE execution
version, sets `localShufflePlanned=false` for versions below 14, skips
AddLocalExchange, uses the same value for receiver-destination planning, and
sends `enable_local_shuffle_planner=false` in TQueryOptions. The old BE
therefore runs its native local-shuffle planner with its original wire
semantics. Version 14 retains FE-deferred unconditional PASS_TO_ONE.
`NereidsCoordinatorTest` now verifies the v13 fallback and that the
version/planner decision remains frozen even if the global version changes
before coordinator construction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
