github-actions[bot] commented on code in PR #67549:
URL: https://github.com/apache/doris/pull/67549#discussion_r4321450907
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/cost/CostModel.java:
##########
@@ -467,18 +468,20 @@ public Cost visitPhysicalHashJoin(
}
}
- // hbo to adjust bc cost parameter to reduce bc cost
+ // hbo to adjust bc cost parameter to reduce bc cost. The lookup
key contains the hbo
+ // fingerprint of this join, which is only worth building when
some learned entry exists
+ // at all (this repository has no pinned fallback here: the skew
ratios are measurements)
if (context.getSessionVariable() != null
- && context.getSessionVariable().isEnableHboOptimization())
{
- PlanNodeAndHash planNodeAndHash = null;
- try {
- planNodeAndHash =
HboUtils.getPlanNodeHash(physicalHashJoin);
- } catch (IllegalStateException e) {
- LOG.warn("failed to get plan node hash", e);
- }
- if (planNodeAndHash != null) {
+ && context.getSessionVariable().isEnableHboOptimization()
+ && hboPlanStatisticsProvider.hasAnyHboPlanStats()) {
+ Optional<PlanNodeAndHash> planNodeAndHashOpt;
+ // the publish path keys a join with the constant agnostic
form, and that is the
+ // form which the read side publishes for skew ratios as well
+ planNodeAndHashOpt =
HboUtils.getHboPlanNodeAndHash(physicalHashJoin,
Review Comment:
[P2] Check learned freshness before applying the skew discount
Consider the reduced plan `HashJoin(t.k=u.k, Scan(t), Scan(u))` over
unpartitioned tables. Collect a qualifying shuffle profile with runtime filters
OFF and skew above the threshold, then grow the same inputs from 1,000 to
100,000 rows using uniformly distributed keys. The new version-free fingerprint
still matches, and `getMatchedPlanStatistics` accepts the unchanged scan
predicates. `HboStatsCalculator` rejects that observation as STALE, but this
independent lookup still consumes its old skew and multiplies the broadcast
input CPU contribution by 0.1. It can therefore bias distribution selection
after the measured skew no longer describes the data. Retain the matched entry
and check its input freshness against the current group before applying the
discount.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
