This is an automated email from the ASF dual-hosted git repository.
jacktengg pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new c54393bcacf [test](regression) Stabilize aggregate state finalize MV
rollup checks (#69042)
c54393bcacf is described below
commit c54393bcacf441e5af34d33c290eae233e193c4e
Author: HappenLee <[email protected]>
AuthorDate: Mon Oct 19 15:03:27 2026 +0800
[test](regression) Stabilize aggregate state finalize MV rollup checks
(#69042)
### What problem does this PR solve?
Related PR: #68312
`agg_state_finalize_roll_up` can fail its chosen-MV assertion even when
rewriting succeeds. In [CloudP0 build
1066869](http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_CloudP0/1066869),
both the 700-row base table and `finalize_sum_mv` were estimated at one
row, and EXPLAIN reported `finalize_sum_mv not chose`.
Synchronous ANALYZE does not wait for tablet row-count reporting, so
data amplification alone does not guarantee a cost advantage for the MV.
Pin the actual fixture cardinalities with the existing `ALTER COLUMN SET
STATS` mechanism: 700 base rows, five rows in each successful `(k, g)`
MV, and three distinct `k` values.
Protect those statistics by disabling automatic analysis before creating
the fixture. Otherwise, a job queued before the injections can remove
the statistics through its empty-table cleanup path before checking
`userInjected`. Reuse `setGlobalVarTemporary` to restore the original
global setting on both success and failure, and mark the suite
`nonConcurrent` to isolate the global change from ordinary concurrent
suites. This prevents new automatic-analysis jobs for the fixture; it
does not cancel already-running analysis jobs.
CI coverage: the framework retains the suite's `p0` classification, but
CloudP0 explicitly runs with `-runNonConcurrent false`. The dedicated
NonConcurrent pipeline now covers this suite.
Keep the strict chosen-MV assertions for both finalizer queries and the
existing MERGE/UNION rollup, the MAX/ABS rejection checks, all inserted
data, and the expected result file unchanged. No production optimizer or
aggregation code changes.
### Release note
None
### Check List (For Author)
- Test:
- [x] Three normal comparison runs of the final patch passed on the
dedicated local ASAN cluster using the original failure's CI package,
`doris-0.0.0-a1f5d53acf`, with tablet reporting/update intervals set to
600 seconds. No expected-output generation was used.
- [x] Verified selection through both `-g p0` and `-g nonConcurrent`;
execution used `non-concurrent-thread-1`. Confirmed restoration of both
initial `enable_auto_analyze=true` and `false`.
- [x] Temporarily inserted a missing-table SQL statement to exercise the
failure path. The suite failed as expected and restored
`enable_auto_analyze=true`. Removed that temporary statement and reran
the final patch successfully.
- [x] Expected-output SHA256 unchanged; `git diff --check` passed.
- Full CloudP0 and NonConcurrent pipelines have not been rerun locally.
Local checks used shared-nothing mode and existing ASAN binaries; no
production binaries were rebuilt for this test-only change.
- Behavior changed:
- [ ] No.
- [x] Yes. Regression execution is nonconcurrent and temporarily
disables automatic analysis. Production behavior is unchanged.
- Does this need documentation?
- [x] No.
- [ ] Yes.
---
.../agg_state_finalize_roll_up.groovy | 156 ++++++++++++---------
1 file changed, 87 insertions(+), 69 deletions(-)
diff --git
a/regression-test/suites/nereids_rules_p0/mv/agg_with_roll_up/agg_state_finalize_roll_up.groovy
b/regression-test/suites/nereids_rules_p0/mv/agg_with_roll_up/agg_state_finalize_roll_up.groovy
index cf57a756b49..14d5d723bc0 100644
---
a/regression-test/suites/nereids_rules_p0/mv/agg_with_roll_up/agg_state_finalize_roll_up.groovy
+++
b/regression-test/suites/nereids_rules_p0/mv/agg_with_roll_up/agg_state_finalize_roll_up.groovy
@@ -15,78 +15,96 @@
// specific language governing permissions and limitations
// under the License.
-suite("agg_state_finalize_roll_up") {
- String db = context.config.getDbNameByFile(context.file)
- sql "set enable_agg_state=true"
- sql "set pre_materialized_view_rewrite_strategy=TRY_IN_RBO"
- for (String mv : ["finalize_max_mv", "finalize_abs_mv", "finalize_sum_mv",
"finalize_union_mv"]) {
- sql "drop materialized view if exists ${mv}"
- }
- sql "drop table if exists agg_finalize_rollup_input"
- sql """
- create table agg_finalize_rollup_input (k int, g int, v double)
- duplicate key(k, g) distributed by hash(k) buckets 1
- properties("replication_num"="1")
- """
- sql """
- insert into agg_finalize_rollup_input values
- (1, 1, -4), (1, 1, 3), (1, 2, -2), (1, 2, 7),
- (2, 1, 5), (2, 2, null), (3, 1, null)
- """
- // Make the finer-group MV cheaper than scanning the base table so the
test executes its rollup.
- sql """
- insert into agg_finalize_rollup_input
- select k, g, v from agg_finalize_rollup_input cross join
numbers("number"="99")
- """
- sql "analyze table agg_finalize_rollup_input with sync"
+suite("agg_state_finalize_roll_up", "nonConcurrent") {
+ // Disable automatic analysis before creating the fixture to prevent
background jobs
+ // from being queued before statistics are injected. Restore the global
setting on failure.
+ setGlobalVarTemporary([enable_auto_analyze: false], {
+ String db = context.config.getDbNameByFile(context.file)
+ sql "set enable_agg_state=true"
+ sql "set pre_materialized_view_rewrite_strategy=TRY_IN_RBO"
+ for (String mv : ["finalize_max_mv", "finalize_abs_mv",
"finalize_sum_mv", "finalize_union_mv"]) {
+ sql "drop materialized view if exists ${mv}"
+ }
+ sql "drop table if exists agg_finalize_rollup_input"
+ sql """
+ create table agg_finalize_rollup_input (k int, g int, v double)
+ duplicate key(k, g) distributed by hash(k) buckets 1
+ properties("replication_num"="1")
+ """
+ sql """
+ insert into agg_finalize_rollup_input values
+ (1, 1, -4), (1, 1, 3), (1, 2, -2), (1, 2, 7),
+ (2, 1, 5), (2, 2, null), (3, 1, null)
+ """
+ // Make the finer-group MV cheaper than scanning the base table so the
test executes its rollup.
+ sql """
+ insert into agg_finalize_rollup_input
+ select k, g, v from agg_finalize_rollup_input cross join
numbers("number"="99")
+ """
+ sql "analyze table agg_finalize_rollup_input with sync"
+ // Synchronous ANALYZE does not wait for tablet row-count reports. Pin
the fixture
+ // cardinalities so CBO chooses the rollup independently of report
timing.
+ sql """
+ alter table agg_finalize_rollup_input modify column k
+ set stats ('row_count'='700', 'ndv'='3')
+ """
- String combinedQuery = """
- select k, sum_finalize(sum_combine(avg_finalize(avg_state(v))))
- from agg_finalize_rollup_input group by k
- """
- String plainQuery = """
- select k, sum(avg_finalize(avg_state(v)))
- from agg_finalize_rollup_input group by k
- """
- order_qt_combined_before combinedQuery
- order_qt_plain_before plainQuery
+ String combinedQuery = """
+ select k, sum_finalize(sum_combine(avg_finalize(avg_state(v))))
+ from agg_finalize_rollup_input group by k
+ """
+ String plainQuery = """
+ select k, sum(avg_finalize(avg_state(v)))
+ from agg_finalize_rollup_input group by k
+ """
+ order_qt_combined_before combinedQuery
+ order_qt_plain_before plainQuery
- // Sharing the inner AVG finalizer must not make SUM compatible with MAX.
- create_async_mv(db, "finalize_max_mv", """
- select k, g, max_combine(avg_finalize(avg_state(v))) s
- from agg_finalize_rollup_input group by k, g
- """)
- mv_rewrite_fail(combinedQuery, "finalize_max_mv")
- mv_rewrite_fail(plainQuery, "finalize_max_mv")
- order_qt_different_aggregate combinedQuery
+ // Sharing the inner AVG finalizer must not make SUM compatible with
MAX.
+ create_async_mv(db, "finalize_max_mv", """
+ select k, g, max_combine(avg_finalize(avg_state(v))) s
+ from agg_finalize_rollup_input group by k, g
+ """)
+ mv_rewrite_fail(combinedQuery, "finalize_max_mv")
+ mv_rewrite_fail(plainQuery, "finalize_max_mv")
+ order_qt_different_aggregate combinedQuery
- // The full value expression matters even when both outer aggregates are
SUM.
- create_async_mv(db, "finalize_abs_mv", """
- select k, g, sum_combine(abs(avg_finalize(avg_state(v)))) s
- from agg_finalize_rollup_input group by k, g
- """)
- mv_rewrite_fail(combinedQuery, "finalize_abs_mv")
- mv_rewrite_fail(plainQuery, "finalize_abs_mv")
- order_qt_different_argument combinedQuery
+ // The full value expression matters even when both outer aggregates
are SUM.
+ create_async_mv(db, "finalize_abs_mv", """
+ select k, g, sum_combine(abs(avg_finalize(avg_state(v)))) s
+ from agg_finalize_rollup_input group by k, g
+ """)
+ mv_rewrite_fail(combinedQuery, "finalize_abs_mv")
+ mv_rewrite_fail(plainQuery, "finalize_abs_mv")
+ order_qt_different_argument combinedQuery
- create_async_mv(db, "finalize_sum_mv", """
- select k, g, sum_combine(avg_finalize(avg_state(v))) s
- from agg_finalize_rollup_input group by k, g
- """)
- mv_rewrite_success(combinedQuery, "finalize_sum_mv")
- mv_rewrite_success(plainQuery, "finalize_sum_mv")
- order_qt_combined_after combinedQuery
- order_qt_plain_after plainQuery
+ create_async_mv(db, "finalize_sum_mv", """
+ select k, g, sum_combine(avg_finalize(avg_state(v))) s
+ from agg_finalize_rollup_input group by k, g
+ """)
+ sql """
+ alter table finalize_sum_mv modify column k
+ set stats ('row_count'='5', 'ndv'='3')
+ """
+ mv_rewrite_success(combinedQuery, "finalize_sum_mv")
+ mv_rewrite_success(plainQuery, "finalize_sum_mv")
+ order_qt_combined_after combinedQuery
+ order_qt_plain_after plainQuery
- // Existing MERGE/UNION/STATE rollup remains available.
- create_async_mv(db, "finalize_union_mv", """
- select k, g, sum_union(sum_state(v)) s
- from agg_finalize_rollup_input group by k, g
- """)
- String mergeQuery = """
- select k, sum_merge(sum_state(v))
- from agg_finalize_rollup_input group by k
- """
- mv_rewrite_success(mergeQuery, "finalize_union_mv")
- order_qt_merge_after mergeQuery
+ // Existing MERGE/UNION/STATE rollup remains available.
+ create_async_mv(db, "finalize_union_mv", """
+ select k, g, sum_union(sum_state(v)) s
+ from agg_finalize_rollup_input group by k, g
+ """)
+ sql """
+ alter table finalize_union_mv modify column k
+ set stats ('row_count'='5', 'ndv'='3')
+ """
+ String mergeQuery = """
+ select k, sum_merge(sum_state(v))
+ from agg_finalize_rollup_input group by k
+ """
+ mv_rewrite_success(mergeQuery, "finalize_union_mv")
+ order_qt_merge_after mergeQuery
+ })
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
