This is an automated email from the ASF dual-hosted git repository.
airborne12 pushed a commit to branch branch-4.2
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-4.2 by this push:
new 788fa0669a8 [fix](regression) Wait for new index jobs in
multi-analyzer test (#69011)
788fa0669a8 is described below
commit 788fa0669a84af8071d34897dbe7e07e3f7001b1
Author: Jack <[email protected]>
AuthorDate: Wed Oct 14 09:26:38 2026 +0800
[fix](regression) Wait for new index jobs in multi-analyzer test (#69011)
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
The multi-analyzer inverted-index regression waits for the latest
schema-change and index-build jobs by table name. Doris retains
completed jobs after a table is dropped and recreated with the same
name. A local check found FINISHED jobs for this test table while the
table did not exist, so the test can mistake a historical job for the
work it just requested.
Cause:
The waits do not identify jobs created by their own ALTER TABLE and
BUILD INDEX statements. They also wait until timeout when a job enters a
failed terminal state.
Changes:
- Record the highest JobId before each asynchronous statement and wait
only for newer jobs.
- Require all new jobs to finish, fail immediately on CANCELLED or
FAILED, and include job details in failure diagnostics.
- Correct the cloud BUILD INDEX comment.
Tests:
- The focused non-cloud regression passed twice consecutively with
`./run-regression-test.sh --run -s test_multi_tokenize_index_not_built
-d inverted_index_p0 -times 2`. All six query results matched the golden
file.
- A local SQL check confirmed that completed schema-change and
build-index jobs remain visible after the table is dropped.
- Cloud mode was not run locally because no cloud cluster is available.
The exact failure stage in the original CI runs remains unconfirmed
because their logs were unavailable without credentials.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
---
.../test_multi_tokenize_index_not_built.groovy | 52 +++++++++++-----------
1 file changed, 25 insertions(+), 27 deletions(-)
diff --git
a/regression-test/suites/inverted_index_p0/test_multi_tokenize_index_not_built.groovy
b/regression-test/suites/inverted_index_p0/test_multi_tokenize_index_not_built.groovy
index 3b7424c971e..0c9814d69c5 100644
---
a/regression-test/suites/inverted_index_p0/test_multi_tokenize_index_not_built.groovy
+++
b/regression-test/suites/inverted_index_p0/test_multi_tokenize_index_not_built.groovy
@@ -23,6 +23,26 @@ suite("test_multi_tokenize_index_not_built", "p0") {
def tableName = "test_multi_tokenize_index_not_built"
+ def alterJobs = "SHOW ALTER TABLE COLUMN WHERE TableName='${tableName}'"
+ def buildJobs = "SHOW BUILD INDEX WHERE TableName='${tableName}'"
+ def latestJobId = { statement ->
+ def jobs = sql_return_maparray("${statement} ORDER BY JobId DESC LIMIT
1")
+ return jobs.isEmpty() ? -1L : jobs[0].JobId.toLong()
+ }
+ def waitForNewJobs = { statement, previousJobId, operation ->
+ def newJobs = []
+ for (int i = 0; i < 60; i++) {
+ newJobs = sql_return_maparray(statement).findAll {
it.JobId.toLong() > previousJobId }
+ def failedJobs = newJobs.findAll { it.State in ["CANCELLED",
"FAILED"] }
+ assertTrue(failedJobs.isEmpty(), "${operation} failed:
${failedJobs}")
+ if (!newJobs.isEmpty() && newJobs.every { it.State == "FINISHED"
}) {
+ return
+ }
+ sleep(1000)
+ }
+ assertTrue(false, "${operation} did not finish in time; jobs after
${previousJobId}: ${newJobs}")
+ }
+
sql "DROP TABLE IF EXISTS ${tableName}"
// Create table with one inverted index using standard analyzer (tokenizes
by whitespace)
@@ -64,24 +84,13 @@ suite("test_multi_tokenize_index_not_built", "p0") {
"""
// Add new index with keyword analyzer (no tokenization) but DO NOT build
it
+ def previousAlterJobId = latestJobId(alterJobs)
sql """
ALTER TABLE ${tableName} ADD INDEX idx_title_keyword (title)
USING INVERTED PROPERTIES("built_in_analyzer" = "none")
"""
- // Wait for schema change to complete
- def waitSchemaChange = { tbl ->
- int maxRetry = 60
- for (int i = 0; i < maxRetry; i++) {
- def result = sql "SHOW ALTER TABLE COLUMN WHERE TableName='${tbl}'
ORDER BY CreateTime DESC LIMIT 1"
- if (result.size() > 0 && result[0][9] == "FINISHED") {
- return true
- }
- sleep(1000)
- }
- return false
- }
- assertTrue(waitSchemaChange(tableName), "Schema change did not finish in
time")
+ waitForNewJobs(alterJobs, previousAlterJobId, "Schema change")
// Query using keyword/none analyzer - idx_title_keyword is NOT built
// This should NOT use idx_title_standard (which would give wrong results)
@@ -103,27 +112,16 @@ suite("test_multi_tokenize_index_not_built", "p0") {
"""
// Now build the keyword index
- // Cloud mode: BUILD INDEX ON table (builds all indexes)
+ // Cloud mode requires BUILD INDEX without an index name.
// Non-cloud mode: BUILD INDEX idx_name ON table (builds specific index)
+ def previousBuildJobId = latestJobId(buildJobs)
if (isCloudMode()) {
sql "BUILD INDEX ON ${tableName}"
} else {
sql "BUILD INDEX idx_title_keyword ON ${tableName}"
}
- // Wait for build index to complete
- def waitBuildIndex = { tbl ->
- int maxRetry = 60
- for (int i = 0; i < maxRetry; i++) {
- def result = sql "SHOW BUILD INDEX WHERE TableName='${tbl}' ORDER
BY CreateTime DESC LIMIT 1"
- if (result.size() > 0 && result[0][7] == "FINISHED") {
- return true
- }
- sleep(1000)
- }
- return false
- }
- assertTrue(waitBuildIndex(tableName), "Build index did not finish in time")
+ waitForNewJobs(buildJobs, previousBuildJobId, "Build index")
sql "SET enable_match_without_inverted_index = false"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
