This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-4.2
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-4.2 by this push:
new 3b574c87867 branch-4.2: [fix](be) Align dynamic regex gram test with
Boost fallback (#69047)
3b574c87867 is described below
commit 3b574c87867a161f67029b86135760e040b5324d
Author: Jack <[email protected]>
AuthorDate: Thu Oct 15 09:18:00 2026 +0800
branch-4.2: [fix](be) Align dynamic regex gram test with Boost fallback
(#69047)
### What problem does this PR solve?
Issue Number: None
Related PR: #67538, #68643, #68501
Problem Summary:
The branch-4.2 `RegexGramRecallTest.DynamicPatternRegexExecution` test
expects every RE2-rejected dynamic pattern to return an error. Dynamic
REGEXP now falls back to Boost.Regex when `enable_extended_regex` is
enabled. Boost accepts `timeout\Z`, `ab\hcdtimeout`, and
`timeout(?=END)`, so BE UT build 1066836 reports six assertion failures.
Port the fallback-case expectations from the master gram test. The test
now checks successful matches for Boost-valid patterns when extended
regex is enabled and still checks rejection for invalid syntax such as
`timeout\xZZ`. This changes one test file. It does not change regex
execution or gram index behavior.
The updated test file matches current master byte for byte. A local
master-to-branch-4.2 merge simulation no longer reports a conflict for
this file.
### Release note
None
### Check List (For Author)
- Test
- [ ] Regression test
- [ ] Unit Test: Not run locally. This isolated worktree has no BE UT
binary. The changed BE UT case has not run on this PR and remains a
merge check.
- [ ] Manual test
- [ ] No need to test or manual test
- Validation completed:
- [x] The fallback-case assertion block matches the master test exactly.
- [x] `clang-format` 16 dry run and the full PR format preflight passed
against `branch-4.2` at `ccb49f1f53df`.
- [x] `git diff --check` and the full PR English description check
passed.
- Behavior changed:
- [x] No. The PR changes only test expectations.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
---
.../index/inverted/gram/regex_gram_recall_test.cpp | 37 ++++++++++++++++------
1 file changed, 27 insertions(+), 10 deletions(-)
diff --git a/be/test/storage/index/inverted/gram/regex_gram_recall_test.cpp
b/be/test/storage/index/inverted/gram/regex_gram_recall_test.cpp
index 87732efcff7..b8d404ec9f0 100644
--- a/be/test/storage/index/inverted/gram/regex_gram_recall_test.cpp
+++ b/be/test/storage/index/inverted/gram/regex_gram_recall_test.cpp
@@ -52,7 +52,8 @@ enum class ScalarPath {
SUBSTRING
};
-// Inspect the scalar regex engine selected by open().
+// BE UT permits private/protected access. Inspect the state selected by
open(), without
+// replacing the production engine or using a second regex implementation as
the oracle.
void check_scalar_path(const LikeState& state, ScalarPath path) {
if (path == ScalarPath::ANY) {
return;
@@ -104,7 +105,8 @@ void check_scalar_path(const LikeState& state, ScalarPath
path) {
}
}
-// Use the expression's selected scalar engine for the recall oracle.
+// The expression chooses the real scalar engine, including Hyperscan and its
RE2 fallback.
+// Calling RE2 directly would miss differences between the compiler and the
production path.
template <typename Function = FunctionRegexpLike>
Status scalar_matches(const std::string& pattern, const
std::vector<std::string>& rows,
std::vector<bool>* matches, bool
enable_hyperscan_fallback = true,
@@ -629,7 +631,8 @@ TEST(RegexGramRecallTest,
InvalidPatternsRejectAcrossEngineOptions) {
}
TEST(RegexGramRecallTest, DynamicPatternRegexExecution) {
- // Dynamic patterns use the scalar regex function, which requires valid
RE2 syntax.
+ // A ColumnString pattern reaches the scalar regex function. Extended
regex allows
+ // Boost when RE2 rejects a pattern, even with Hyperscan fallback disabled.
const std::vector<RecallCase> regexp_cases = {
{R"(ab\vcdtimeout)",
{"ab\vcdtimeout", "ab\ncdtimeout", "ab\rcdtimeout", "unrelated"},
@@ -676,14 +679,28 @@ TEST(RegexGramRecallTest, DynamicPatternRegexExecution) {
for (bool extended : {false, true}) {
check_cases(regexp_cases, fallback, extended);
check_cases<FunctionLike>(like_cases, fallback, extended);
- const std::vector<const char*> unsupported_patterns = {
- R"(timeout\Z)", R"(ab\hcdtimeout)", R"(timeout\xZZ)",
"timeout(?=END)"};
- for (const char* pattern : unsupported_patterns) {
- SCOPED_TRACE(pattern);
+ struct FallbackCase {
+ const char* pattern;
+ bool boost_valid;
+ std::vector<bool> expected;
+ };
+ const std::vector<FallbackCase> fallback_cases = {
+ {R"(timeout\Z)", true, {false, false}},
+ {R"(ab\hcdtimeout)", true, {false, false}},
+ {R"(timeout\xZZ)", false, {}},
+ {"timeout(?=END)", true, {true, false}}};
+ for (const auto& test_case : fallback_cases) {
+ SCOPED_TRACE(test_case.pattern);
std::vector<bool> matches;
- auto status = scalar_matches(pattern, {"timeoutEND",
"unrelated"}, &matches,
- fallback, extended,
ScalarPath::DYNAMIC_REGEX);
- EXPECT_FALSE(status.ok());
+ auto status =
+ scalar_matches(test_case.pattern, {"timeoutEND",
"unrelated"}, &matches,
+ fallback, extended,
ScalarPath::DYNAMIC_REGEX);
+ if (extended && test_case.boost_valid) {
+ ASSERT_TRUE(status.ok()) << status;
+ EXPECT_EQ(matches, test_case.expected);
+ } else {
+ EXPECT_FALSE(status.ok());
+ }
}
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
