github-actions[bot] commented on code in PR #69087: URL: https://github.com/apache/doris/pull/69087#discussion_r4291168463
########## regression-test/suites/fault_injection_p0/test_segment_limit_condition_cache.groovy: ########## @@ -0,0 +1,74 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +// A storage TopN (ORDER BY key LIMIT n) stops each segment after n rows. It must not leave +// a condition cache entry behind, or a later query with the same predicate skips row ranges +// that do have matching rows. +// +// The bug needs a rowset with overlapping segments, which is read by VMergeIterator: it asks +// every segment for one more batch, so a segment that hit its LIMIT reaches EOF. +// MemTable.need_flush writes one segment per inserted block. +suite("test_segment_limit_condition_cache", "nonConcurrent") { + def tbl = "test_segment_limit_condition_cache" + sql "DROP TABLE IF EXISTS ${tbl}" + sql """ + CREATE TABLE ${tbl} (k INT, v INT) + DUPLICATE KEY(k) + DISTRIBUTED BY HASH(k) BUCKETS 1 + PROPERTIES ('replication_num' = '1', 'disable_auto_compaction' = 'true') + """ + + try { + GetDebugPoint().enableDebugPointForAllBEs("MemTable.need_flush") + // 4 blocks of 4096 rows -> 4 segments, each larger than one 2048-row cache granule. + // Every block has keys from the whole range, so the segments overlap. + sql "SET batch_size = 4096" + sql """ + INSERT INTO ${tbl} + SELECT (number * 7919) % 16384, (number * 7919) % 16384 % 7 - 3 + FROM numbers('number' = '16384') + """ + } finally { + GetDebugPoint().disableDebugPointForAllBEs("MemTable.need_flush") + } + + def tablets = sql_return_maparray("SHOW TABLETS FROM ${tbl}") + def (code, out, err) = curl("GET", tablets[0].MetaUrl) + assertEquals(0, code) + def hasMergeLayout = parseJson(out.trim()).rs_metas.any { meta -> + meta.segments_overlap_pb == "OVERLAPPING" && meta.num_segments > 1 + } + assertTrue(hasMergeLayout, "expected an OVERLAPPING rowset with more than one segment") + + // Both switches are randomized in fuzzy runs; this case needs both on. + sql "SET enable_condition_cache = true" + sql "SET enable_segment_limit_pushdown = true" Review Comment: [P2] Disable parallel scans in the cache regression Please also set `enable_parallel_scan=false` before these queries. Fuzzy settings can enable parallel scanning with `parallel_scan_max_scanners_count=32` and `parallel_scan_min_rows_per_scanner=64` (clamped to 1024 by BE), splitting these 4096-row segments into scanners that each see only one segment. `BetaRowsetReader::is_merge_iterator()` then returns false, and storage TopN stops after the first LIMIT-1 block without requesting SegmentIterator EOF, so the old cache-publication code is never exercised. The rowset layout and EXPLAIN assertions still pass, allowing this regression to pass with the cache fix reverted. This is separate from the existing runtime-filter issue: `topn_filter_ratio=0` keeps caching eligible but does not prevent these splits. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
