gavinchou commented on code in PR #54694:
URL: https://github.com/apache/doris/pull/54694#discussion_r2318693373
##########
be/src/cloud/cloud_base_compaction.cpp:
##########
@@ -91,7 +99,23 @@ Status CloudBaseCompaction::prepare_compact() {
.tag("input_segments", _input_segments)
.tag("input_rowsets_data_size", _input_rowsets_data_size)
.tag("input_rowsets_index_size", _input_rowsets_index_size)
- .tag("input_rowsets_total_size", _input_rowsets_total_size);
+ .tag("input_rowsets_total_size", _input_rowsets_total_size)
+ .tag("input_rowsets_cache_data_size",
_input_rowsets_cache_data_size)
+ .tag("input_rowsets_cache_index_size",
_input_rowsets_cache_index_size);
+
+ if (_input_rowsets_data_size > 0) {
+ base_compaction_input_rowsets_data_hit_cache_ratio.set_value(
Review Comment:
这个统计没有意义
##########
be/src/olap/rowset/rowset.h:
##########
@@ -323,6 +323,10 @@ class Rowset : public
std::enable_shared_from_this<Rowset>, public MetadataAdder
// set the rowset as a hole rowset
void set_hole_rowset(bool is_hole_rowset) { _is_hole_rowset =
is_hole_rowset; }
+ int64_t approximate_cache_data_size();
Review Comment:
cache -> cached
##########
be/src/cloud/cloud_base_compaction.cpp:
##########
@@ -38,6 +38,12 @@ bvar::Adder<uint64_t> base_output_size("base_compaction",
"output_size");
bvar::LatencyRecorder g_base_compaction_hold_delete_bitmap_lock_time_ms(
"base_compaction_hold_delete_bitmap_lock_time_ms");
+bvar::Status<double> base_compaction_input_rowsets_data_hit_cache_ratio(
+ "base_compaction_input_rowsets_data_hit_cache_ratio", 0.0);
Review Comment:
这个ratio是某一次做base compaction的ratio, 没有意义.
用bvar adder 统计累计值(从BE启动开始) 多少从cache里读了 以及 base compaction 总共读了多少数据
来统计
##########
regression-test/suites/compaction/test_filecache_with_base_compaction_thresthold.groovy:
##########
@@ -0,0 +1,309 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+import java.util.concurrent.atomic.AtomicBoolean
+import org.apache.doris.regression.suite.ClusterOptions
+import org.codehaus.groovy.runtime.IOGroovyMethods
+import org.apache.doris.regression.util.Http
+
+suite("test_filecache_with_base_compaction_thresthold", "docker") {
Review Comment:
看看是否方便加上 bvar 的测试?
##########
be/src/olap/compaction.h:
##########
@@ -232,6 +235,36 @@ class CloudCompactionMixin : public Compaction {
int64_t get_compaction_permits();
void update_compaction_level();
+
+ bool output_rowset_write_file_cache() {
Review Comment:
建议放到.cpp里实现, 函数加上注释
建议名称 `should_cache_compaction_output()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
