gavinchou commented on code in PR #47473:
URL: https://github.com/apache/doris/pull/47473#discussion_r1938527170
##########
be/src/io/cache/block_file_cache.cpp:
##########
@@ -1648,11 +1666,69 @@ void BlockFileCache::check_disk_resource_limit() {
}
}
+void BlockFileCache::check_need_evict_cache_in_advance() {
+ if (_storage->get_type() != FileCacheStorageType::DISK) {
+ return;
+ }
+
+ std::pair<int, int> percent;
+ int ret = disk_used_percentage(_cache_base_path, &percent);
+ if (ret != 0) {
+ LOG_ERROR("").tag("file cache path", _cache_base_path).tag("error",
strerror(errno));
+ return;
+ }
+ auto [space_percentage, inode_percentage] = percent;
+ size_t size_percentage = static_cast<size_t>(
+ (static_cast<double>(_cur_cache_size) /
static_cast<double>(_capacity)) * 100);
+ auto is_insufficient = [](const int& percentage) {
+ return percentage >=
config::file_cache_enter_need_evict_cache_in_advance_percent;
+ };
+ DCHECK_GE(space_percentage, 0);
+ DCHECK_LE(space_percentage, 100);
+ DCHECK_GE(inode_percentage, 0);
+ DCHECK_LE(inode_percentage, 100);
+ // ATTN: due to that can be changed dynamically, set it to default value
if it's invalid
+ // FIXME: reject with config validator
+ if (config::file_cache_enter_need_evict_cache_in_advance_percent <=
+ config::file_cache_exit_need_evict_cache_in_advance_percent) {
+ LOG_WARNING("config error, set to default value")
+ .tag("enter",
config::file_cache_enter_need_evict_cache_in_advance_percent)
+ .tag("exit",
config::file_cache_exit_need_evict_cache_in_advance_percent);
+ config::file_cache_enter_need_evict_cache_in_advance_percent = 78;
Review Comment:
file_cache_enter_need_evict_cache_in_advance_percent and
file_cache_exit_need_evict_cache_in_advance_percent are not a standalone config
pair, they are affected by file_cache_enter_resource_limit_mode_percent
##########
be/src/common/config.cpp:
##########
@@ -1048,6 +1048,12 @@ DEFINE_Bool(clear_file_cache, "false");
DEFINE_Bool(enable_file_cache_query_limit, "false");
DEFINE_mInt32(file_cache_enter_disk_resource_limit_mode_percent, "88");
DEFINE_mInt32(file_cache_exit_disk_resource_limit_mode_percent, "80");
+DEFINE_mBool(enable_evict_file_cache_in_advance, "true");
+DEFINE_mInt32(file_cache_enter_need_evict_cache_in_advance_percent, "88");
+DEFINE_mInt32(file_cache_exit_need_evict_cache_in_advance_percent, "80");
Review Comment:
88 and 80 are not consistent with the "default" values in function
`BlockFileCache::check_need_evict_cache_in_advance()`
##########
be/src/common/config.cpp:
##########
@@ -1048,6 +1048,12 @@ DEFINE_Bool(clear_file_cache, "false");
DEFINE_Bool(enable_file_cache_query_limit, "false");
DEFINE_mInt32(file_cache_enter_disk_resource_limit_mode_percent, "88");
DEFINE_mInt32(file_cache_exit_disk_resource_limit_mode_percent, "80");
+DEFINE_mBool(enable_evict_file_cache_in_advance, "true");
+DEFINE_mInt32(file_cache_enter_need_evict_cache_in_advance_percent, "88");
+DEFINE_mInt32(file_cache_exit_need_evict_cache_in_advance_percent, "80");
+DEFINE_mInt32(file_cache_evict_in_advance_interval_ms, "1000");
+DEFINE_mInt64(file_cache_evict_in_advance_batch_bytes, "31457280"); // 30MB
Review Comment:
Too may handles for the functionality "evict in advance", e.g. what if we
set `file_cache_enter_need_evict_cache_in_advance_percent` with a number larger
than 100, is `enable_evict_file_cache_in_advance` still needed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
