github-actions[bot] commented on code in PR #63504:
URL: https://github.com/apache/doris/pull/63504#discussion_r3338547238
##########
be/src/io/cache/block_file_cache.cpp:
##########
@@ -2314,19 +2406,28 @@ void BlockFileCache::update_ttl_atime(const
UInt128Wrapper& hash) {
void BlockFileCache::run_background_lru_log_replay() {
Thread::set_self_name("run_background_lru_log_replay");
while (!_close) {
- int64_t interval_ms =
config::file_cache_background_lru_log_replay_interval_ms;
+ size_t backlog = _lru_recorder->get_total_lru_log_queue_size();
+ QueueConsumePlan plan = build_lru_log_replay_plan(backlog);
{
std::unique_lock close_lock(_close_mtx);
- _close_cv.wait_for(close_lock,
std::chrono::milliseconds(interval_ms));
+ _close_cv.wait_for(close_lock,
std::chrono::milliseconds(plan.interval_ms));
if (_close) {
break;
}
Review Comment:
This bounded replay can leave log records in `_lru_recorder` while
`run_background_lru_dump()` concurrently dumps the shadow queue.
`CacheLRUDumper::dump_queue()` persists that shadow queue and then resets
`_lru_queue_update_cnt_from_last_dump`; any log records that were already
counted but not yet replayed will later update the shadow queue without
increasing the dump counter again. If the process crashes before enough new
events arrive to trigger another dump, restore uses a stale LRU tail and loses
those recent ADD/MOVE/REMOVE/RESIZE operations. Please make the dump path first
drain/replay all pending records for the queue, or otherwise avoid resetting
the update count until the corresponding log queue is empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
