xinyiZzz opened a new pull request, #38494: URL: https://github.com/apache/doris/pull/38494
when construct workload group, mem_limit is equal to (process_memory_limit * group_limit_percent), here, it is assumed that the available memory of workload groups is equal to process_memory_limit. but process_memory_usage is actually bigger than all_workload_groups_mem_usage, because public_memory of page cache, allocator cache, segment cache etc. are included in process_memory_usage. so actual available memory of the workload groups is equal to (process_memory_limit - public_memory) we will exclude this public_memory when calculate workload group mem_limit. so a ratio is calculated to multiply the workload group mem_limit from the previous construction. if all_workload_groups_mem_usage is greater than process_memory_usage, it means that the memory statistics of the workload group are inaccurate. the reason is that query/load/etc. tracked is virtual memory, and virtual memory is not used in time. At this time, weighted_memory_limit_ratio is equal to 1, and workload group mem_limit is still equal to (process_memory_limit * group_limit_percent), this may cause query spill to occur earlier, However, there is no good solution at present, but we cannot predict when these virtual memory will be used. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
