[doris] 03/05: [fix](memory) Fix MacOS mem_limit parse error and GC after env Init #17528

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git

commit f47fbe9979aad24261e4300b7e98e72fb2e04d29
Author: Xinyi Zou <zouxiny...@gmail.com>
AuthorDate: Wed Mar 8 14:00:57 2023 +0800

    [fix](memory) Fix MacOS mem_limit parse error and GC after env Init #17528
    
    Fix MacOS mem_limit parse result is 0.
    Fix GC after env Init, otherwise, when the memory is insufficient, BE will 
start failure.
    *** Query id: 0-0 ***
    *** Aborted at 1677833773 (unix time) try "date -d @1677833773" if you are 
using GNU date ***
    *** Current BE git commitID: 8ee5f45 ***
    *** SIGSEGV address not mapped to object (@0x70) received by PID 24145 (TID 
0x7fa53c9fd700) from PID 112; stack trace: ***
     0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, 
siginfo_t*, void*) at be/src/common/signal_handler.h:420
     1# os::Linux::chained_handler(int, siginfo*, void*) in 
/usr/local/jdk/jre/lib/amd64/server/libjvm.so
     2# JVM_handle_linux_signal in /usr/local/jdk/jre/lib/amd64/server/libjvm.so
     3# signalHandler(int, siginfo*, void*) in 
/usr/local/jdk/jre/lib/amd64/server/libjvm.so
     4# 0x00007FA56295A400 in /lib64/libc.so.6
     5# 
doris::MemTrackerLimiter::log_process_usage_str(std::__cxx11::basic_string<char,
 std::char_traits<char>, std::allocator<char> > const&, bool) at 
be/src/runtime/memory/mem_tracker_limiter.cpp:208
     6# 
doris::MemTrackerLimiter::print_log_process_usage(std::__cxx11::basic_string<char,
 std::char_traits<char>, std::allocator<char> > const&, bool) at 
be/src/runtime/memory/mem_tracker_limiter.cpp:226
     7# doris::Daemon::memory_maintenance_thread() at 
be/src/common/daemon.cpp:245
     8# doris::Thread::supervise_thread(void*) at be/src/util/thread.cpp:455
     9# start_thread in /lib64/libpthread.so.0
    10# clone in /lib64/libc.so.6
---
 be/src/common/daemon.cpp | 2 +-
 be/src/util/mem_info.cpp | 9 +++++++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/be/src/common/daemon.cpp b/be/src/common/daemon.cpp
index d36bf08e8c..f45eb1700e 100644
--- a/be/src/common/daemon.cpp
+++ b/be/src/common/daemon.cpp
@@ -247,7 +247,7 @@ void Daemon::memory_gc_thread() {
     int64_t cache_gc_freed_mem = 0;
     while (!_stop_background_threads_latch.wait_for(
             std::chrono::milliseconds(interval_milliseconds))) {
-        if (!MemInfo::initialized()) {
+        if (!MemInfo::initialized() || !ExecEnv::GetInstance()->initialized()) 
{
             continue;
         }
         if (memory_full_gc_sleep_time_ms <= 0 &&
diff --git a/be/src/util/mem_info.cpp b/be/src/util/mem_info.cpp
index 818223c1a0..bb0f6bb172 100644
--- a/be/src/util/mem_info.cpp
+++ b/be/src/util/mem_info.cpp
@@ -311,8 +311,13 @@ void MemInfo::init() {
         _s_physical_mem = -1;
     }
 
-    bool is_percent = true;
-    _s_mem_limit = ParseUtil::parse_mem_spec(config::mem_limit, -1, 
_s_physical_mem, &is_percent);
+    if (config::mem_limit == "auto") {
+        _s_mem_limit = std::max<int64_t>(_s_physical_mem * 0.9, 
_s_physical_mem - 6871947672);
+    } else {
+        bool is_percent = true;
+        _s_mem_limit =
+                ParseUtil::parse_mem_spec(config::mem_limit, -1, 
_s_physical_mem, &is_percent);
+    }
     _s_mem_limit_str = PrettyPrinter::print(_s_mem_limit, TUnit::BYTES);
     _s_soft_mem_limit = _s_mem_limit * config::soft_mem_limit_frac;
     _s_soft_mem_limit_str = PrettyPrinter::print(_s_soft_mem_limit, 
TUnit::BYTES);


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org