zy-kkk opened a new pull request, #19639:
URL: https://github.com/apache/doris/pull/19639
# Proposed changes
Issue Number: close #xxx
## Problem summary
When I set the `storage_page_cache_shard_size` parameter to something other
than a power of two, the be would crash and fail to start
```
0515 14:26:21.037065 3604161 daemon.cpp:214] OS physical memory 62.06 GB.
Process memory usage 435.77 MB, limit 55.85 GB, soft limit 50.27 GB. Sys
available memory 55.94 GB, low water mark 1.60 GB, warning water mark 3.20 GB.
Refresh interval memory growth 0 B
I0515 14:26:21.060109 3604119 load_path_mgr.cpp:71] Load path configured to
[/mnt/disk1/data/doris/doris-master/storage/mini_download]
I0515 14:26:21.060180 3604488 result_buffer_mgr.cpp:147] result buffer
manager cancel thread begin.
F0515 14:26:21.066411 3604119 lru_cache.cpp:528] Check failed: (num_shards &
(num_shards - 1)) == 0 (14 vs. 0) num_shards should be power of two, but got 15
*** Check failure stack trace: ***
@ 0x55b7a6295e1d google::LogMessage::Fail()
@ 0x55b7a6298359 google::LogMessage::SendToLog()
@ 0x55b7a6295986 google::LogMessage::Flush()
@ 0x55b7a62989c9 google::LogMessageFatal::~LogMessageFatal()
@ 0x55b79f27d74d doris::ShardedLRUCache::ShardedLRUCache()
@ 0x55b79f27e53e doris::new_lru_cache()
@ 0x55b79f356f06 doris::StoragePageCache::StoragePageCache()
@ 0x55b79f356d5a doris::StoragePageCache::create_global_cache()
@ 0x55b79f4357e4 doris::ExecEnv::_init_mem_env()
@ 0x55b79f434b01 doris::ExecEnv::_init()
@ 0x55b79f433c5e doris::ExecEnv::init()
@ 0x55b79eb731ba main
@ 0x7efe5e669083 __libc_start_main
@ 0x55b79eb7202a _start
@ (nil) (unknown)
*** Query id: 0-0 ***
*** Aborted at 1684131981 (unix time) try "date -d @1684131981" if you are
using GNU date ***
*** Current BE git commitID: 92bf485abd ***
*** SIGABRT unknown detail explain (@0x36fe97) received by PID 3604119 (TID
3604119 OR 0x7efe5e4c8d00) from PID 3604119; stack trace: ***
I0515 14:26:21.438664 3604161 daemon.cpp:214] OS physical memory 62.06 GB.
Process memory usage 672.64 MB, limit 55.85 GB, soft limit 50.27 GB. Sys
available memory 55.87 GB, low water mark 1.60 GB, warning water mark 3.20 GB.
Refresh interval memory growth 0 B
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at
/mnt/disk1/compile/doris-master/be/src/common/signal_handler.h:413
1# 0x00007EFE5E688090 in /lib/x86_64-linux-gnu/libc.so.6
2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
4# 0x000055B7A62A0809 in /mnt/disk1/software/doris-master/be/lib/doris_be
5# 0x000055B7A6295E1D in /mnt/disk1/software/doris-master/be/lib/doris_be
6# google::LogMessage::SendToLog() in
/mnt/disk1/software/doris-master/be/lib/doris_be
7# google::LogMessage::Flush() in
/mnt/disk1/software/doris-master/be/lib/doris_be
8# google::LogMessageFatal::~LogMessageFatal() in
/mnt/disk1/software/doris-master/be/lib/doris_be
9# doris::ShardedLRUCache::ShardedLRUCache(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, unsigned long,
doris::LRUCacheType, unsigned int, unsigned int) at
/mnt/disk1/compile/doris-master/be/src/olap/lru_cache.cpp:528
10# doris::new_lru_cache(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, unsigned long,
doris::LRUCacheType, unsigned int) at
/mnt/disk1/compile/doris-master/be/src/olap/lru_cache.cpp:663
11# doris::StoragePageCache::StoragePageCache(unsigned long, int, unsigned
int) at /mnt/disk1/compile/doris-master/be/src/olap/page_cache.cpp:45
12# doris::StoragePageCache::create_global_cache(unsigned long, int,
unsigned int) at /mnt/disk1/compile/doris-master/be/src/olap/page_cache.cpp:31
13# doris::ExecEnv::_init_mem_env() at
/mnt/disk1/compile/doris-master/be/src/runtime/exec_env_init.cpp:234
14# doris::ExecEnv::_init(std::vector<doris::StorePath,
std::allocator<doris::StorePath> > const&) at
/mnt/disk1/compile/doris-master/be/src/runtime/exec_env_init.cpp:178
15# doris::ExecEnv::init(doris::ExecEnv*, std::vector<doris::StorePath,
std::allocator<doris::StorePath> > const&) at
/mnt/disk1/compile/doris-master/be/src/runtime/exec_env_init.cpp:97
16# main at /mnt/disk1/compile/doris-master/be/src/service/doris_main.cpp:432
17# __libc_start_main at ../csu/libc-start.c:342
18# _start in /mnt/disk1/software/doris-master/be/lib/doris_be
```
So in this PR, even if the user sets the wrong configuration, the be will
still start normally, but it will prompt an alarm message in the log
`W0515 14:37:40.072067 3876133696 exec_env_init.cpp:239] num_shards should
be power of two, but got 104. Rounded up to 128. Please modify the
'storage_page_cache_shard_size' parameter in your conf file to be a power of
two for better performance.`
## Checklist(Required)
* [ ] Does it affect the original behavior
* [ ] Has unit tests been added
* [ ] Has document been added or modified
* [ ] Does it need to update dependencies
* [ ] Is this PR support rollback (If NO, please explain WHY)
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
