xinyiZzz commented on code in PR #37066:
URL: https://github.com/apache/doris/pull/37066#discussion_r1665289269
##########
be/src/runtime/thread_context.h:
##########
@@ -460,17 +478,22 @@ class ScopeSkipMemoryCheck {
// Mem Hook to consume thread mem tracker
#define CONSUME_THREAD_MEM_TRACKER_BY_HOOK(size) \
do { \
+ allocator_detect(size); \
if (doris::use_mem_hook) { \
doris::thread_context()->consume_memory(size); \
} \
} while (0)
+
#define RELEASE_THREAD_MEM_TRACKER_BY_HOOK(size)
CONSUME_THREAD_MEM_TRACKER_BY_HOOK(-size)
+
#define CONSUME_THREAD_MEM_TRACKER_BY_HOOK_WITH_FN(size_fn, ...) \
do { \
+ allocator_detect(size_fn(__VA_ARGS__)); \
Review Comment:
```
#ifndef NDEBUG
allocator_detect(size_fn(__VA_ARGS__));
#endif
```
avoid unnecessary function calls `size_fn(__VA_ARGS__)` and
`allocator_detect`
##########
be/src/vec/common/allocator.cpp:
##########
@@ -244,13 +244,15 @@ void Allocator<clear_memory_, mmap_populate,
use_mmap>::remove_address_sanitizer
template <bool clear_memory_, bool mmap_populate, bool use_mmap>
void* Allocator<clear_memory_, mmap_populate, use_mmap>::alloc(size_t size,
size_t alignment) {
+ doris::ScopedAllocatorTagger tagger;
Review Comment:
```
doris::allocator_working = true;
Defer defer {[] { doris::allocator_working = false; }};
```
seems simpler
##########
be/src/vec/common/allocator.cpp:
##########
@@ -244,13 +244,15 @@ void Allocator<clear_memory_, mmap_populate,
use_mmap>::remove_address_sanitizer
template <bool clear_memory_, bool mmap_populate, bool use_mmap>
void* Allocator<clear_memory_, mmap_populate, use_mmap>::alloc(size_t size,
size_t alignment) {
+ doris::ScopedAllocatorTagger tagger;
return alloc_impl(size, alignment);
}
template <bool clear_memory_, bool mmap_populate, bool use_mmap>
void* Allocator<clear_memory_, mmap_populate, use_mmap>::realloc(void* buf,
size_t old_size,
size_t
new_size,
size_t
alignment) {
+ doris::ScopedAllocatorTagger tagger;
Review Comment:
```
doris::allocator_working = true;
Defer defer {[] { doris::allocator_working = false; }};
```
seems simpler
##########
be/src/common/config.cpp:
##########
@@ -1318,6 +1318,9 @@ DEFINE_Int64(min_row_group_size, "134217728");
// filter wrong data.
DEFINE_mBool(enable_parquet_page_index, "true");
+// If native memory allocator allocates more memory than this value, an
exception will be thrown.
+DEFINE_Int64(debug_max_memory_size_by_native_allocator, "1048576")
Review Comment:
`debug_max_memory_size_by_native_allocator_bytes`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
