Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24168
to look at the new patch set (#3).
Change subject: PROTOTYPE: Turn off tcmalloc's aggressive decommit
......................................................................
PROTOTYPE: Turn off tcmalloc's aggressive decommit
This introduces a startup option tcmalloc_aggressive_memory_decommit
that tunes tcmalloc's setting. The buffer pool's support for huge
pages currently relies on aggressive decommit being set, because it
doesn't want huge pages reused and broken up. This allows setting
tcmalloc_aggressive_memory_decommit=false if madvise_huge_pages
is also false. When tcmalloc_aggressive_memory_decommit=false, this
does garbage collection in the memory maintenance thread. This also
adds a garbage collector hook to the process MemTracker to allow it
to free memory when it hits the memory limit.
This also adds a startup flag mmap_large_buffers which will use
mmap() rather than regular malloc for buffer pool allocations that
are big enough to use huge pages (regardless of whether
madvise_huge_pages is true). This bypasses tcmalloc, so
there is no longer a concern with tcmalloc reusing huge pages
and breaking them up. The code allows tcmalloc_aggressive_memory_decommit=false
if mmap_large_buffers=true (or mmap_buffers=true).
For testing purposes, this sets tcmalloc_aggressive_memory_decommit=false
and mmap_large_buffers=true by default.
Change-Id: I11a6f1a68a49ebb706e5699be9e0ee5f842bf29f
---
M be/src/common/global-flags.cc
M be/src/common/init.cc
M be/src/runtime/bufferpool/buffer-allocator.cc
M be/src/runtime/bufferpool/buffer-allocator.h
M be/src/runtime/bufferpool/buffer-pool.cc
M be/src/runtime/bufferpool/buffer-pool.h
M be/src/runtime/bufferpool/system-allocator.cc
M be/src/runtime/bufferpool/system-allocator.h
M be/src/runtime/exec-env.cc
M be/src/runtime/mem-tracker.cc
M be/src/util/CMakeLists.txt
M be/src/util/memory-metrics.cc
M be/src/util/memory-metrics.h
A be/src/util/process-memory-util.cc
A be/src/util/process-memory-util.h
M common/thrift/metrics.json
16 files changed, 218 insertions(+), 18 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/24168/3
--
To view, visit http://gerrit.cloudera.org:8080/24168
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I11a6f1a68a49ebb706e5699be9e0ee5f842bf29f
Gerrit-Change-Number: 24168
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
