I think I have found a solution for this bug. >From what I have observed and read, VmallocUsed and SwapCached should not be included in the stack forming total memory and they should not be deducted from apps. Most of the servers I monitor do not have significant amount of either of these, but on three servers with memory hungry applications the problem shows up (kernel versions 2.6.7, 2.6.12 and 2.6.15.1).
See my comment on Fri Apr 7 00:38:46 2006 here: http://munin.projects.linpro.no/ticket/119 I think the attached patch should fix this bug.
--- munin-1.2.4/node/node.d.linux/memory.in 2005-10-08 13:10:59.000000000 +0300 +++ memory 2006-04-07 01:27:00.000000000 +0300 @@ -68,14 +68,6 @@ { print "page_tables "; } - if (exists $mems{'SwapCached'}) - { - print "swap_cache "; - } - if (exists $mems{'VmallocUsed'}) - { - print "vmalloc_used "; - } if (exists $mems{'Slab'}) { print "slab "; @@ -106,12 +98,6 @@ print "slab.draw STACK\n"; print "slab.info Memory used by the kernel (major users are caches like inode, dentry, etc).\n"; } - if (exists $mems{'SwapCached'}) - { - print "swap_cache.label swap_cache\n"; - print "swap_cache.draw STACK\n"; - print "swap_cache.info A piece of memory that keeps track of pages that have been fetched from swap but not yet been modified.\n"; - } if (exists $mems{'PageTables'}) { print "page_tables.label page_tables\n"; @@ -121,9 +107,15 @@ if (exists $mems{'VmallocUsed'}) { print "vmalloc_used.label vmalloc_used\n"; - print "vmalloc_used.draw STACK\n"; + print "vmalloc_used.draw LINE2\n"; print "vmalloc_used.info Virtual memory used by the kernel (used when the memory does not have to be physically contigious).\n"; } + if (exists $mems{'SwapCached'}) + { + print "swap_cache.label swap_cache\n"; + print "swap_cache.draw LINE2\n"; + print "swap_cache.info A piece of memory that keeps track of pages that have been fetched from swap but not yet been modified.\n"; + } if (exists $mems{'Committed_AS'}) { print "committed.label committed\n"; @@ -190,14 +182,6 @@ { $mems{'Slab'} = 0; } -if (exists $mems{'SwapCached'}) -{ - print "swap_cache.value ", $mems{'SwapCached'}, "\n"; -} -else -{ - $mems{'SwapCached'} = 0; -} if (exists $mems{'PageTables'}) { print "page_tables.value ", $mems{'PageTables'}, "\n"; @@ -206,29 +190,35 @@ { $mems{'PageTables'} = 0; } -if (exists $mems{'VmallocUsed'}) -{ - print "vmalloc_used.value ", $mems{'VmallocUsed'}, "\n"; -} -else -{ - $mems{'VmallocUsed'} = 0; -} print "apps.value ", $mems{'MemTotal'} -$mems{'MemFree'} -$mems{'Buffers'} -$mems{'Cached'} - -$mems{'SwapCached'} -$mems{'Slab'} -$mems{'PageTables'} - -$mems{'VmallocUsed'} ,"\n"; print "free.value ", $mems{'MemFree'}, "\n"; print "buffers.value ", $mems{'Buffers'}, "\n"; print "cached.value ", $mems{'Cached'}, "\n"; print "swap.value ", $mems{'SwapTotal'} - $mems{'SwapFree'}, "\n"; +if (exists $mems{'SwapCached'}) +{ + print "swap_cache.value ", $mems{'SwapCached'}, "\n"; +} +else +{ + $mems{'SwapCached'} = 0; +} +if (exists $mems{'VmallocUsed'}) +{ + print "vmalloc_used.value ", $mems{'VmallocUsed'}, "\n"; +} +else +{ + $mems{'VmallocUsed'} = 0; +} if (exists $mems{'Committed_AS'}) { print "committed.value ", $mems{'Committed_AS'}, "\n";