On Fri, 14 Dec 2007, Mark Hahn wrote: > (it also depends on whether your glibc contains a malloc that > switches to mmap for large allocations rather than sbrk, etc.)
I was looking at that the other day wondering why the maximum RAM & data segment size limits set by Torque with -l mem=2g were not being enforced. After a bit of head scratching I tracked it down to the fact that between somewhere around glibc 2.3 the old malloc() implementation using brk() was ripped out and replaced with one that uses mmap() for allocations of 128KB or more. Unfortunately the kernel implementation of mmap() doesn't check the maximum memory size (RLIMIT_RSS) or maximum data size (RLIMIT_DATA) limits which were being set, but only the maximum virtual RAM size (RLIMIT_AS) - this is documented in the setrlimit(2) man page: > RLIMIT_AS > The maximum size of the process’s virtual memory > (address space) in bytes. This limit affects calls > to brk(2), mmap(2) and mremap(2), which fail with > the error ENOMEM upon exceeding this limit. (it also says that RLIMIT_RSS hasn't worked since 2.4.29, which seems to be born out by a quick grep of 2.6.24-rc3 I have to hand) In other words you can set a low memory limit of say 10MB with: $ ulimit -m $((10*1024) and then run a program that allocates 2GB RAM in large chunks successfully and only fails when it tries to request a trivial amount of RAM. :-( I've submitted a patch for Torque to set RLIMIT_AS as well as RLIMIT_RSS & RLIMIT_DATA. cheers, Chris -- Christopher Samuel - (03) 9925 4751 - Systems Manager The Victorian Partnership for Advanced Computing P.O. Box 201, Carlton South, VIC 3053, Australia VPAC is a not-for-profit Registered Research Agency
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
