On 20/07/15 23:47, Trevor Gale wrote: > I am building a small scale cluster (most likely around 8-10 nodes). > I was wondering if anyone has any general optimizations that they > like to implement on their clusters.
These are the sysctl settings we push out to our x86 clusters, one set for GPFS tuning (set by our on-site IBM staff over the past few years), one set to disable the OOM killer by getting malloc() to fail when there's no memory for it to work and one to disable zone_reclaim_mode for reasons previously discussed on the list (though checking it looks like RHEL6 disables it by default these days): :::::::::::::: gpfs.sysctl :::::::::::::: # # These are changes to the kernel configuration recommend by GPFS # # Memory # vm.min_free_kbytes = 370560 # # Communications # net.core.rmem_max = 8388608 net.core.wmem_max = 8388608 net.ipv4.tcp_rmem = 4096 262144 8388608 net.ipv4.tcp_wmem = 4096 262144 8388608 net.core.netdev_max_backlog = 2500 net.ipv4.tcp_mtu_probing = 1 :::::::::::::: oom.sysctl :::::::::::::: # Stop nodes OOM'ing vm.overcommit_memory = 2 vm.overcommit_ratio = 99 :::::::::::::: zone_reclaim_mode.sysctl :::::::::::::: # Disable zone_reclaim_mode as it can cause massive # performance problems with mmap()'d IO on GPFS # (amongst other things). See RT#10793. vm.zone_reclaim_mode = 0 -- Christopher Samuel Senior Systems Administrator VLSCI - Victorian Life Sciences Computation Initiative Email: sam...@unimelb.edu.au Phone: +61 (0)3 903 55545 http://www.vlsci.org.au/ http://twitter.com/vlsci _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
