I am just encountering the same problem on my current server. About the machine:
model name : AMD Athlon(tm) 64 Processor 3700+ cpu MHz : 2199.541 cache size : 1024 KB MemTotal: 1025708 kB cat /etc/debian_version: 6.0.7 Apache packages: ii apache2 2.2.16-6+squeeze11 ii apache2-mpm-prefork 2.2.16-6+squeeze11 ii apache2-utils 2.2.16-6+squeeze11 ii apache2.2-bin 2.2.16-6+squeeze11 ii apache2.2-common 2.2.16-6+squeeze11 PHP5 packages: rc libapache2-mod-php5 5.3.3-7+squeeze15 ii libapache2-mod-php5filter 5.3.3-7+squeeze15 ii php5-cli 5.3.3-7+squeeze15 ii php5-common 5.3.3-7+squeeze15 ii php5-curl 5.3.3-7+squeeze15 ii php5-dev 5.3.3-7+squeeze15 ii php5-gd 5.3.3-7+squeeze15 ii php5-mcrypt 5.3.3-7+squeeze15 ii php5-mysql 5.3.3-7+squeeze15 PHP caching: eaccelerator-42067ac Web and PHP applications: 3 x WordPress 1 x MediaWiki 1 x Etherpad lite 1 x status.net I let that system send me a status mail with some outputs every 15 minutes. The last one before the last crash said: cat /proc/meminfo: MemTotal: 1025708 kB MemFree: 8184 kB Buffers: 1272 kB Cached: 31704 kB SwapCached: 278060 kB Active: 473396 kB Inactive: 471092 kB Active(anon): 455708 kB Inactive(anon): 455796 kB Active(file): 17688 kB Inactive(file): 15296 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 2000052 kB SwapFree: 854820 kB Dirty: 0 kB Writeback: 1040 kB AnonPages: 639924 kB Mapped: 12632 kB Shmem: 20 kB Slab: 19988 kB SReclaimable: 8800 kB SUnreclaim: 11188 kB KernelStack: 1944 kB PageTables: 28516 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 2512904 kB Committed_AS: 2688324 kB VmallocTotal: 34359738367 kB VmallocUsed: 8124 kB VmallocChunk: 34359723104 kB HardwareCorrupted: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 9152 kB DirectMap2M: 1038336 kB Uptime: 15:31:36 up 3 days, 23:08, 0 users, load average: 58.61, 58.42, 57.90 Parts from ps fax: PID TTY STAT TIME COMMAND 354 ? S<s 0:00 udevd --daemon 1559 ? S< 0:00 \_ udevd --daemon 1560 ? S< 0:00 \_ udevd --daemon 938 ? Sl 0:04 /usr/sbin/rsyslogd -c4 981 ? Ss 0:11 /usr/sbin/apache2 -k start 13140 ? D 0:08 \_ /usr/sbin/apache2 -k start 13148 ? D 0:04 \_ /usr/sbin/apache2 -k start 13160 ? D 0:03 \_ /usr/sbin/apache2 -k start 13172 ? D 0:03 \_ /usr/sbin/apache2 -k start 13179 ? D 0:02 \_ /usr/sbin/apache2 -k start 13180 ? D 0:02 \_ /usr/sbin/apache2 -k start 13181 ? D 0:02 \_ /usr/sbin/apache2 -k start 13182 ? D 0:02 \_ /usr/sbin/apache2 -k start 13183 ? D 0:02 \_ /usr/sbin/apache2 -k start 13184 ? D 0:02 \_ /usr/sbin/apache2 -k start 13185 ? D 0:02 \_ /usr/sbin/apache2 -k start 13186 ? D 0:02 \_ /usr/sbin/apache2 -k start 13187 ? D 0:02 \_ /usr/sbin/apache2 -k start 13188 ? S 0:02 \_ /usr/sbin/apache2 -k start 13189 ? D 0:02 \_ /usr/sbin/apache2 -k start 13190 ? D 0:02 \_ /usr/sbin/apache2 -k start 13191 ? S 0:02 \_ /usr/sbin/apache2 -k start 13192 ? D 0:02 \_ /usr/sbin/apache2 -k start 13193 ? D 0:02 \_ /usr/sbin/apache2 -k start 13200 ? D 0:02 \_ /usr/sbin/apache2 -k start 13201 ? D 0:02 \_ /usr/sbin/apache2 -k start 13202 ? D 0:02 \_ /usr/sbin/apache2 -k start 13203 ? D 0:02 \_ /usr/sbin/apache2 -k start 13204 ? D 0:02 \_ /usr/sbin/apache2 -k start 13205 ? S 0:02 \_ /usr/sbin/apache2 -k start 13206 ? D 0:02 \_ /usr/sbin/apache2 -k start 13207 ? S 0:02 \_ /usr/sbin/apache2 -k start 13208 ? D 0:02 \_ /usr/sbin/apache2 -k start 13209 ? D 0:02 \_ /usr/sbin/apache2 -k start 13210 ? D 0:02 \_ /usr/sbin/apache2 -k start 13211 ? D 0:02 \_ /usr/sbin/apache2 -k start 13212 ? D 0:02 \_ /usr/sbin/apache2 -k start 13213 ? S 0:02 \_ /usr/sbin/apache2 -k start 13214 ? D 0:02 \_ /usr/sbin/apache2 -k start 13215 ? S 0:02 \_ /usr/sbin/apache2 -k start 13228 ? D 0:02 \_ /usr/sbin/apache2 -k start 13232 ? D 0:02 \_ /usr/sbin/apache2 -k start 13233 ? D 0:02 \_ /usr/sbin/apache2 -k start 13236 ? D 0:02 \_ /usr/sbin/apache2 -k start 13237 ? D 0:02 \_ /usr/sbin/apache2 -k start 13238 ? D 0:02 \_ /usr/sbin/apache2 -k start 13239 ? D 0:02 \_ /usr/sbin/apache2 -k start 13240 ? D 0:02 \_ /usr/sbin/apache2 -k start 13241 ? D 0:02 \_ /usr/sbin/apache2 -k start 13242 ? D 0:02 \_ /usr/sbin/apache2 -k start 13253 ? D 0:02 \_ /usr/sbin/apache2 -k start 13256 ? D 0:02 \_ /usr/sbin/apache2 -k start 13257 ? D 0:02 \_ /usr/sbin/apache2 -k start 13259 ? D 0:02 \_ /usr/sbin/apache2 -k start 13260 ? S 0:02 \_ /usr/sbin/apache2 -k start 12277 ? S 0:00 /bin/sh /usr/bin/mysqld_safe 12388 ? Sl 7:23 \_ /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/run/mysqld/mysqld.sock --port=3306 12389 ? S 0:00 \_ logger -t mysqld -p daemon.error 12742 ? Sl 0:30 /usr/bin/tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc --hush 12766 ? S 0:00 /bin/sh /home/etherpad-lite/pad/etherpad-lite/bin/safeRun.sh /var/log/etherpad-lite/etherpad-lite.log 12768 ? S 0:00 \_ /bin/sh bin/run.sh 12791 ? Dl 0:15 \_ node node_modules/ep_etherpad-lite Output of this script (I called it „whatswaps.sh“): http://northernmost.org/blog/find-out-what-is-using-your-swap/index.html PID= 1 - Swap used: 108 - (init ) PID= 2 - Swap used: 0 - (kthreadd ) PID= 3 - Swap used: 0 - (migration/0 ) PID= 4 - Swap used: 0 - (ksoftirqd/0 ) PID= 5 - Swap used: 0 - (watchdog/0 ) PID= 20 - Swap used: 0 - (kondemand/0 ) PID= 23 - Swap used: 0 - (khungtaskd ) PID= 24 - Swap used: 0 - (kswapd0 ) PID= 25 - Swap used: 0 - (ksmd ) PID= 26 - Swap used: 0 - (aio/0 ) PID= 27 - Swap used: 0 - (crypto/0 ) PID= 208 - Swap used: 0 - (ata/0 ) PID= 762 - Swap used: 0 - (kjournald ) PID= 763 - Swap used: 0 - (kjournald ) PID= 764 - Swap used: 0 - (kjournald ) PID= 765 - Swap used: 0 - (kjournald ) PID= 766 - Swap used: 0 - (kjournald ) PID=12791 - Swap used: 14880 - (node ) PID=13140 - Swap used: 28596 - (apache2 ) PID=13148 - Swap used: 21208 - (apache2 ) PID=13160 - Swap used: 24972 - (apache2 ) PID=13172 - Swap used: 26560 - (apache2 ) PID=13179 - Swap used: 23452 - (apache2 ) PID=13180 - Swap used: 21548 - (apache2 ) PID=13181 - Swap used: 21868 - (apache2 ) PID=13182 - Swap used: 20056 - (apache2 ) PID=13183 - Swap used: 21132 - (apache2 ) PID=13184 - Swap used: 24016 - (apache2 ) PID=13185 - Swap used: 22820 - (apache2 ) PID=13186 - Swap used: 26772 - (apache2 ) PID=13187 - Swap used: 26260 - (apache2 ) PID=13188 - Swap used: 35144 - (apache2 ) PID=13189 - Swap used: 23172 - (apache2 ) PID=13190 - Swap used: 22472 - (apache2 ) PID=13191 - Swap used: 27792 - (apache2 ) PID=13192 - Swap used: 20328 - (apache2 ) PID=13200 - Swap used: 22056 - (apache2 ) PID=13201 - Swap used: 23860 - (apache2 ) PID=13202 - Swap used: 19908 - (apache2 ) PID=13203 - Swap used: 23300 - (apache2 ) PID=13204 - Swap used: 23496 - (apache2 ) PID=13205 - Swap used: 27804 - (apache2 ) PID=13206 - Swap used: 21108 - (apache2 ) PID=13207 - Swap used: 35468 - (apache2 ) PID=13208 - Swap used: 25552 - (apache2 ) PID=13209 - Swap used: 25700 - (apache2 ) PID=15329 - Swap used: 0 - ( ) PID=15331 - Swap used: 0 - ( ) PID=20704 - Swap used: 604 - (icecast2 ) PID=20718 - Swap used: 308 - (dovecot ) PID=20721 - Swap used: 736 - (dovecot-auth ) PID=21927 - Swap used: 20624 - (eggdrop ) Overall swap used: 1342904 From apache2.conf: <IfModule mpm_prefork_module> StartServers 2 MinSpareServers 3 MaxSpareServers 5 MaxClients 50 MaxRequestsPerChild 500 </IfModule> As it seems it will happen again I ask for advice which data to obtain before I stop the apache2 process. Load of 58 is heavy, but although it sometimes needs up to half an hour, I can still get in when it happens. There is one connection I see: It didn't happen until the last kernel update in Debian Squeeze. That machine is running since 10/2010 and never saw that problem before. -- http://www.atari-frosch.de/ | http://blog.atari-frosch.de/ PGP encryption welcome! Key-ID: 0xCC0AEF3E @ usual places. Und außerdem bin ich der Meinung, daß Deutschland kein Rechtsstaat ist. Manchmal möchte ich 'ne Bombe sein und einfach explodier'n ... -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
