I'm not sure if this is related, but I found a similar problem. We've also extended the memcached class with wrappers. We're getting segfaults after the get call in some cases, double-frees and aborts in other cases. However, the problem does not appear to be triggered by the get call itself. Additionally, it occurs outside of the web environment, suggesting Apache may not be involved at all.
The bug seems to come up, for us, when we call getServerByKey($key) and then subsequently call get. What I assume is a related bug is easily triggered by calling getServerByKey after calling set or get. Josip, are you calling getServerByKey anywhere? I think the segfault has more to do with some invalid memory access than open file handles. I bet that reducing the number of virtual hosts just reduced the possible memory access issues somehow, or got the memory usage down low enough to avoid the bug. This is a long email. I've split it up in to sections, separating a proposed fix from some code that triggers a segfault and code that triggers an abort. ======== getServerByKey seems to trigger a "quit" (see below). At the end of getServerByKey, in php_memcached.c, we have: 1540: memcached_server_free(server); And the first thing that function does is sends "quit" to the memcache server. This has been fixed upstream btw, with a note: https://github.com/php-memcached-dev/php-memcached/blob/master/php_memcached.c " /* memcached_server_add(3) states that the server instance is cloned. */ /* In actuality it is not, possibly a bug in libmemcached 0.40. */ /* remove server freeing */ /* memcached_server_free(server); */ " I'm guessing that the client is not automatically reconnecting ('cause it doesn't know to) and then we end up with a segfault or abort. Would it be possible for this to be used as a patch in an update for squeeze's version of php-memcached-1.0.2? ======== This code triggers the segfault: <?php class Foo extends memcached { function set($key,$value,$exp=0) { $x = parent::set($key,$value,$exp); $this->recordStat('set',$key); return $x; } function get($key) { $x = parent::get($key); $this->recordStat('get',$key); return $x; } function recordStat($cmd,$key) { $server = $this->getServerByKey($key); error_log("$cmd $key " . serialize($server)); } } $foo = new Foo(); $foo->addServer('127.0.0.1', '8000', 1); $result = $foo->set('anykey', 2); print "Set\n"; $result = $foo->get('anykey'); print "Get\n"; ?> Output: " davidk@tetra:~$ php test-mc2.php Set Segmentation fault " strace: " write(3, "set anykey 1 0 1\r\n2\r\n", 21) = 21 read(3, 0x10f4dd0, 8196) = -1 EAGAIN (Resource temporarily unavailable) poll([{fd=3, events=POLLIN}], 1, -1) = 1 ([{fd=3, revents=POLLIN}]) read(3, "STORED\r\n", 8196) = 8 write(3, "quit\r\n", 6) = 6 shutdown(3, 2 /* send and receive */) = 0 close(3) = 0 open("/usr/local/www/server/logs/php.error_log", O_WRONLY|O_CREAT|O_APPEND, 0644) = 3 open("/usr/share/zoneinfo/", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 4 getdents(4, /* 71 entries */, 32768) = 2064 getdents(4, /* 0 entries */, 32768) = 0 ... zoneinfo nonsense ... write(3, "[28-Feb-2012 17:13:35] set anyke"..., 103) = 103 close(3) = 0 write(1, "Set\n", 4Set ) = 4 socket(PF_NETLINK, SOCK_RAW, 0) = 3 bind(3, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0 getsockname(3, {sa_family=AF_NETLINK, pid=17027, groups=00000000}, [12]) = 0 sendto(3, "\24\0\0\0\26\0\1\3\277{MO\0\0\0\0\0\0\0\0", 20, 0, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 20 recvmsg(3, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"0\0\0\0\24\0\2\0\277{MO\203B\0\0\2\10\200\376\1\0\0\0\10\0\1\0\177\0\0\1"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 108 recvmsg(3, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"@\0\0\0\24\0\2\0\277{MO\203B\0\0\n\200\200\376\1\0\0\0\24\0\1\0\0\0\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 128 recvmsg(3, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000}, msg_iov(1)=[{"\24\0\0\0\3\0\2\0\277{MO\203B\0\0\0\0\0\0\1\0\0\0\24\0\1\0\0\0\0\0"..., 4096}], msg_controllen=0, msg_flags=0}, 0) = 20 close(3) = 0 --- SIGSEGV (Segmentation fault) @ 0 (0) --- +++ killed by SIGSEGV +++ Segmentation fault " It's interesting that it closes after the set, eh? FWIW, it only does that if you call getServerByKey. ======== This code duplicates the sigabort: <?php class Foo extends memcached { function get($key) { $x = parent::get($key); return $x; } } $foo = new Foo(); $foo->addServer('127.0.0.1', '8000', 1); $result = $foo->get('anykey'); # This doesn't have to match the key in get but it does # have to happen after the get. $server = $foo->getServerByKey('anykey'); ?> Truncated output from this: " *** glibc detected *** php: double free or corruption (fasttop): 0x00000000010f13d0 *** ======= Backtrace: ========= /lib/libc.so.6(+0x71ad6)[0x7ffedfb73ad6] /lib/libc.so.6(cfree+0x6c)[0x7ffedfb7884c] /lib/libc.so.6(freeaddrinfo+0x28)[0x7ffedfbac558] /usr/lib/libmemcached.so.5(memcached_server_list_free+0x3d)[0x7ffed3aec12d] /usr/lib/libmemcached.so.5(memcached_free+0x19)[0x7ffed3aea759] /usr/lib/php5/20090626/memcached.so(+0x6252)[0x7ffed3cfb252] php(zend_objects_store_del_ref_by_handle_ex+0x28c)[0x6a844c] php(zend_objects_store_del_ref+0x13)[0x6a8473] php(_zval_ptr_dtor+0x3d)[0x67676d] php[0x68f8a2] php(zend_hash_reverse_apply+0x59)[0x68f9a9] php[0x676e05] php[0x6835f4] php(php_request_shutdown+0x325)[0x62ef15] php[0x7128c4] /lib/libc.so.6(__libc_start_main+0xfd)[0x7ffedfb20c4d] php[0x42d4b9] " strace shows (truncated) suggests that something is failing during some cleanup phase: " write(3, "get anykey \r\n", 13) = 13 read(3, "END\r\n", 8196) = 5 write(3, "quit\r\n", 6) = 6 shutdown(3, 2 /* send and receive */) = 0 close(3) = 0 open("/dev/tty", O_RDWR|O_NOCTTY|O_NONBLOCK) = 3 writev(3, [{"*** glibc detected *** ", 23}, {"php", 3}, {": ", 2}, {"double free or corruption (fastt"..., 35}, {": 0x", 4}, {"00000000017463d0", 16}, {" ***\n", 5}], 7*** glibc detected *** php: double free or corruption (fasttop): 0x00000000017463d0 *** ) = 88 " -- David 'dpk' Kirchner -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
