Package: pdns-server Version: 2.9.17-13 Severity: normal
I set up PowerDNS yesterday on our home router machine, which is a dual PPro, running sarge. It's acting as a caching resolver for our network, which currently only has a couple of other machines attached; it's also serving a local zone (at.fivegeeks.net) and its reverse using BindBackend. (The zone files are automatically generated from our old djbdns config, but they're pretty unremarkable.) While it's working, it works very nicely. The problem is that after anywhere between a few minutes and a few hours of operation, all three distributor threads will go into a tight loop inside BindBackend, and the server will stop responding to queries. I've never found it with just one or two of the threads jammed. I have to "killall pdns_server" to get it to stop; "/etc/init.d/pdns stop" has no effect. I haven't yet tried using a different backend; I note that in 2.9.18 the PowerDNS authors have replaced BindBackend entirely, so it's quite possible they've fixed this bug while doing so. I've tarred up my complete /etc/powerdns directory and put it here: http://offog.org/stuff/powerdns-hang-config.tar.gz Here's some more information (including backtraces down the bottom): # grep -v '^\(#.*\|\)$' pdns.conf allow-recursion=127.0.0.1, 192.168.0.0/16 config-dir=/etc/powerdns daemon=yes disable-axfr=yes guardian=yes launch=bind lazy-recursion=yes local-address=192.168.58.1 local-port=53 logfile=/var/log/pdns.log master=yes module-dir=/usr/lib/powerdns recursor=127.0.0.1 setgid=pdns setuid=pdns socket-dir=/var/run webserver=yes webserver-address=192.168.58.1 webserver-port=8081 version-string=powerdns include=/etc/powerdns/pdns.d # cat pdns.d/pdns.local # Here comes the local changes the user made, like configuration of # the several backends that exists. bind-check-interval=5 bind-config=/etc/powerdns/named.conf # dpkg -l | grep pdns ii pdns-recursor 2.9.17-13 PowerDNS recursor ii pdns-server 2.9.17-13 extremely powerful and versatile nameserver # grep 'veloci pdns' /var/log/syslog [...] Jul 18 09:26:52 veloci pdns[3413]: Listening on controlsocket in '/var/run/pdns.controlsocket' Jul 18 09:26:52 veloci pdns[3416]: Guardian is launching an instance Jul 18 09:26:52 veloci pdns[3416]: This is a guarded instance of pdns Jul 18 09:26:52 veloci pdns[3416]: UDP server bound to 192.168.58.1:53 Jul 18 09:26:52 veloci pdns[3416]: TCP server bound to 192.168.58.1:53 Jul 18 09:26:52 veloci pdns[3416]: PowerDNS 2.9.17 (C) 2001-2005 PowerDNS.COM BV (May 31 2005, 12:24:59) starting up Jul 18 09:26:52 veloci pdns[3416]: PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2. Jul 18 09:26:52 veloci pdns[3416]: Set effective group id to 104 Jul 18 09:26:52 veloci pdns[3416]: Set effective user id to 110 Jul 18 09:26:52 veloci pdns[3416]: DNS Proxy launched, local port 33936, remote 127.0.0.1:53 Jul 18 09:26:52 veloci pdns[3420]: Launched webserver on 192.168.58.1:8081 Jul 18 09:26:52 veloci pdns[3422]: Master/slave communicator launching Jul 18 09:26:52 veloci pdns[3416]: Creating backend connection for TCP Jul 18 09:26:52 veloci pdns[3422]: [bindbackend] Parsing 2 domain(s), will report when done Jul 18 09:26:53 veloci pdns[3422]: [bindbackend] Done parsing domains, 0 rejected, 2 new, 0 removed Jul 18 09:26:53 veloci pdns[3416]: About to create 3 backend threads Jul 18 09:26:53 veloci pdns[3422]: All slave domains are fresh Jul 18 09:26:53 veloci pdns[3422]: No master domains need notifications Jul 18 14:25:56 veloci pdns[3422]: 1 domain for which we are master needs notifications Jul 18 14:26:56 veloci pdns[3422]: No master domains need notifications [I think this was where I attached the debugger...] Jul 18 14:28:31 veloci pdns[3413]: In guardian loop, waitpid returned error: No child processes Jul 18 14:28:31 veloci pdns[3413]: Dying Jul 18 14:32:46 veloci pdns[3419]: Fatal error in control listener: Guardian exited - going down as well # top [...] PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3426 pdns 20 0 4380 3720 3176 R 72.7 2.9 2:35.69 pdns_server 3425 pdns 14 0 4380 3720 3176 R 63.0 2.9 2:39.82 pdns_server 3424 pdns 12 0 4380 3720 3176 R 60.7 2.9 2:35.91 pdns_server # gdb /usr/sbin/pdns_server 3425 [...] (gdb) info threads 12 Thread 163851 (LWP 3427) 0x40116008 in recvfrom () from /lib/libpthread.so.0 11 Thread 147466 (LWP 3426) 0x0811fdae in __gnu_cxx::hashtable<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > >, std::string, hash_string, std::_Select1st<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >, compare_string, std::allocator<std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >::find_or_insert () 10 Thread 131081 (LWP 3425) 0x08121077 in __gnu_cxx::hashtable<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > >, std::string, hash_string, std::_Select1st<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >, compare_string, std::allocator<std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >::resize () 9 Thread 114696 (LWP 3424) 0x0811fd99 in __gnu_cxx::hashtable<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > >, std::string, hash_string, std::_Select1st<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >, compare_string, std::allocator<std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >::find_or_insert () 8 Thread 98311 (LWP 3423) 0x4022c081 in select () from /lib/libc.so.6 7 Thread 81926 (LWP 3422) 0x40202db6 in nanosleep () from /lib/libc.so.6 6 Thread 65541 (LWP 3421) 0x40202db6 in nanosleep () from /lib/libc.so.6 5 Thread 49156 (LWP 3420) 0x40115e88 in accept () from /lib/libpthread.so.0 4 Thread 32771 (LWP 3419) 0x40115abb in read () from /lib/libpthread.so.0 3 Thread 16386 (LWP 3418) 0x40115f88 in recv () from /lib/libpthread.so.0 2 Thread 32769 (LWP 3417) 0x40229ada in poll () from /lib/libc.so.6 1 Thread 16384 (LWP 3416) 0x40112604 in __pthread_sigsuspend () from /lib/libpthread.so.0 (gdb) thread apply all bt Thread 12 (Thread 163851 (LWP 3427)): #0 0x40116008 in recvfrom () from /lib/libpthread.so.0 #1 0x080e3650 in UDPNameserver::receive () #2 0x080de163 in qthread () #3 0x4010fe51 in pthread_start_thread () from /lib/libpthread.so.0 #4 0x4023292a in clone () from /lib/libc.so.6 Thread 11 (Thread 147466 (LWP 3426)): #0 0x0811fdae in __gnu_cxx::hashtable<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > >, std::string, hash_string, std::_Select1st<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >, compare_string, std::allocator<std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >::find_or_insert () #1 0x0811dc3d in BindBackend::lookup () #2 0x080b9a49 in UeberBackend::lookup () #3 0x0807ef62 in PacketHandler::doWildcardRecords () #4 0x08085f67 in PacketHandler::question () #5 0x080e2226 in Distributor<DNSPacket, DNSPacket, PacketHandler>::makeThread () #6 0x4010fe51 in pthread_start_thread () from /lib/libpthread.so.0 #7 0x4023292a in clone () from /lib/libc.so.6 Thread 10 (Thread 131081 (LWP 3425)): #0 0x08121077 in __gnu_cxx::hashtable<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > >, std::string, hash_string, std::_Select1st<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >, compare_string, std::allocator<std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >::resize () #1 0x0811fce7 in __gnu_cxx::hashtable<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > >, std::string, hash_string, std::_Select1st<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >, compare_string, std::allocator<std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >::find_or_insert () #2 0x0811dc3d in BindBackend::lookup () #3 0x080b9a49 in UeberBackend::lookup () #4 0x0807ef62 in PacketHandler::doWildcardRecords () #5 0x08085f67 in PacketHandler::question () #6 0x080e2226 in Distributor<DNSPacket, DNSPacket, PacketHandler>::makeThread () #7 0x4010fe51 in pthread_start_thread () from /lib/libpthread.so.0 #8 0x4023292a in clone () from /lib/libc.so.6 Thread 9 (Thread 114696 (LWP 3424)): #0 0x0811fd99 in __gnu_cxx::hashtable<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > >, std::string, hash_string, std::_Select1st<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >, compare_string, std::allocator<std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >::find_or_insert () #1 0x0811dc3d in BindBackend::lookup () #2 0x080b9a49 in UeberBackend::lookup () #3 0x0807ef62 in PacketHandler::doWildcardRecords () #4 0x08085f67 in PacketHandler::question () #5 0x080e2226 in Distributor<DNSPacket, DNSPacket, PacketHandler>::makeThread () #6 0x4010fe51 in pthread_start_thread () from /lib/libpthread.so.0 #7 0x4023292a in clone () from /lib/libc.so.6 Thread 8 (Thread 98311 (LWP 3423)): #0 0x4022c081 in select () from /lib/libc.so.6 #1 0x40118d00 in __JCR_LIST__ () from /lib/libpthread.so.0 #2 0x0814e574 in ?? () #3 0xbedffa34 in ?? () #4 0x00000000 in ?? () #5 0x0809145b in TCPNameserver::thread () #6 0x08088fb1 in TCPNameserver::launcher () #7 0x4010fe51 in pthread_start_thread () from /lib/libpthread.so.0 #8 0x4023292a in clone () from /lib/libc.so.6 Thread 7 (Thread 81926 (LWP 3422)): #0 0x40202db6 in nanosleep () from /lib/libc.so.6 #1 0x00000000 in ?? () #2 0x40202bff in sleep () from /lib/libc.so.6 #3 0x080d00a7 in CommunicatorClass::mainloop () #4 0x080e2bd1 in CommunicatorClass::launchhelper () #5 0x4010fe51 in pthread_start_thread () from /lib/libpthread.so.0 #6 0x4023292a in clone () from /lib/libc.so.6 Thread 6 (Thread 65541 (LWP 3421)): #0 0x40202db6 in nanosleep () from /lib/libc.so.6 #1 0x00000000 in ?? () #2 0x40202bff in sleep () from /lib/libc.so.6 #3 0x080a23cb in StatWebServer::statThread () #4 0x080a2601 in StatWebServer::statThreadHelper () #5 0x4010fe51 in pthread_start_thread () from /lib/libpthread.so.0 #6 0x4023292a in clone () from /lib/libc.so.6 Thread 5 (Thread 49156 (LWP 3420)): #0 0x40115e88 in accept () from /lib/libpthread.so.0 #1 0x080ab7d5 in Server::accept () #2 0x080a83b4 in WebServer::go () #3 0x080a587d in StatWebServer::launch () #4 0x080a2621 in StatWebServer::threadHelper () #5 0x4010fe51 in pthread_start_thread () from /lib/libpthread.so.0 #6 0x4023292a in clone () from /lib/libc.so.6 Thread 4 (Thread 32771 (LWP 3419)): #0 0x40115abb in read () from /lib/libpthread.so.0 #1 0x00000200 in ?? () #2 0x00000000 in ?? () #3 0x080be00e in DynListener::getLine () #4 0x080be940 in DynListener::theListener () #5 0x080bdbd1 in DynListener::theListenerHelper () #6 0x4010fe51 in pthread_start_thread () from /lib/libpthread.so.0 #7 0x4023292a in clone () from /lib/libc.so.6 Thread 3 (Thread 16386 (LWP 3418)): #0 0x40115f88 in recv () from /lib/libpthread.so.0 #1 0x080d5463 in DNSProxy::mainloop () #2 0x080d6901 in DNSProxy::launchhelper () #3 0x4010fe51 in pthread_start_thread () from /lib/libpthread.so.0 #4 0x4023292a in clone () from /lib/libc.so.6 Thread 2 (Thread 32769 (LWP 3417)): #0 0x40229ada in poll () from /lib/libc.so.6 #1 0x4010fb50 in __pthread_manager () from /lib/libpthread.so.0 #2 0x4023292a in clone () from /lib/libc.so.6 Thread 1 (Thread 16384 (LWP 3416)): #0 0x40112604 in __pthread_sigsuspend () from /lib/libpthread.so.0 #1 0x401123c8 in __pthread_wait_for_restart_signal () from /lib/libpthread.so.0 #2 0x4010f73c in pthread_join () from /lib/libpthread.so.0 #3 0x080df600 in mainthread () #4 0x080b4749 in main () #0 0x0811fdae in __gnu_cxx::hashtable<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > >, std::string, hash_string, std::_Select1st<std::pair<std::string const, std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >, compare_string, std::allocator<std::vector<BBResourceRecord, std::allocator<BBResourceRecord> > > >::find_or_insert () -- System Information: Debian Release: 3.1 Architecture: i386 (i686) Kernel: Linux 2.4.30 Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968) Versions of packages pdns-server depends on: ii adduser 3.63 Add and remove users and groups ii debconf [debconf-2.0] 1.4.30.13 Debian configuration management sy ii libc6 2.3.2.ds1-22 GNU C Library: Shared libraries an ii libgcc1 1:3.4.3-13 GCC support library ii libstdc++5 1:3.3.5-13 The GNU Standard C++ Library v3 ii ucf 1.17 Update Configuration File: preserv -- debconf information: * pdns-server/localaddress: 192.168.58.1 * pdns-server/allowrecursion: 127.0.0.1;192.168.0.0/16 * pdns-server/autostart: true -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
