making socket.getaddrinfo use cached dns

[high bandwidth]

I use cached dns lookups with pdnsd on my ubuntu machine to speed up web
access as regular lookups can take 15-30 seconds. However, python's
mechanize and urllib etc use socket.getaddrinfo, which seems not to be using
dns cacheing or taking a long time because of ipv6 lookups. In either case,
I subsequent access to the same site to be fast and not require lengthy
calls to getaddrinfo. How can I get python to correctly use cached dns
lookups and ipv4 only (at least in those cases where it is appropriate).

As mentioned here:
http://stackoverflow.com/questions/2014534/force-python-mechanize-urllib2-to-only-use-a-requests/2035686#2035686
It seems that many python libraries are hardcoded to look for both ipv6 and
ipv4. Since ipv6 lookups are extremely slow at least for some users, perhaps
the devs should carry over these optional arguments to higher level
libraries like mechanize, urllib, httplib etc.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: making socket.getaddrinfo use cached dns

[high bandwidth]

my /etc/resolv.conf says:

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by
> resolvconf(8)
> # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
> nameserver 127.0.0.1
> search Dynex
>

But getaddrinfo still takes a lot of time for repeated queries. After
installing BIND9 and starting it, getaddrinfo is almost instantaneous for
repeated queries.

Thanks!


*Chris Angelico* rosuav at gmail.com

wrote:

On Fri, Jul 8, 2011 at 4:18 AM, high bandwidth http://mail.python.org/mailman/listinfo/python-list>>
wrote:
>* I use cached dns lookups with pdnsd on my ubuntu machine to speed up web*>* 
>access as regular lookups can take 15-30 seconds. However, python's*>* 
>mechanize and urllib etc use socket.getaddrinfo, which seems not to be 
>using*>* dns cacheing or taking a long time because of ipv6 lookups. In either 
>case,*>* I subsequent access to the same site to be fast and not require 
>lengthy*>* calls to getaddrinfo. How can I get python to correctly use cached 
>dns*>* lookups and ipv4 only (at least in those cases where it is 
>appropriate).*
One solution would be to do your own DNS lookups and then pass urllib
an IP address. Is pdnsd set up to be your computer's primary resolver?
(Is /etc/resolv.conf pointing to localhost?) If not, that might help.
I've generally done my DNS caching using BIND, so I can't help with
pdnsd specifically.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list