Did you consider robots.txt? Well behaved spiders should respect that, although it does not prevent anything.
I would expect google and yahoo to respect it, I have no clue about he.net. On 08/11/10 2:02, Michelle Konzack wrote: > Hello experts and *, > > I have (since several years) collected some domain names which do not > exist (since years) and registered it in the last 4 month for the > internal use of my Internet Service. > > Now I see Googlebot, Yahoo and <he.net> quering my DNS Servers for > exactly those domains. > > If I read the conditions of Networksolutions and Co, spidering of WHOIS > records is prohibited also the commercial use of the data. > > Does someone have an experience with his crap? > > Unfortunately I can not deny access to the 180 servers and Google, Yahoo > and He is bombing my network with to much useless requests. I have > written a mail to Google not to attack my network of VOIP and IPTV > servers, but they continue... > > The webservers have only an SHTTP administrativ VHost, but not <exp.com> > or <www.exp.com> but the webserver get any requests from <*.exp.com> > because it is an administrative VServer and the error logfile is per day > VERY long. > > An htaccess does not work, because I have more then 800 VHosts on each > server. > > Thanks, Greetings and nice Day/Evening > Michelle Konzack > > > > _______________________________________________ > bind-users mailing list > bind-users@lists.isc.org > https://lists.isc.org/mailman/listinfo/bind-users -- Best regards Sten Carlsen No improvements come from shouting: "MALE BOVINE MANURE!!!"
_______________________________________________ bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
