Hi, > It has never been necessary to modify the source code to ignore > robots.txt files; rather, you can specify "robots=off" in your .wgetrc; > or "-e robots=off" on the commandline. More information about this is in > the "Robot Exclusion" section of the Texinfo manual; use "info wget" to > access that manual (unfortunately, this information does not appear in > the man page, which was never intended to be complete).
Then the manual of wget really sucks, searching for robots returns three matches and none these is related to this option. Cheers Nico -- Nico Golde - http://www.ngolde.de - [EMAIL PROTECTED] - GPG: 0x73647CFF For security reasons, all text in this mail is double-rot13 encrypted.
pgp1hFlggx992.pgp
Description: PGP signature
