[htdig3-dev] Re: robots.txt bug (was [ANNOUNCE] ht://Dig 3.2.0b1)

[Geoff Hutchison]

At 7:40 PM +0200 2/6/00, Valdas Andrulis wrote:
>I doesnt work with 3.2.0b1, and do work with 3.1.4, with that redundant
>robots.txt configurations, or just
>User-agent: *
>Disallow: /cat/
>
>or
>
>User-agent: htdig
>Disallow: /cat/

Hmph. This one's harder than your other bug report. The pattern is 
set correctly and the code looks correct. The new version switched to 
using regex internally so I bet this is a bug somewhere in the logic.

First off, have you set case_sensitive to anything in your config file?

Secondly, could you try setting another (bogus) pattern in your robots.txt?

Add:
User-agent: htdig
Disallow: /foobar/

Then let us know what pattern it sets in the debug output--I don't 
really want the whole thing but I want to see if it's setting the 
pattern OK.

I'll try something like this on my machine to see if I can reproduce it.

-Geoff


------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
[EMAIL PROTECTED] 
You will receive a message to confirm this.