Re: Don't let search bots look at buglist.cgi

2011-05-17 Thread Michael Matz
Hi, On Mon, 16 May 2011, Ian Lance Taylor wrote: > >>> httpd being in the top-10 always, fiddling with bugzilla URLs? > >>> (Note, I don't have access to gcc.gnu.org, I'm relaying info from > >>> multiple instances of discussion on #gcc and richi poking on it; > >>> that said, it still might n

Re: Don't let search bots look at buglist.cgi

2011-05-17 Thread Axel Freyn
On Mon, May 16, 2011 at 10:27:44PM -0700, Ian Lance Taylor wrote: > On Mon, May 16, 2011 at 6:42 AM, Richard Guenther > wrote: > >>> > >>> httpd being in the top-10 always, fiddling with bugzilla URLs? > >>> (Note, I don't have access to gcc.gnu.org, I'm relaying info from multiple > >>> instances

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Ian Lance Taylor
On Mon, May 16, 2011 at 6:42 AM, Richard Guenther wrote: >>> >>> httpd being in the top-10 always, fiddling with bugzilla URLs? >>> (Note, I don't have access to gcc.gnu.org, I'm relaying info from multiple >>> instances of discussion on #gcc and richi poking on it; that said, it >>> still might n

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Joseph S. Myers
On Mon, 16 May 2011, Ian Lance Taylor wrote: > Richard Guenther writes: > > > On Fri, May 13, 2011 at 7:14 PM, Ian Lance Taylor wrote: > >> I noticed that buglist.cgi was taking quite a bit of CPU time.  I looked > >> at some of the long running instances, and they were coming from > >> searchb

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Ian Lance Taylor
Richard Guenther writes: > On Fri, May 13, 2011 at 7:14 PM, Ian Lance Taylor wrote: >> I noticed that buglist.cgi was taking quite a bit of CPU time.  I looked >> at some of the long running instances, and they were coming from >> searchbots.  I can't think of a good reason for this, so I have >

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Andrew Haley
On 05/16/2011 02:42 PM, Richard Guenther wrote: > On Mon, May 16, 2011 at 3:34 PM, Andrew Haley wrote: >> On 05/16/2011 02:32 PM, Michael Matz wrote: >>> >>> On Mon, 16 May 2011, Andrew Haley wrote: >>> > It routinely is. bugzilla performance is terrible most of the time > for me (up to t

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Richard Guenther
On Mon, May 16, 2011 at 3:34 PM, Andrew Haley wrote: > On 05/16/2011 02:32 PM, Michael Matz wrote: >> >> On Mon, 16 May 2011, Andrew Haley wrote: >> It routinely is.  bugzilla performance is terrible most of the time for me (up to the point of five timeouts in sequence), svn speed is >>>

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Andrew Haley
On 05/16/2011 02:32 PM, Michael Matz wrote: > > On Mon, 16 May 2011, Andrew Haley wrote: > >>> It routinely is. bugzilla performance is terrible most of the time >>> for me (up to the point of five timeouts in sequence), svn speed is >>> mediocre at best, and people with access to gcc.gnu.org

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Michael Matz
Hi, On Mon, 16 May 2011, Andrew Haley wrote: > > It routinely is. bugzilla performance is terrible most of the time > > for me (up to the point of five timeouts in sequence), svn speed is > > mediocre at best, and people with access to gcc.gnu.org often observe > > loads > 25, mostly due to I

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Andrew Haley
On 05/16/2011 02:22 PM, Michael Matz wrote: > Hi, > > On Mon, 16 May 2011, Andrew Haley wrote: > It's not quite the same information, surely. Wouldn't searchers be directed to an email rather than the bug itself? >>> >>> Yes, though there is a link in all mails. >> >> Right, so we are

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Michael Matz
Hi, On Mon, 16 May 2011, Andrew Haley wrote: > >> It's not quite the same information, surely. Wouldn't searchers be > >> directed to an email rather than the bug itself? > > > > Yes, though there is a link in all mails. > > Right, so we are contemplating a reduction in search quality in exch

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Andrew Haley
On 05/16/2011 02:10 PM, Richard Guenther wrote: > On Mon, May 16, 2011 at 3:04 PM, Andrew Haley wrote: >> On 05/16/2011 01:09 PM, Michael Matz wrote: >>> Hi, >>> >>> On Mon, 16 May 2011, Andrew Haley wrote: >>> On 16/05/11 10:45, Richard Guenther wrote: > On Fri, May 13, 2011 at 7:14 PM,

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Richard Guenther
On Mon, May 16, 2011 at 3:04 PM, Andrew Haley wrote: > On 05/16/2011 01:09 PM, Michael Matz wrote: >> Hi, >> >> On Mon, 16 May 2011, Andrew Haley wrote: >> >>> On 16/05/11 10:45, Richard Guenther wrote: On Fri, May 13, 2011 at 7:14 PM, Ian Lance Taylor wrote: > I noticed that buglist.cgi

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Andreas Schwab
Andrew Haley writes: > It's not quite the same information, surely. Wouldn't searchers be directed > to an email rather than the bug itself? The mail contains the bugzilla link, so they can easily get there if needed. Andreas. -- Andreas Schwab, sch...@redhat.com GPG Key fingerprint = D4E8 D

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Andrew Haley
On 05/16/2011 01:09 PM, Michael Matz wrote: > Hi, > > On Mon, 16 May 2011, Andrew Haley wrote: > >> On 16/05/11 10:45, Richard Guenther wrote: >>> On Fri, May 13, 2011 at 7:14 PM, Ian Lance Taylor wrote: I noticed that buglist.cgi was taking quite a bit of CPU time. I looked at some o

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Michael Matz
Hi, On Mon, 16 May 2011, Andrew Haley wrote: > On 16/05/11 10:45, Richard Guenther wrote: > > On Fri, May 13, 2011 at 7:14 PM, Ian Lance Taylor wrote: > >> I noticed that buglist.cgi was taking quite a bit of CPU time. I looked > >> at some of the long running instances, and they were coming fr

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Andrew Haley
On 16/05/11 10:45, Richard Guenther wrote: > On Fri, May 13, 2011 at 7:14 PM, Ian Lance Taylor wrote: >> I noticed that buglist.cgi was taking quite a bit of CPU time. I looked >> at some of the long running instances, and they were coming from >> searchbots. I can't think of a good reason for t

Re: Don't let search bots look at buglist.cgi

2011-05-16 Thread Richard Guenther
On Fri, May 13, 2011 at 7:14 PM, Ian Lance Taylor wrote: > I noticed that buglist.cgi was taking quite a bit of CPU time.  I looked > at some of the long running instances, and they were coming from > searchbots.  I can't think of a good reason for this, so I have > committed this patch to the gcc

Don't let search bots look at buglist.cgi

2011-05-13 Thread Ian Lance Taylor
I noticed that buglist.cgi was taking quite a bit of CPU time. I looked at some of the long running instances, and they were coming from searchbots. I can't think of a good reason for this, so I have committed this patch to the gcc.gnu.org robots.txt file to not let searchbots search through list