Some spiders are hitting bugzilla hard generating dependency trees or graphs, downloading large attachements or requesting all bugs in xml format. Disallow all that. --- htdocs/robots.txt | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/htdocs/robots.txt b/htdocs/robots.txt index b9fc830d..057c5899 100644 --- a/htdocs/robots.txt +++ b/htdocs/robots.txt @@ -10,4 +10,8 @@ Disallow: /cgit/ Disallow: /svn Disallow: /cgi-bin/ Disallow: /bugzilla/buglist.cgi +Disallow: /bugzilla/show_bug.cgi*ctype=xml* +Disallow: /bugzilla/attachment.cgi +Disallow: /bugzilla/showdependencygraph.cgi +Disallow: /bugzilla/showdependencytree.cgi Crawl-Delay: 60 -- 2.39.3
