On Wed, Jul 14, 2021 at 04:00:53PM -0400, [email protected] wrote: > On 2021-07-14 14:51, Michael Niedermayer wrote: > > Signed-off-by: Michael Niedermayer <[email protected]> > > --- > > htdocs/robots.txt | 13 ++++++++++++- > > 1 file changed, 12 insertions(+), 1 deletion(-) > > > > diff --git a/htdocs/robots.txt b/htdocs/robots.txt > > index eb05362..4bbc395 100644 > > --- a/htdocs/robots.txt > > +++ b/htdocs/robots.txt > > @@ -1,2 +1,13 @@ > > User-agent: * > > -Disallow: > > +Crawl-delay: 10 > > +Disallow: /gitweb/ > > +Disallow: /*a=search* > > +Disallow: /*/search/* > > +Disallow: /*a=blobdiff* > > +Disallow: /*/blobdiff/* > > +Disallow: /*a=commitdiff* > > +Disallow: /*/commitdiff/* > > +Disallow: /*a=snapshot* > > +Disallow: /*/snapshot/* > > +Disallow: /*a=blame* > > +Disallow: /*/blame/* > LGTM based on my own personal experiences. But the robots.txt has to be
will apply > applied for git.ffmpeg.org as well, and not just ffmpeg.org. Or else they > will just do the same for git.ffmpeg since there are treated separately. was expecting this a bit ... i will look into that tomorrow or so unless someone else does before me thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Into a blind darkness they enter who follow after the Ignorance, they as if into a greater darkness enter who devote themselves to the Knowledge alone. -- Isha Upanishad
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list [email protected] https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email [email protected] with subject "unsubscribe".
