They need to tune their crawlers or they're going to end up permablocked in a lot of places.
Mark Sent from my mobile On Dec 13, 2016, at 8:28 AM, Grant <[email protected]> wrote: >> Just curious, was the "fairly legit search engine" semrush.com? >> >> I've seen a few sites get overwhelmed by them recently. > > > Yes indeed it was. > > >>> Well I certainly use it to defend from that kind of attack all the time. >>> Can you give us some idea of the rate (ie: how many requests per >>> second)? Also, for that kind of attack it's important to be using the >>> recidive filter. By any chance is it a wordpress site? > > > So you're saying fail2ban should have caught it so they must have been > making requests at a rate lower than my configured maximum? How does > fail2ban know to lump together stats for requests coming from > different IP addresses? > > - Grant > > >>>> I recently suffered DoS from a series of 10 sequential IP addresses >>>> which identified themselves as being associated with a fairly legit >>>> search engine. fail2ban would have dealt with the problem if a single >>>> IP address had been used. Can it be made to work in a situation like >>>> this where a series of sequential IP addresses are in play? > ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Fail2ban-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fail2ban-users
