On Saturday 09 November 2019 10:37:09 john doe wrote: > On 11/9/2019 2:43 PM, Gene Heskett wrote: > > On Saturday 09 November 2019 03:36:49 john doe wrote: > >> On 11/9/2019 8:30 AM, Gene Heskett wrote: > >>> I have a list of ipv4's I want fail2ban to block. But amongst the > >>> numerous subdirs for fail2ban, I cannot find one that looks > >>> suitable to put this list of addresses in so the are blocked > >>> forever. Can someone more familiar with how fail2ban works give > >>> me a hand? These are the ipv4 addresses of bingbot, semrush, > >>> yandex etc etc that are DDOSing me by repeatedly downloading my > >>> whole site and using up 100% of my upload bandwidth. > >>> > >>> Thanks all. > >>> > >>> Cheers, Gene Heskett > >> > >> Rather then to use fail2ban for this, I would create un ipset that > >> fail2ban can populate then use that ipset in iptables. > >> > >> One advantage of this is that you can add/delete ip from the ipset > >> without having to restart fail2ban/iptables. > > > > I've done that with the help of a previous responder and now have > > 99% of the pigs that ignore my robots.txt blocked. semrush is > > extremely determined and has switched to a 4th address I've not seen > > before, but is no longer DDOSing my site. > > Then, I don't understand your question, if you have fail2ban > populating an ipset and that ipset is used in iptables. > You can simply add those set of IPs to the ipset manually.
Fail2ban might be running and I likely should stop it, but ATM I'm manually adding rules to iptables. And I am about down to seeing only the fetchmail scans that actually find something to download. Tracking actual net traffic with gkrellm. > Note that using IPs directly is an red herring; you need to use other > means (UserAgent ...) to identify those bots. I'll repeat that semrush has at least 6 variations of their User-agent names, maybe more. Easier to use the ip's with a broad /24 brush. They can name it anything they want, but the ip isn't phony. Hit them with a /24 and you've got everything I've seen so far except bytespider. They cover 2 /24 blocks. > By the sound of it, you cleerly need to learn the httpd server you are > using, then if it is not enough, add fail2ban and iptables into the > mix. Agreed, but the man pages for both apache2 and fail2ban are a poor tut. iptables is better. Adding these on the fly: root@coyote:action.d$ iptables -L -nv --line-numbers Chain INPUT (policy ACCEPT 103 packets, 12830 bytes) num pkts bytes target prot opt in out source destination 1 0 0 DROP all -- * * 73.229.203.175 0.0.0.0/0 2 0 0 DROP all -- * * 77.88.5.200 0.0.0.0/0 3 0 0 DROP all -- * * 66.249.64.226 0.0.0.0/0 4 0 0 DROP all -- * * 40.77.167.82 0.0.0.0/0 5 0 0 DROP all -- * * 111.225.149.199 0.0.0.0/0 6 0 0 DROP all -- * * 40.77.167.142 0.0.0.0/0 7 4 240 DROP all -- * * 220.243.136.25 0.0.0.0/0 8 416 24960 DROP all -- * * 46.229.168.146 0.0.0.0/0 9 3 180 DROP all -- * * 141.8.143.160 0.0.0.0/0 10 0 0 DROP all -- * * 111.225.148.159 0.0.0.0/0 11 48 2880 DROP all -- * * 46.229.168.134 0.0.0.0/0 12 0 0 DROP all -- * * 46.229.168.137 0.0.0.0/0 13 0 0 DROP all -- * * 111.225.148.49 0.0.0.0/0 14 0 0 DROP all -- * * 220.243.136.54 0.0.0.0/0 15 0 0 DROP all -- * * 110.249.202.57 0.0.0.0/0 16 68 4080 DROP all -- * * 111.225.149.0/24 0.0.0.0/0 17 50 3000 DROP all -- * * 110.249.201.0/24 0.0.0.0/0 18 35 2100 DROP all -- * * 110.249.202.0/24 0.0.0.0/0 19 8 480 DROP all -- * * 111.225.148.0/24 0.0.0.0/0 20 8 480 DROP all -- * * 46.229.168.0/24 0.0.0.0/0 obviously a bit dirty, but its stopping the DDOS. Which is what I came here to do. > -- > John Doe Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) If we desire respect for the law, we must first make the law respectable. - Louis D. Brandeis Genes Web page <http://geneslinuxbox.net:6309/gene>
