Nginx Kodi User Agent secure_link blocking / banning

2016-11-01 Thread c0nw0nk
So with Nginx my access.logs show allot of Kodi user agents from what I look up online Kodi is a app that runs on Phones, TV sticks, Mac, PC etc and it is used for watching live TV I reckon its a pretty abusive app or service since there is allot going around about IPTV and how illegal it is. The

Re: setting up client ip and hostname in nginx?

2016-11-01 Thread dec w
you can get client ip, but you can't get client hostname. $host just your server hostname. 2016-11-01 9:52 GMT+08:00 ngineo : > i am working on AWS Elastic Beanstalk Instance, which runs Java applicaiton > servered through Nginx ( no load balancer in front, just a standalone > instance ) > I need

Re: Blocking tens of thousands of IP's

2016-11-01 Thread Tom Thorogood
Hi Eric, This is a rather shameless plug here, but I wrote an nginx module designed to efficiently block (or filter) large numbers of IP addresses. It's a two part system with the nginx module being https://github.com/tmthrgd/nginx-ip-blocker and a separate agent daemon here https://github.com/tmt

Re: Blocking tens of thousands of IP's

2016-11-01 Thread lists
If you get hammered, even serving the 403-page is actually noticeable traffic. - Nginx rate limiting works very well. ‎ ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx

RE: Blocking tens of thousands of IP's

2016-11-01 Thread Cox, Eric S
Is it. However our frontend capacity spans across multiple data centers, a dozen+ nginx instances, and over 70 cores of processing power. We are not as concerned with overloading the frontend as we are with certain endpoints that might be single instance legacy apps etc. -Original Message--

Re: Blocking tens of thousands of IP's

2016-11-01 Thread Rainer Duffner
> Am 01.11.2016 um 23:43 schrieb Cox, Eric S : > > Unfortunately much like others have stated, we also don't have the automation > at the firewall layer to move as quickly as we would like. So at the moment > its not an option. If you get hammered, even serving the 403-page is actually notic

Re: location regex

2016-11-01 Thread olat
Looks like there is a bug in the forum. 2 the same topics and the response ended up in the wrong thread, mixed up ;-) Anyway, Thanks Igor for a quick response. Could you explain more why regex is not a good idea? I am asking about regex in the context of caching some of the requests on front-end p

Re: Blocking tens of thousands of IP's

2016-11-01 Thread lists
‎   Original Message   From: Cox, Eric S Sent: Tuesday, November 1, 2016 3:35 PM To: nginx@nginx.org Reply To: nginx@nginx.org Subject: RE: Blocking tens of thousands of IP's Currently we track all access logs realtime via an in house built log aggregation solution. Various algorithms are setup

RE: Blocking tens of thousands of IP's

2016-11-01 Thread Cox, Eric S
Unfortunately much like others have stated, we also don't have the automation at the firewall layer to move as quickly as we would like. So at the moment its not an option. -Original Message- From: Rainer Duffner [rai...@ultra-secure.de] Received: Tuesday, 01 Nov 2016, 6:41PM To: nginx@n

Re: Blocking tens of thousands of IP's

2016-11-01 Thread Rainer Duffner
> Am 01.11.2016 um 23:35 schrieb Cox, Eric S : > > Currently we track all access logs realtime via an in house built log > aggregation solution. Various algorithms are setup to detect said IPS whether > it be by hit rate, country, known types of attacks etc. These IPS are > typically identifie

RE: Blocking tens of thousands of IP's

2016-11-01 Thread Cox, Eric S
Currently we track all access logs realtime via an in house built log aggregation solution. Various algorithms are setup to detect said IPS whether it be by hit rate, country, known types of attacks etc. These IPS are typically identified within a few mins and we reload to banned list every 60 s

Re: Blocking tens of thousands of IP's

2016-11-01 Thread lists
‎   Original Message   From: Cox, Eric S Sent: Tuesday, November 1, 2016 8:16 AM To: nginx@nginx.org Reply To: nginx@nginx.org Subject: Blocking tens of thousands of IP's Is anyone aware of a difference performance wise between using   return 403;   vs   deny all;   When mapping against a list of

Re: Blocking tens of thousands of IP's

2016-11-01 Thread Lucas Rolff
You could very well do a small ipset together with iptables, it's fast, and you don't have to reload for every subnet / ip you add. Doing it within nginx is rather.. Yeah. -- Best Regards, Lucas Rolff Cox, Eric S wrote: Random, blocks, certain durations, etc. Its very random and or short live

Re: Blocking tens of thousands of IP's

2016-11-01 Thread Rainer Duffner
> Am 01.11.2016 um 22:46 schrieb Jeff Dyke : > > what is your firewall?, that is the place to block subnets etc, i assume they > are not random ips, they are likely from a block owned by someone?? Depends on the firewall, but our network-guys would refuse to do that (and have so in the past)

Re: nginx and FIX server

2016-11-01 Thread Alex Samad
Hi So you have done a setup ? Alex On 2 November 2016 at 08:42, CJ Ess wrote: > Probably a better solution then most! > > On Fri, Oct 28, 2016 at 10:42 PM, Alex Samad wrote: >> >> Hi >> >> Not really an option in current setup. The rate limit is to stop >> clients with bad fix servers that sp

RE: Blocking tens of thousands of IP's

2016-11-01 Thread Cox, Eric S
Random, blocks, certain durations, etc. Its very random and or short lived which is something we don't want to move to the firewall at the moment -Original Message- From: Jeff Dyke [jeff.d...@gmail.com] Received: Tuesday, 01 Nov 2016, 5:46PM To: nginx@nginx.org [nginx@nginx.org] Subject:

Re: Blocking tens of thousands of IP's

2016-11-01 Thread Jeff Dyke
what is your firewall?, that is the place to block subnets etc, i assume they are not random ips, they are likely from a block owned by someone?? On Tue, Nov 1, 2016 at 5:37 PM, CJ Ess wrote: > I don't think managing large lists of IPs is nginx's strength - as far as > I can tell all of its ACLs

Re: nginx and FIX server

2016-11-01 Thread CJ Ess
Probably a better solution then most! On Fri, Oct 28, 2016 at 10:42 PM, Alex Samad wrote: > Hi > > Not really an option in current setup. The rate limit is to stop > clients with bad fix servers that spam our fix server. > > Right now we have a custom bit of java code that that bit rate limits

Re: Blocking tens of thousands of IP's

2016-11-01 Thread CJ Ess
I don't think managing large lists of IPs is nginx's strength - as far as I can tell all of its ACLs are arrays that have the be iterated through on each request. When I do have to manage IP lists in Nginx I try to compress the lists into the most compact CIDR representation so there is less to se

Blocking tens of thousands of IP's

2016-11-01 Thread Cox, Eric S
Is anyone aware of a difference performance wise between using return 403; vs deny all; When mapping against a list of tens of thousands of ip? Thanks This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may co

Re: location regex

2016-11-01 Thread Igor Sysoev
On 01 Nov 2016, at 16:44, olat wrote: > Hello, > > I would like to ask about regex. Why these 2 doesn't behave the same? > > location ~ /(apple/|pear/(small|big)/|test(ing|er)/(fruit|vegis)_)* > > location ~ /apple/*|/pear/(small|big)/*|/test(ing|er)/(fruit|vegis)_* > > > could you point me

location regex

2016-11-01 Thread olat
Hello, I would like to ask about regex. Why these 2 doesn't behave the same? location ~ /(apple/|pear/(small|big)/|test(ing|er)/(fruit|vegis)_)* location ~ /apple/*|/pear/(small|big)/*|/test(ing|er)/(fruit|vegis)_* could you point me to good practice? Ola Posted at Nginx Forum: https://for