Re: 2 of 16 cores are constantly maxing out - how to balance the load?

2018-01-11 Thread Peter Booth
Perhaps you should use pidstat to validate which processes are running on the two busy cores? > On Jan 11, 2018, at 6:25 AM, Vlad K. wrote: > > On 2018-01-11 11:59, Lucas Rolff wrote: >> Now, in your case with php-fpm in the mix as well, controlling that >> can be hard ( not sure if you can pin

Re: proxy protocol over a plain tcp with ssl

2018-01-11 Thread nir
Well, seems that you need to read the manual with the right perspective... https://stackoverflow.com/questions/48211083/proxy-protocol-and-ssl Posted at Nginx Forum: https://forum.nginx.org/read.php?2,278113,278128#msg-278128 ___ nginx mailing list ngi

Re: proxy protocol over a plain tcp with ssl

2018-01-11 Thread nir
Hi Roman, I'm trying to pass the proxy protocol to my backend through Nginx when the traffic is encrypted This configuration block listen 8012; proxy_pass backend; proxy_protocol on; allows me to pass a non encrypted traffic and the proxy protocol This configuration block: listen 8012 proxy_prot

Re: limit_req is not working in virutal location?

2018-01-11 Thread Maxim Dounin
Hello! On Thu, Jan 11, 2018 at 12:22:56PM -0500, pva wrote: > Hi, Maxim. > > Maxim Dounin Wrote: > > That's because try_files is not a mechanism to "conditionally select > > configurations"[1] like the rewrite module directives (including > > "return"), but rather a way to choose which file wi

Re: limit_req is not working in virutal location?

2018-01-11 Thread pva
Hi, Maxim. Maxim Dounin Wrote: > That's because try_files is not a mechanism to "conditionally select > configurations"[1] like the rewrite module directives (including > "return"), but rather a way to choose which file will be used for > request processing. As such, try_files checks happen ri

Re: proxy protocol over a plain tcp with ssl

2018-01-11 Thread Roman Arutyunyan
Hi, On Thu, Jan 11, 2018 at 08:22:47AM -0500, nir wrote: > I'm trying to configure nginx which is behind an haproxy to pass the proxy > protocol over a plain tcp connection. It works well. > When I add ssl to the equation it fails. Below is the nginx configuration > block I'm using. > Is it a conf

Re: limit_req is not working in virutal location?

2018-01-11 Thread Maxim Dounin
Hello! On Thu, Jan 11, 2018 at 10:47:29AM -0500, pva wrote: > Francis, thank you for you answer. > > > On Thu, Jan 11, 2018 at 03:32:07AM -0500, pva wrote: > > I'm slightly guessing, so apologies if I mislead you and hopefully > > someone else will correct this if necessary... > > > > > Hi. Cou

Re: limit_req is not working in virutal location?

2018-01-11 Thread pva
Francis, thank you for you answer. > On Thu, Jan 11, 2018 at 03:32:07AM -0500, pva wrote: > I'm slightly guessing, so apologies if I mislead you and hopefully > someone else will correct this if necessary... > > > Hi. Could you, please, explain why limit_req in @limitspeed location > > is not wor

RE: [IE] GeoIP Module for Blocking IP in http_x_forwarded_for

2018-01-11 Thread Jason Whittington
If you control Frontend Server A I would suggest not using X-Forwarded-For for this purpose. Can you have the front end server send a distinct header to server B? X-Real-IP would be a good choice of header. Then Server B could key off that header instead of XFF. You might find this page inte

Re: GeoIP Module for Blocking IP in http_x_forwarded_for

2018-01-11 Thread Maxim Dounin
Hello! On Thu, Jan 11, 2018 at 07:17:20AM -0500, anish10dec wrote: > GeoIP module is able to block request on basis of remote address which is IP > of the remote device or user but not on basis of X-Forwarded-For IP if it > has multiple IP address in it. > > There is Frontend Server( Server A) w

proxy protocol over a plain tcp with ssl

2018-01-11 Thread nir
I'm trying to configure nginx which is behind an haproxy to pass the proxy protocol over a plain tcp connection. It works well. When I add ssl to the equation it fails. Below is the nginx configuration block I'm using. Is it a configuration issue or might be that it's not at all possible for nginx

Re: limit_req is not working in virutal location?

2018-01-11 Thread Francis Daly
On Thu, Jan 11, 2018 at 03:32:07AM -0500, pva wrote: Hi there, I'm slightly guessing, so apologies if I mislead you and hopefully someone else will correct this if necessary... > Hi. Could you, please, explain why limit_req in @limitspeed location is not > working in case of redirect to @allowed

Re: Example body filter hangs when modified a little bit - request is not terminating.

2018-01-11 Thread Roman Arutyunyan
Hi, On Wed, Jan 10, 2018 at 08:05:15PM -0500, ptcell wrote: > I tried out this nginx example ngx_http_foo_body_filter body filter (here > http://nginx.org/en/docs/dev/development_guide.html#http_body_buffers_reuse > ) and got that to work just fine. It inserts a "foo" string before each > incomin

GeoIP Module for Blocking IP in http_x_forwarded_for

2018-01-11 Thread anish10dec
GeoIP module is able to block request on basis of remote address which is IP of the remote device or user but not on basis of X-Forwarded-For IP if it has multiple IP address in it. There is Frontend Server( Server A) which receives the request and send it to Intermediate Server (Server B) We have

Re: Nginx error log parser

2018-01-11 Thread mohit Agrawal
I finally end up writing my own error log fluentd custom parser in ruby. It's working now. Thanks for help anyways, much appreciated On 11 January 2018 at 14:26, Aziz Rozyev wrote: > Hi, > > seems, that fluentd has an nginx_parser plugin already, another solution > that probably should work is

Re: 2 of 16 cores are constantly maxing out - how to balance the load?

2018-01-11 Thread Vlad K.
On 2018-01-11 11:59, Lucas Rolff wrote: Now, in your case with php-fpm in the mix as well, controlling that can be hard ( not sure if you can pin php-fpm processes to cores ) – but for nginx and RX/TX queues, it’s for sure possible. Should be doable with cgroups / cpusets? CPUAffinity directi

Re: 2 of 16 cores are constantly maxing out - how to balance the load?

2018-01-11 Thread Lucas Rolff
In high traffic environments it generally make sense to “dedicate” a core to each RX and TX queue you have on the NIC – this way you lower the chances of a single core being overloaded from handling network and thus degrading performance. And then at same time within nginx, map the individual p

Re: 2 of 16 cores are constantly maxing out - how to balance the load?

2018-01-11 Thread Raffael Vogler
Or would it make sense (if possible at all) to assign two or three more cores to networking interrupts? ___ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx

Re: 2 of 16 cores are constantly maxing out - how to balance the load?

2018-01-11 Thread Raffael Vogler
Hey Lucas, your assumption seems to be correct. According to /proc/interrupts the following stats are significantly higher for those two cores (CPU5, CPU6 - 0-based indexing): - CPU5: xen-percpu-ipi callfuncsingle5 - CPU6: xen-percpu-ipi callfuncsingle6 - CPU5: xen-pirq-msi-x eth0-

Re: 2 of 16 cores are constantly maxing out - how to balance the load?

2018-01-11 Thread Lucas Rolff
If it’s the same two cores, it might be another process that uses the same two cores and thus happens to max out. One very likely possibility would be interrupts from e.g. networking. You can check /proc/interrupts to see where interrupts from the network happens. From: nginx on behalf of Raffa

2 of 16 cores are constantly maxing out - how to balance the load?

2018-01-11 Thread Raffael Vogler
Hello! I have nginx with php-fpm running on a 16 core Ubuntu 16.04 instance. The server is handling more than 10 million requests per hour. https://imgur.com/a/iRZ7V As you can see on the htop screenshot cores 6 and 7 are maxed out and that's the case constantly - even after restarting nginx tho

Re: Nginx error log parser

2018-01-11 Thread Aziz Rozyev
Hi, seems, that fluentd has an nginx_parser plugin already, another solution that probably should work is to use the grep filters, something as follows: @type grep key client patter ^client.*\ $ key server pattern ^server.*\ $ key host

limit_req is not working in virutal location?

2018-01-11 Thread pva
Hi. Could you, please, explain why limit_req in @limitspeed location is not working in case of redirect to @allowed virtual location and works in case I copy @allowed virtual location contents inside @limitspeed? = This configuration is not limiting speed at all ==