Re: Status code 0
Hello Just to close that conversation, it seems this was an error of our devops in charge of alerting, who was using curl in a bad way. Best regards, Sébastien Le lun. 6 mai 2024 à 11:33, Sébastien Rebecchi a écrit : > Hello! > > There is nothing regarding this issue in nginx logs. > > Now I think the issue is not with nginx itself, but in front of nginx, > with Linux itself. > We monitor using curl, and it seems that curl can print status code 0 when > it can not establish a connection with the server. > I think the Linux kernel is not configured properly to handle our > connection peaks. Looking at net.core.somaxconn, we have the default of > 128. More generally, I will deep dive into this and possibly other kernel > settings to see if optimizing them will solve the problem. For information, > on each of our servers we have an average of around 80K simultaneous TCP > connections globally and around 35K in state ESTABLISHED (according to > netstat/ss -nta), and more during high peaks. > I saw this doc section which is a good starting point > https://www.nginx.com/blog/tuning-nginx/#Tuning-Your-Linux-Configuration > If you have any advice on other settings to increase, this would be very > appreciated. > > I will keep you in touch about my investigations, to confirm at least that > there is no bug on nginx side, which i am quite confident about now. > > Thank you very much for your help! > > Le sam. 4 mai 2024 à 20:47, Maxim Dounin a écrit : > >> Hello! >> >> On Sat, May 04, 2024 at 07:31:43PM +0200, Sébastien Rebecchi wrote: >> >> > Hello >> > >> > What does it mean when nginx returns an http status code 0? >> > >> > We see that cause we monitor nginx response status code, which is used >> as >> > front of our apps. The apps themselves can not return 0, only 200 or >> 500. >> > So it seems issue here comes from nginx itself which can not process the >> > connection under high peak of load, but why 0, is that expected? >> >> Status code 0 as seen in nginx http access logs means that nginx >> wasn't able to generate any reasonable status code, even some >> generic failure like 500 (Internal Server Error), yet the request >> was closed. >> >> This usually happens due to some fatal issues, like unexpected >> conditions (which might indicate a bug somewhere), unexpected >> errors, or memory allocation errors if it wasn't possible to >> return 500. >> >> In most cases there should be additional details in the error log >> explaining the reasons. If there aren't any, or reasons aren't >> clear, it might be a good idea to dig further. >> >> -- >> Maxim Dounin >> http://mdounin.ru/ >> -- >> nginx mailing list >> ng...@freenginx.org >> https://freenginx.org/mailman/listinfo/nginx >> > ___ nginx mailing list nginx@nginx.org https://mailman.nginx.org/mailman/listinfo/nginx
nginx configured as loadbalancer returning 404 not found
Hi,
I am running nginx version 1.26 on "Ubuntu 22.04.4 LTS" I have configured
the nginx as load balancer and the configuration details are as follows
# nginx -v
nginx version: nginx/1.26.0
#
server {
listen 8085;
#server_name 172.30.2.11;
server name 210.11.1.110;
location / {
# Define the upstream servers for load balancing
proxy_pass http://backend/;
# Set HTTP headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /api/docs/ {
proxy_pass http://backend/api/docs/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
upstream backend {
server tead-local.com:80;
}
When i hit http://tead-local.com:80/api/docs/ I get http 200 response from
the backend server whereas when I try to hit using public IP :-
http://210.11.1.110:8085/api/docs/ I encounter http 404 not found.
101.0.62.200 - - [17/May/2024:16:38:24 +0530] "GET /api/docs/ HTTP/1.1" 404
153 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)
AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15
Ddg/17.5" "-"
[image: image.png]
Please guide me. Thanks in advance.
Best Regards,
Kaushal
___
nginx mailing list
nginx@nginx.org
https://mailman.nginx.org/mailman/listinfo/nginx
Re: nginx configured as loadbalancer returning 404 not found
Hi Kaushal,
On Fri, May 17, 2024 at 04:49:59PM +0530, Kaushal Shriyan wrote:
>
> I am running nginx version 1.26 on "Ubuntu 22.04.4 LTS" I have configured
> the nginx as load balancer and the configuration details are as follows
>
> # nginx -v
> nginx version: nginx/1.26.0
> #
>
> server {
[...]
>
> location / {
> # Define the upstream servers for load balancing
> proxy_pass http://backend/;
Could you please explain a reason why did you decide to use `/' after
the backend's name in the proxy_pass directive.
> # Set HTTP headers
> proxy_set_header Host $host;
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header X-Forwarded-Proto $scheme;
> }
>
> location /api/docs/ {
> proxy_pass http://backend/api/docs/;
It seems like '/api/docs/' can be safely removed, so
I'd recommend to read the documentation for the proxy_pass directive, [1]
If proxy_pass is specified without a URI, the request URI is passed to the
server in the same form as sent by a client when the original request is
processed, or the full normalized request URI is passed when processing
the changed URI:
location /some/path/ {
proxy_pass http://127.0.0.1;
}
[...]
> When i hit http://tead-local.com:80/api/docs/ I get http 200 response from
> the backend server whereas when I try to hit using public IP :-
> http://210.11.1.110:8085/api/docs/ I encounter http 404 not found.
>
> 101.0.62.200 - - [17/May/2024:16:38:24 +0530] "GET /api/docs/ HTTP/1.1" 404
> 153 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)
> AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15
> Ddg/17.5" "-"
To see the whole picture of processing a request by nginx, I'd
also recommend to enable a debugging log, [2].
Hope that helps.
References
--
1. https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass
2. https://nginx.org/en/docs/debugging_log.html
--
Sergey A. Osokin
___
nginx mailing list
nginx@nginx.org
https://mailman.nginx.org/mailman/listinfo/nginx
Re: nginx configured as loadbalancer returning 404 not found
On Fri, May 17, 2024 at 7:39 PM Sergey A. Osokin wrote:
> Hi Kaushal,
>
> On Fri, May 17, 2024 at 04:49:59PM +0530, Kaushal Shriyan wrote:
> >
> > I am running nginx version 1.26 on "Ubuntu 22.04.4 LTS" I have configured
> > the nginx as load balancer and the configuration details are as follows
> >
> > # nginx -v
> > nginx version: nginx/1.26.0
> > #
> >
> > server {
> [...]
> >
> > location / {
> > # Define the upstream servers for load balancing
> > proxy_pass http://backend/;
>
> Could you please explain a reason why did you decide to use `/' after
> the backend's name in the proxy_pass directive.
>
> > # Set HTTP headers
> > proxy_set_header Host $host;
> > proxy_set_header X-Real-IP $remote_addr;
> > proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> > proxy_set_header X-Forwarded-Proto $scheme;
> > }
> >
> > location /api/docs/ {
> > proxy_pass http://backend/api/docs/;
>
> It seems like '/api/docs/' can be safely removed, so
> I'd recommend to read the documentation for the proxy_pass directive, [1]
>
>
>
> If proxy_pass is specified without a URI, the request URI is passed to the
> server in the same form as sent by a client when the original request is
> processed, or the full normalized request URI is passed when processing
> the changed URI:
>
> location /some/path/ {
> proxy_pass http://127.0.0.1;
> }
>
>
>
> [...]
>
> > When i hit http://tead-local.com:80/api/docs/ I get http 200 response
> from
> > the backend server whereas when I try to hit using public IP :-
> > http://210.11.1.110:8085/api/docs/ I encounter http 404 not found.
> >
> > 101.0.62.200 - - [17/May/2024:16:38:24 +0530] "GET /api/docs/ HTTP/1.1"
> 404
> > 153 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)
> > AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15
> > Ddg/17.5" "-"
>
> To see the whole picture of processing a request by nginx, I'd
> also recommend to enable a debugging log, [2].
>
> Hope that helps.
>
> References
> --
> 1. https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass
> 2. https://nginx.org/en/docs/debugging_log.html
>
> --
> Sergey A. Osokin
>
Thanks Sergey for the detailed explanation. I have modified the
/etc/nginx/conf.d/loadbalancer.conf file (nginx server running in
loadbalancer mode). The upstream backend -> tead-local.com:80 is hosted on
docker based container running nginx service (version :- 1.21.6)
##loadbalancer.conf###
server {
listen 80;
server_name testbe.mydomain.com;
error_log /var/log/nginx/nginxdebug.log debug;
location / {
# Define the upstream servers for load balancing
proxy_pass http://backend;
# Set HTTP headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
error_log /var/log/nginx/nginxlocationdebug.log debug;
}
}
upstream backend {
server tead-local.com:80;
}
##
[image: image.png]
# ll
total 12
drwxr-xr-x 2 root adm 93 May 18 01:05 ./
drwxrwxr-x 15 root syslog 4096 May 16 16:33 ../
-rw-r--r-- 1 root root621 May 18 01:05 access.log
-rw-r--r-- 1 root root594 May 18 01:05 error.log
-rw-r--r-- 1 root root 0 May 18 01:05 nginxdebug.log
-rw-r--r-- 1 root root 0 May 18 01:05 nginxlocationdebug.log
#
root@lb-01:/var/log/nginx# cat error.log
2024/05/18 01:05:15 [notice] 539625#539625: using the "epoll" event method
2024/05/18 01:05:15 [notice] 539625#539625: nginx/1.26.0
2024/05/18 01:05:15 [notice] 539625#539625: built by gcc 11.4.0 (Ubuntu
11.4.0-1ubuntu1~22.04)
2024/05/18 01:05:15 [notice] 539625#539625: OS: Linux 5.15.0-105-generic
2024/05/18 01:05:15 [notice] 539625#539625: getrlimit(RLIMIT_NOFILE):
1024:524288
2024/05/18 01:05:15 [notice] 539626#539626: start worker processes
2024/05/18 01:05:15 [notice] 539626#539626: start worker process 539627
2024/05/18 01:05:15 [notice] 539626#539626: start worker process 539628
root@lb-01:/var/log/nginx# ll
# cat access.log
101.0.62.200 - - [18/May/2024:01:05:19 +0530] "GET /api/docs HTTP/1.1" 404
555 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36" "-"
101.0.62.200 - - [18/May/2024:01:05:20 +0530] "GET /api/docs HTTP/1.1" 404
555 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36" "-"
101.0.62.200 - - [18/May/2024:01:05:21 +0530] "GET /api/docs HTTP/1.1" 404
555 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36
(KHTML, like Gec
