Hi, On Thu, 21 Jan 2021 15:27:40 +0000 Witold Baryluk <witold.bary...@gmail.com> wrote: > Package: apt > Version: 2.1.18 > Followup-For: Bug #973861 > X-Debbugs-Cc: witold.bary...@gmail.com, mir...@init7.net > > I started getting this error recently too. > > To the point that using live-build (that downloads ~6100 package in one > go) is not possible, because some package will fail to download and fail > entire apt install request. > > > This is related to keepalive. > > For example most nginx server has keepalive_requests 100 by default. That > is, after serving 100 request, nginx server will close the connection. > This is because some dynamic memory allocated by nginx is allocated on > arenas that are tied to TCP connection, and they are never freed until > the connection is closed. This is a high performance design, but to > prevent leaks and abuse, the connection must be closed after some number > of requests. > > Most tools (wget), respect the response from the server, close the > existing connection and open a new one. It is easy to see with wget -i - > and feeding the same file 10000 times. Every 100th request will start a > new TCP connection. > > > In apt this manifests as some XX00 request failing. > > Get:4500 http://mirror.init7.net/debian bullseye/main amd64 libplymouth5 > amd64 0.9.5-2 [108 kB] > Err:4500 http://mirror.init7.net/debian bullseye/main amd64 libplymouth5 > amd64 0.9.5-2 > Undetermined Error [IP: 2001:1620::1620 80] > > > Get:1600 http://mirror.init7.net/debian bullseye/main amd64 libastyle3 amd64 > 3.1-2+b1 [109 kB] > Err:1600 http://mirror.init7.net/debian bullseye/main amd64 libastyle3 amd64 > 3.1-2+b1] > Undetermined Error [IP: 2001:1620::1620 80] > > > > This is repeatable and reproducible. I can try 10 times doing the > download, and every time the same file and number will fail. (Sometimes I > might get lucky and one of them will succeed, but not ther other). > > I don't know why files 100, 200, 300, ... doesn't fail, but it looks like > some bug in apt in general. > > (The above files do exist and are served correctly by the server). > > > Does apt use maybe HTTP pipelining in addition to keepalive, that causes > issues? Or maybe in general the retry logic is broken somehow. > > I wasn't able to reproduce this issue with Fastly CDN or Apache 2.4, > probably because they use different keepalive request limit policy. It > might be a bug in nginx too. I am not sure. The server I was using > appears to be using nginx/1.14.2.
I recently reported this issue to nginx developers and they have issued a fix for handling the close of pipelined connections. See: https://hg.nginx.org/nginx/rev/cffaf3f2eec8 > > Regards, Cheers, Miao Wang
