Hi!
The link from message #19 is no longer valid, but a similarly named
branch was merged into apt's main branch in
https://salsa.debian.org/apt-team/apt/-/commit/b19bc330d597d4b872ee9416ebeb9a86b9fc0763
This may mean that the problem from this ticket has been fixed in 2005.
This was an old commit so I didn't even attempt to re-compile that old
version with the patch removed to reproduce the original issue. I didn't
try to remove that patch from today's code either, because the code
looks very different today than it did in 2005.
So instead I tried to replicate the situation where apt would try to
download multiple index files from the same repository even if a timeout
occurred.
Step 1
First the normal situation (without timeouts) that shows that there is a
remote repository with multiple index files, and that apt downloads them
all. There are multiple components, some of them with made up names but
with real deb packages and with real Packages.gz files (generated with
dpkg-scanpackages). But there is no Release or InRelease file on the
server.
root@debian:~# apt-get --version
apt 2.9.29 (amd64)
[...]
root@debian:~#
root@debian:~#
root@debian:~# cat /etc/apt/sources.list
deb [trusted=yes] http://server.local/sid sid main alpha beta gamma
delta omega
root@debian:~#
root@debian:~#
root@debian:~# apt-get -o Acquire::CompressionTypes::Order="gz" update |
grep -vE "Translation|Components"
Ign:1 http://server.local/sid sid InRelease
Ign:2 http://server.local/sid sid Release
Get:3 http://server.local/sid sid/main all Packages [79.5 kB]
Get:4 http://server.local/sid sid/main amd64 Packages [135 kB]
Get:9 http://server.local/sid sid/alpha all Packages [9,101 B]
Get:10 http://server.local/sid sid/alpha amd64 Packages [16.6 kB]
Get:15 http://server.local/sid sid/beta amd64 Packages [12.1 kB]
Get:16 http://server.local/sid sid/beta all Packages [565 B]
Get:21 http://server.local/sid sid/gamma amd64 Packages [10.6 kB]
Get:22 http://server.local/sid sid/gamma all Packages [1,926 B]
Get:27 http://server.local/sid sid/delta all Packages [15.9 kB]
Get:28 http://server.local/sid sid/delta amd64 Packages [1,573 B]
Get:33 http://server.local/sid sid/omega all Packages [4,236 B]
Get:34 http://server.local/sid sid/omega amd64 Packages [4,236 B]
Fetched 291 kB in 1s (559 kB/s)
Reading package lists...
root@debian:~#
root@debian:~#
Step 2
In this step I'm blocking responses from the server (adding "iptables -P
OUTPUT DROP" on the server) to trigger timeout in transfers.
Now the apt-get update looks like this:
root@debian:~# apt-get -o Acquire::CompressionTypes::Order="gz" update
Ign:1 http://server.local/sid sid InRelease
Ign:1 http://server.local/sid sid InRelease
Ign:1 http://server.local/sid sid InRelease
Err:1 http://server.local/sid sid InRelease
Could not connect to server.local:80 (192.168.0.50), connection timed out
Unable to connect to server.local:http:
Reading package lists... Done
W: Failed to fetch http://server.local/sid/dists/sid/InRelease Unable
to connect to server.local:http:
W: Some index files failed to download. They have been ignored, or old
ones used instead.
N: Some sources can be modernized. Run 'apt modernize-sources' to do so.
As you can see, now apt recognizes timeout of first download, and
doesn't move to next download from the same server.
Given the observation from step 2, I think that this problem no longer
occurs. If no one will have problems with the tests or the conclusion, I
will close this ticket in ~week from now. I just can't find apt's
version number in which the fix was available.
Also there is a config option that controls the connection timeout
(Acquire::http::Timeout) and its default value has been decreased some
time ago from 120 to 30 seconds
(https://salsa.debian.org/apt-team/apt/-/commit/329a4a6159f1972ff5ec7bc2db26430f26dc61f3).
Kamil
