Hi Joey (2011.08.20_17:55:45_+0200)
> I've not been able to work on this due to
>
> * Not having sufficient bandwidth to work on debmirror lately
> (my own mirror is 2 months out of date and the only machine
> where I can run debmirror iteratively right now is a slow arm box).
No problem, thanks for replying anyway. You don't need a mirror you can
run demirror iteratively on, you can recreate the issue when mirroring
for the first time (assuming you have a local, consistent mirror, that
debmirror can mirror).
I meant to find you at Debconf, but it was a pretty busy week :/
> * Not really understanding the above description. How are the
> Packages files supposed to be broken to reproduce it?
Just insert random junk at the end, or anything to break the checksum in
the Releases file, that it will be verified against.
> How can debmirror be checking Packages files before it downloads them?
It checks to see if the already downloaded / generated file is correct.
e.g.
| if (exists $file_lists{"$tempdir/$subdir/$file"}) {
| if (!check_lists("$tempdir/$subdir/$file")) {
| if (remote_get("$subdir/$file")) {
The check_lists prepends $tempdir (because the file is currently stored
in $tempdir and we are currently in $mirrordir).
remote_get changes to $tempdir, downloads the Packages* file, and can't
check the checksum because there is no $subdir/$file in $file_lists.
> Where does bz2 fit into all this?
debmirror first requests Packages.gz, this fails to verify.
Then it requests Packages, which doesn't exist server side, so it
receives Packages.bz2 instead:
| HEAD
http://localhost/debian-security/dists/squeeze/updates/main/binary-amd64/Packages
HTTP/1.0
|
| HTTP/1.1 200 OK
| Date: Sat, 20 Aug 2011 20:54:12 GMT
| Server: Apache/2.2.19 (Debian)
| Content-Location: Packages.bz2
| Vary: negotiate,accept
| TCN: choice
| Last-Modified: Thu, 18 Aug 2011 03:10:13 GMT
| ETag: "1c268d-2370f-4aabef3934336;4aaf5c75c7361"
| Accept-Ranges: bytes
| Content-Length: 145167
| Connection: close
| Content-Type: application/x-bzip2
This file isn't deleted again, because remote_get can't check the
checksum. So debmirror goes on to try and parse the file.
> Obviously it would sorta fix your problem, but it's not clear what the
> user should do if this happens, or even what is broken.
I think it's sensible for debmirror to abort if its trying to do things
with a file that it clearly can't read. Similar parsing code in Python
would probably have resulted in an exception.
> > 0002-Before-download-Package-list-files-are-checked-with-.patch:
> > Before download, Package list files are checked with prepended $tempdir.
> > At download time, Package list files are verified without it. Store both
> > forms in file_lists.
>
> If storing the same data under a different filename fixes this, then
> surely debmirror is broken at a deeper level, and should be fixed there,
> so it always verifies the Packages files using the same filename?
The problem is that it needs to be under both names. Or maybe the
solution is to add a parameter to check_lists to change into a directory
before checking. Then we can avoid prepending $tempdir, everywhere.
SR
--
Stefano Rivera
http://tumbleweed.org.za/
H: +27 21 465 6908 C: +27 72 419 8559 UCT: x3127
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
