On 26/06/08 at 19:24 +0200, Lucas Nussbaum wrote: > On 26/06/08 at 10:04 +0200, Lucas Nussbaum wrote: > > Package: approx > > Version: 3.2.0+b1 > > Severity: important > > > > Hi, > > > > I run regular rebuilds of all packages in Debian. My process is: > > 1. Update my local mirror > > 2. Install some build nodes > > 3. build each package in Debian on one of the build nodes > > > > Each node builds several packages (8) at the same time. So approx might > > have to answer several requests at the same time. > > > > I'm running etch with some packages from testing (including approx). > > > > My local mirror is accessible using HTTP. Each build node runs a local > > cache. I used to use apache's caching features, but I recently switched > > to approx. > > > > My problem is that sometimes, packages fail to be fetched with a 404 > > error. For example: > > Get:21 http://localhost sid/main libxcb-xlib0-dev 1.1-1.1 [14.7kB] > > Err http://localhost sid/main libx11-dev 2:1.1.4-2 > > 404 Not Found > > > > This is a quite rare event: out of ~12000 source packages, 282 packages > > failed to build because of that. But it's still super-annoying in my > > case, of course :-) > > > > The packages that fail to be fetched are available on my local mirror. > > They don't systematically fail to be fetched. 280 packages failed to be > > fetched at least once. (see attachment: failed_to_be_fetched.txt) As an > > example, libfontconfig1 failed to be fetched 52 times, but was needed to > > build 2745 packages. > > > > I'll run another rebuild, with approx verbose and debug options on, and > > I'll tell you if I can find something... > > Here is the approx debug log for one of the failures > (/debian/pool/main/t/texlive-extra/texlive-publishers_2007.dfsg.2-1_all.deb) > > Jun 26 10:35:33 bordemer-6 approx: Connection from 127.0.0.1 port 33542 > Jun 26 10:35:33 bordemer-6 approx: Request > /debian/pool/main/t/texlive-extra/texlive-publishers_2007.dfsg.2-1_all.deb > Jun 26 10:35:33 bordemer-6 approx: Host: localhost:9999 > Jun 26 10:35:33 bordemer-6 approx: Connection: keep-alive > Jun 26 10:35:33 bordemer-6 approx: User-Agent: Debian APT-HTTP/1.3 (0.7.14) > Jun 26 10:35:33 bordemer-6 approx: => cache miss > Jun 26 10:35:33 bordemer-6 approx: Command: /usr/bin/curl --fail --silent > --include > "http://idpot.grenoble.grid5000.fr/mirrors/debian/pool/main/t/texlive-extra/texlive-publishers_2007.dfsg.2-1_all.deb"; > > Jun 26 10:35:33 bordemer-6 approx: Connection from 127.0.0.1 port 33319 > Jun 26 10:35:33 bordemer-6 approx: Request > /debian/pool/main/t/texlive-lang/texlive-lang-italian_2007.dfsg.3-2_all.deb > Jun 26 10:35:33 bordemer-6 approx: Host: localhost:9999 > Jun 26 10:35:33 bordemer-6 approx: Connection: keep-alive > Jun 26 10:35:33 bordemer-6 approx: User-Agent: Debian APT-HTTP/1.3 (0.7.14) > Jun 26 10:35:33 bordemer-6 approx: => delivering from cache > Jun 26 10:35:33 bordemer-6 approx: Local response > Jun 26 10:35:33 bordemer-6 approx: Content-Type: text/plain > Jun 26 10:35:33 bordemer-6 approx: Content-Length: 26438 > Jun 26 10:35:33 bordemer-6 approx: Last-Modified: Mon, 28 Apr 2008 09:33:03 > GMT > Jun 26 10:35:33 bordemer-6 approx: Connection from 127.0.0.1 port 33319 > Jun 26 10:35:33 bordemer-6 approx: Request > /debian/pool/main/t/texlive-extra/texlive-publishers_2007.dfsg.2-1_all.deb > Jun 26 10:35:33 bordemer-6 approx: Host: localhost:9999 > Jun 26 10:35:33 bordemer-6 approx: Connection: keep-alive > Jun 26 10:35:33 bordemer-6 approx: User-Agent: Debian APT-HTTP/1.3 (0.7.14) > Jun 26 10:35:33 bordemer-6 approx: => cache miss > Jun 26 10:35:33 bordemer-6 approx: Command: /usr/bin/curl --fail --silent > --include > "http://idpot.grenoble.grid5000.fr/mirrors/debian/pool/main/t/texlive-extra/texlive-publishers_2007.dfsg.2-1_all.deb"; > > Jun 26 10:35:33 bordemer-6 approx: HTTP/1.1 200 OK > Jun 26 10:35:33 bordemer-6 approx: Date: Thu, 26 Jun 2008 08:35:34 GMT > Jun 26 10:35:33 bordemer-6 approx: Server: Apache/2.0.55 (Ubuntu) > Jun 26 10:35:33 bordemer-6 approx: Last-Modified: Sun, 11 May 2008 09:47:05 > GMT > Jun 26 10:35:33 bordemer-6 approx: ETag: "4364072-1ae646-504e9840" > Jun 26 10:35:33 bordemer-6 approx: Accept-Ranges: bytes > Jun 26 10:35:33 bordemer-6 approx: Content-Length: 1762886 > Jun 26 10:35:33 bordemer-6 approx: Content-Type: > application/x-debian-package > Jun 26 10:35:33 bordemer-6 approx: X-Pad: avoid browser bug > Jun 26 10:35:33 bordemer-6 approx: open cache > debian/pool/main/t/texlive-extra/texlive-publishers_2007.dfsg.2-1_all.deb > Jun 26 10:35:33 bordemer-6 approx: Proxy response > Jun 26 10:35:33 bordemer-6 approx: Content-Type: text/plain > Jun 26 10:35:33 bordemer-6 approx: Status: 200 > Jun 26 10:35:33 bordemer-6 approx: Content-Length: 1762886 > Jun 26 10:35:33 bordemer-6 approx: Last-Modified: Sun, 11 May 2008 09:47:05 > GMT > Jun 26 10:35:33 bordemer-6 approx: HTTP/1.1 200 OK > Jun 26 10:35:33 bordemer-6 approx: Date: Thu, 26 Jun 2008 08:35:34 GMT > Jun 26 10:35:33 bordemer-6 approx: Server: Apache/2.0.55 (Ubuntu) > Jun 26 10:35:33 bordemer-6 approx: Last-Modified: Sun, 11 May 2008 09:47:05 > GMT > Jun 26 10:35:33 bordemer-6 approx: ETag: "4364072-1ae646-504e9840" > Jun 26 10:35:33 bordemer-6 approx: Accept-Ranges: bytes > Jun 26 10:35:33 bordemer-6 approx: Content-Length: 1762886 > Jun 26 10:35:33 bordemer-6 approx: Content-Type: > application/x-debian-package > Jun 26 10:35:33 bordemer-6 approx: X-Pad: avoid browser bug > Jun 26 10:35:33 bordemer-6 approx: open cache > debian/pool/main/t/texlive-extra/texlive-publishers_2007.dfsg.2-1_all.deb > Jun 26 10:35:33 bordemer-6 approx: Cannot cache > debian/pool/main/t/texlive-extra/texlive-publishers_2007.dfsg.2-1_all.deb > Jun 26 10:35:33 bordemer-6 approx: > http://idpot.grenoble.grid5000.fr/mirrors/debian/pool/main/t/texlive-extra/texlive-publishers_2007.dfsg.2-1_all.deb: > not found > Jun 26 10:35:33 bordemer-6 approx: Connection from 127.0.0.1 port 33319 > Jun 26 10:35:33 bordemer-6 approx: Request > /debian/pool/main/t/texlive-lang/texlive-lang-dutch_2007.dfsg.3-2_all.deb > Jun 26 10:35:33 bordemer-6 approx: Host: localhost:9999 > Jun 26 10:35:33 bordemer-6 approx: Connection: keep-alive > Jun 26 10:35:33 bordemer-6 approx: User-Agent: Debian APT-HTTP/1.3 (0.7.14) > Jun 26 10:35:33 bordemer-6 approx: => cache miss > Jun 26 10:35:33 bordemer-6 approx: Command: /usr/bin/curl --fail --silent > --include > "http://idpot.grenoble.grid5000.fr/mirrors/debian/pool/main/t/texlive-lang/texlive-lang-dutch_2007.dfsg.3-2_all.deb"; > > Jun 26 10:35:33 bordemer-6 approx: HTTP/1.1 200 OK > Jun 26 10:35:33 bordemer-6 approx: Date: Thu, 26 Jun 2008 08:35:34 GMT > Jun 26 10:35:33 bordemer-6 approx: Server: Apache/2.0.55 (Ubuntu) > Jun 26 10:35:33 bordemer-6 approx: Last-Modified: Mon, 28 Apr 2008 09:33:03 > GMT > Jun 26 10:35:33 bordemer-6 approx: ETag: "52e804a-10316-9a2ed1c0" > Jun 26 10:35:33 bordemer-6 approx: Accept-Ranges: bytes > Jun 26 10:35:33 bordemer-6 approx: Content-Length: 66326 > Jun 26 10:35:33 bordemer-6 approx: Content-Type: > application/x-debian-package > Jun 26 10:35:33 bordemer-6 approx: open cache > debian/pool/main/t/texlive-lang/texlive-lang-dutch_2007.dfsg.3-2_all.deb > Jun 26 10:35:33 bordemer-6 approx: Proxy response > Jun 26 10:35:33 bordemer-6 approx: Content-Type: text/plain > Jun 26 10:35:33 bordemer-6 approx: Status: 200 > Jun 26 10:35:33 bordemer-6 approx: Content-Length: 66326 > Jun 26 10:35:33 bordemer-6 approx: Last-Modified: Mon, 28 Apr 2008 09:33:03 > GMT > Jun 26 10:35:33 bordemer-6 approx: close cache > debian/pool/main/t/texlive-lang/texlive-lang-dutch_2007.dfsg.3-2_all.deb > Jun 26 10:35:33 bordemer-6 approx: setting mtime to Mon, 28 Apr 2008 > 09:33:03 GMT > Jun 26 10:35:33 bordemer-6 approx: > http://idpot.grenoble.grid5000.fr/mirrors/debian/pool/main/t/texlive-lang/texlive-lang-dutch_2007.dfsg.3-2_all.deb: > delivered > Jun 26 10:35:33 bordemer-6 approx: Connection from 127.0.0.1 port 33319 > Jun 26 10:35:33 bordemer-6 approx: Request > /debian/pool/main/t/texlive-lang/texlive-lang-hungarian_2007.dfsg.3-2_all.deb > Jun 26 10:35:33 bordemer-6 approx: Host: localhost:9999 > Jun 26 10:35:33 bordemer-6 approx: Connection: keep-alive > Jun 26 10:35:33 bordemer-6 approx: User-Agent: Debian APT-HTTP/1.3 (0.7.14) > Jun 26 10:35:33 bordemer-6 approx: => cache miss > Jun 26 10:35:33 bordemer-6 approx: Command: /usr/bin/curl --fail --silent > --include > "http://idpot.grenoble.grid5000.fr/mirrors/debian/pool/main/t/texlive-lang/texlive-lang-hungarian_2007.dfsg.3-2_all.deb"; > > Jun 26 10:35:33 bordemer-6 approx: HTTP/1.1 200 OK > Jun 26 10:35:33 bordemer-6 approx: Date: Thu, 26 Jun 2008 08:35:34 GMT > Jun 26 10:35:33 bordemer-6 approx: Server: Apache/2.0.55 (Ubuntu) > Jun 26 10:35:33 bordemer-6 approx: Last-Modified: Mon, 28 Apr 2008 09:33:03 > GMT > Jun 26 10:35:33 bordemer-6 approx: ETag: "52e8050-38eda-9a2ed1c0" > Jun 26 10:35:33 bordemer-6 approx: Accept-Ranges: bytes > Jun 26 10:35:33 bordemer-6 approx: Content-Length: 233178 > Jun 26 10:35:33 bordemer-6 approx: Content-Type: > application/x-debian-package > Jun 26 10:35:33 bordemer-6 approx: open cache > debian/pool/main/t/texlive-lang/texlive-lang-hungarian_2007.dfsg.3-2_all.deb > Jun 26 10:35:33 bordemer-6 approx: Proxy response > Jun 26 10:35:33 bordemer-6 approx: Content-Type: text/plain > Jun 26 10:35:33 bordemer-6 approx: Status: 200 > Jun 26 10:35:33 bordemer-6 approx: Content-Length: 233178 > Jun 26 10:35:33 bordemer-6 approx: Last-Modified: Mon, 28 Apr 2008 09:33:03 > GMT > Jun 26 10:35:33 bordemer-6 approx: close cache > debian/pool/main/t/texlive-lang/texlive-lang-hungarian_2007.dfsg.3-2_all.deb > Jun 26 10:35:33 bordemer-6 approx: setting mtime to Mon, 28 Apr 2008 > 09:33:03 GMT > Jun 26 10:35:33 bordemer-6 approx: > http://idpot.grenoble.grid5000.fr/mirrors/debian/pool/main/t/texlive-lang/texlive-lang-hungarian_2007.dfsg.3-2_all.deb: > delivered > Jun 26 10:35:33 bordemer-6 approx: Connection from 127.0.0.1 port 33319 > Jun 26 10:35:33 bordemer-6 approx: Request > /debian/pool/main/t/texlive-lang/texlive-lang-latin_2007.dfsg.3-2_all.deb > Jun 26 10:35:33 bordemer-6 approx: Host: localhost:9999 > Jun 26 10:35:33 bordemer-6 approx: Connection: keep-alive > Jun 26 10:35:33 bordemer-6 approx: User-Agent: Debian APT-HTTP/1.3 (0.7.14) > Jun 26 10:35:33 bordemer-6 approx: => cache miss > Jun 26 10:35:33 bordemer-6 approx: Command: /usr/bin/curl --fail --silent > --include > "http://idpot.grenoble.grid5000.fr/mirrors/debian/pool/main/t/texlive-lang/texlive-lang-latin_2007.dfsg.3-2_all.deb"; > > Jun 26 10:35:33 bordemer-6 approx: HTTP/1.1 200 OK > Jun 26 10:35:33 bordemer-6 approx: Date: Thu, 26 Jun 2008 08:35:34 GMT > Jun 26 10:35:33 bordemer-6 approx: Server: Apache/2.0.55 (Ubuntu) > Jun 26 10:35:33 bordemer-6 approx: Last-Modified: Mon, 28 Apr 2008 09:33:03 > GMT > Jun 26 10:35:33 bordemer-6 approx: ETag: "52e8052-6f12-9a2ed1c0" > Jun 26 10:35:33 bordemer-6 approx: Accept-Ranges: bytes > Jun 26 10:35:33 bordemer-6 approx: Content-Length: 28434 > Jun 26 10:35:33 bordemer-6 approx: Content-Type: > application/x-debian-package > Jun 26 10:35:33 bordemer-6 approx: open cache > debian/pool/main/t/texlive-lang/texlive-lang-latin_2007.dfsg.3-2_all.deb > Jun 26 10:35:33 bordemer-6 approx: Proxy response > Jun 26 10:35:33 bordemer-6 approx: Content-Type: text/plain > Jun 26 10:35:33 bordemer-6 approx: Status: 200 > Jun 26 10:35:33 bordemer-6 approx: Content-Length: 28434 > Jun 26 10:35:33 bordemer-6 approx: Last-Modified: Mon, 28 Apr 2008 09:33:03 > GMT > Jun 26 10:35:33 bordemer-6 approx: close cache > debian/pool/main/t/texlive-lang/texlive-lang-latin_2007.dfsg.3-2_all.deb > Jun 26 10:35:33 bordemer-6 approx: setting mtime to Mon, 28 Apr 2008 > 09:33:03 GMT > Jun 26 10:35:33 bordemer-6 approx: > http://idpot.grenoble.grid5000.fr/mirrors/debian/pool/main/t/texlive-lang/texlive-lang-latin_2007.dfsg.3-2_all.deb: > delivered > Jun 26 10:35:33 bordemer-6 approx: Connection from 127.0.0.1 port 33319 > Jun 26 10:35:33 bordemer-6 approx: Request > /debian/pool/main/t/texlive-base/tetex-extra_2007.dfsg.1-1_all.deb > Jun 26 10:35:33 bordemer-6 approx: Host: localhost:9999 > Jun 26 10:35:33 bordemer-6 approx: Connection: keep-alive > Jun 26 10:35:33 bordemer-6 approx: User-Agent: Debian APT-HTTP/1.3 (0.7.14) > Jun 26 10:35:33 bordemer-6 approx: => cache miss > Jun 26 10:35:33 bordemer-6 approx: Command: /usr/bin/curl --fail --silent > --include > "http://idpot.grenoble.grid5000.fr/mirrors/debian/pool/main/t/texlive-base/tetex-extra_2007.dfsg.1-1_all.deb"; > > Jun 26 10:35:33 bordemer-6 approx: close cache > debian/pool/main/t/texlive-extra/texlive-publishers_2007.dfsg.2-1_all.deb > Jun 26 10:35:33 bordemer-6 approx: setting mtime to Sun, 11 May 2008 > 09:47:05 GMT > Jun 26 10:35:33 bordemer-6 approx: > http://idpot.grenoble.grid5000.fr/mirrors/debian/pool/main/t/texlive-extra/texlive-publishers_2007.dfsg.2-1_all.deb: > delivered > > > Looking at the log, it seems that approx has a problem with handling > concurrent > requests for the same file.
The bug was easily reproduced using: rm /var/cache/approx/debian/pool/main/i/ia32-libs/ia32-libs_2.5_amd64.deb for i in $(seq 1 5); do wget -O /dev/null http://localhost:9999/debian/pool/main/i/ia32-libs/ia32-libs_2.5_amd64.deb & done Looks like a nice race condition... -- | Lucas Nussbaum | [EMAIL PROTECTED] http://www.lucas-nussbaum.net/ | | jabber: [EMAIL PROTECTED] GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
