Package: apt-cacher-ng
Version: 3-5
Severity: normal
Tags: patch
Hello,
I arrived at this problem when I was trying to figure out why
'apt-get update' was trying to download complete indexes instead of
patching them with diffs. I decided that I would try deleting the Index
file of the diffs from the cache, e.g.
rm -f
/var/cache/apt-cacher-ng/debrep/dists/sid/contrib/binary-amd64/Packages.diff/Index*
Anyway, the problem with updating via diffs seemed to resolve itself, but
after some time I noticed that the expiration task was always failing
and I couldn't figure out why.
After some debugging, I found that it had to do with the deletion of
those Index files. I have attached a copy of the maintenance logs for
you. Basically, because of the deletion, it tries to restore it from a
hashed file, however it gets the incorrect file name. This file doesn't
exist so the task fails.
I have attached some patches which fix the problem. I have also pushed
my changes to the fix-expire branch of this git repo:
https://github.com/e7appew/apt-cacher-ng.git.
Best regards,
Carlos
-- Package-specific info:
-- System Information:
Debian Release: buster/sid
APT prefers unstable
APT policy: (990, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 4.12.0-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8),
LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages apt-cacher-ng depends on:
ii adduser 3.116
ii debconf 1.5.63
ii dpkg 1.18.24
ii init-system-helpers 1.49
ii libbz2-1.0 1.0.6-8.1
ii libc6 2.24-14
ii libgcc1 1:7.2.0-1
ii liblzma5 5.2.2-1.3
ii libssl1.1 1.1.0f-4
ii libstdc++6 7.2.0-1
ii libsystemd0 234-2
ii libwrap0 7.6.q-26
ii lsb-base 9.20161125
ii zlib1g 1:1.2.8.dfsg-5
apt-cacher-ng recommends no packages.
Versions of packages apt-cacher-ng suggests:
ii avahi-daemon 0.6.32-2
ii doc-base 0.10.7
ii libfuse2 2.9.7-1
-- Configuration Files:
/etc/apt-cacher-ng/acng.conf changed [not included]
/etc/apt-cacher-ng/security.conf [Errno 13] Permission denied:
'/etc/apt-cacher-ng/security.conf'
-- debconf information excluded
<html><head><style type="text/css">.WARNING { color: orange; }
.ERROR { color: red; }
</style></head><body>Maintenance task <b>Expiration</b>, apt-cacher-ng version:
3<br>
Server link: <a
href="http://192.168.0.1:3142/acng-report.html";>http://192.168.0.1:3142/acng-report.html</a><br>
<br>
<b>Locating potentially expired files in the cache...</b><br>
Scanning, found 1 file...<br />
Scanning, found 2 files...<br />
Scanning, found 4 files...<br />
Scanning, found 8 files...<br />
Scanning, found 16 files...<br />
Scanning, found 32 files...<br />
Scanning, found 64 files...<br />
Scanning, found 128 files...<br />
Scanning, found 256 files...<br />
Scanning, found 512 files...<br />
Scanning, found 1024 files...<br />
Scanning, found 2048 files...<br />
Scanning, found 4096 files...<br />
Scanning, found 8192 files...<br />
Scanning, found 16384 files...<br />
Scanning, found 32768 files...<br />
Scanning, found 65536 files...<br />
Found 82898 files.<br />
<b>Checking implicitly referenced files...</b><br>Checking/Updating
archive.canonical.com/ubuntu/dists/zesty/InRelease...
<i>(0KiB)</i>
<br>
Checking/Updating deb.avast.com/lin/repo/dists/debian/Release...
<i>(0KiB)</i>
<br>
Checking/Updating debrep/dists/buster/Release...
<i>(0KiB)</i>
<br>
Checking/Updating debrep/dists/jessie/Release...
<i>(0KiB)</i>
<br>
Checking/Updating debrep/dists/rc-buggy/InRelease...
<i>(0KiB)</i>
<br>
Checking/Updating debrep/dists/stable/Release...
<i>(0KiB)</i>
<br>
Checking/Updating debrep/dists/testing/Release...
<i>(0KiB)</i>
<br>
Restoring virtual file
debrep/dists/unstable/contrib/binary-amd64/Packages.diff/Index (equal to )<br>
Couldn't read/var/cache/apt-cacher-ng/.head<br>Restoring virtual file
debrep/dists/unstable/contrib/binary-amd64/Packages.xz (equal to
debrep/dists/unstable/contrib/binary-amd64/by-hash/SHA256/5e39c53b422035f39a69c8cffc05bfe4d36c853d9b0e4906555ee9bbc0f77ea0)<br>
Restoring virtual file debrep/dists/unstable/contrib/binary-i386/Packages.xz
(equal to
debrep/dists/unstable/contrib/binary-i386/by-hash/SHA256/4c5351b9c57cfc0af2bdc899c73d3191d742e1c35cb837458e935404914fec00)<br>
Restoring virtual file debrep/dists/unstable/contrib/i18n/Translation-en.bz2
(equal to
debrep/dists/unstable/contrib/i18n/by-hash/SHA256/3f097d347196f3eea992977805a4fe0f6ef975ce63e3fbbb8e31bc3f2b99486d)<br>
Restoring virtual file debrep/dists/unstable/contrib/source/Sources.diff/Index
(equal to )<br>
Couldn't read/var/cache/apt-cacher-ng/.head<br>Restoring virtual file
debrep/dists/unstable/contrib/source/Sources.xz (equal to
debrep/dists/unstable/contrib/source/by-hash/SHA256/850991bf3f17f867a812ce1401d226c54bc92b462ae68b56f595e741bafc4f2f)<br>
Restoring virtual file debrep/dists/unstable/main/binary-amd64/Packages.xz
(equal to
debrep/dists/unstable/main/binary-amd64/by-hash/SHA256/d9c0b51e8200f2433484a1f0d924c0ad7b16d382a8111b58019269524c88106d)<br>
Restoring virtual file debrep/dists/unstable/main/binary-i386/Packages.xz
(equal to
debrep/dists/unstable/main/binary-i386/by-hash/SHA256/853ecb13ad86b1f2980440affec6136a17e53b711af1ac6f17d6ea9823909765)<br>
Restoring virtual file debrep/dists/unstable/main/i18n/Translation-en.bz2
(equal to
debrep/dists/unstable/main/i18n/by-hash/SHA256/bf4bee3ecb49175365d6d966531af658b4b21a654d765f3765573203d6a131cc)<br>
Restoring virtual file debrep/dists/unstable/main/source/Sources.xz (equal to
debrep/dists/unstable/main/source/by-hash/SHA256/9cd01c7e7de2193cd700a7043e5592280d1ceb24a33132e7236d8f1598a66bc6)<br>
Restoring virtual file debrep/dists/unstable/non-free/binary-amd64/Packages.xz
(equal to
debrep/dists/unstable/non-free/binary-amd64/by-hash/SHA256/4a32c127be004bc1596ab2479e21469244d04c86a0f09028848b3c0f92b7a5a1)<br>
Restoring virtual file debrep/dists/unstable/non-free/i18n/Translation-en.bz2
(equal to
debrep/dists/unstable/non-free/i18n/by-hash/SHA256/23793924a9b1d1159f3572c5f45f2db2ad0c506ab5c4dcf41c8364ff8e15693a)<br>
Restoring virtual file debrep/dists/unstable/non-free/source/Sources.diff/Index
(equal to )<br>
Couldn't read/var/cache/apt-cacher-ng/.head<br>Restoring virtual file
debrep/dists/unstable/non-free/source/Sources.xz (equal to
debrep/dists/unstable/non-free/source/by-hash/SHA256/b2c8f454c0e8c76f110e00a7ae53ffd901b09f1bd1c207cb00ebe6aa07350acb)<br>
<b>Bringing index files up to date...</b><br>
Restoring virtual file
debrep/dists/sid/contrib/binary-amd64/Packages.diff/Index (equal to )<br>
Couldn't read/var/cache/apt-cacher-ng/.head<br>Restoring virtual file
debrep/dists/sid/contrib/binary-amd64/Packages.xz (equal to
debrep/dists/sid/contrib/binary-amd64/by-hash/SHA256/b5bd93eb5605221c0555c172d6ffaac8f390db163026bc57cabae8550d1d6879)<br>
Restoring virtual file debrep/dists/sid/contrib/binary-i386/Packages.xz (equal
to
debrep/dists/sid/contrib/binary-i386/by-hash/SHA256/d41f6c901f10a440c5c03e4d48c7be656840a4f3bac7cbe139a45c9a722fa249)<br>
Restoring virtual file debrep/dists/sid/contrib/i18n/Translation-en.bz2 (equal
to
debrep/dists/sid/contrib/i18n/by-hash/SHA256/977cb68d1c860fb38319cfab3b59e819d40a8214fbea96288e240d44614b714e)<br>
Restoring virtual file debrep/dists/sid/contrib/source/Sources.diff/Index
(equal to )<br>
Couldn't read/var/cache/apt-cacher-ng/.head<br>Restoring virtual file
debrep/dists/sid/contrib/source/Sources.xz (equal to
debrep/dists/sid/contrib/source/by-hash/SHA256/e1f03e196f369ceabfe48c94de33b0bf3b2f027128b7c9a3e571ec65bc6685fa)<br>
Restoring virtual file debrep/dists/sid/main/binary-amd64/Packages.xz (equal to
debrep/dists/sid/main/binary-amd64/by-hash/SHA256/df3930904f712e34e086030193119be1273430a630c693d533702e06777e01da)<br>
Restoring virtual file debrep/dists/sid/main/binary-i386/Packages.xz (equal to
debrep/dists/sid/main/binary-i386/by-hash/SHA256/e5d4bf9210bc3516aeb69d13886e81479ebb7ec81280ffd92b11cf345caf7709)<br>
Restoring virtual file debrep/dists/sid/main/i18n/Translation-en.bz2 (equal to
debrep/dists/sid/main/i18n/by-hash/SHA256/9f6cb58d219d4b7d779d681b5b1e332d5f2cb254151d99bb4bbf8bb9b605581e)<br>
Restoring virtual file debrep/dists/sid/main/source/Sources.xz (equal to
debrep/dists/sid/main/source/by-hash/SHA256/36499b71adaf3589951f065fed405511600ac7d2f62f07f6099af2ca707946f8)<br>
Restoring virtual file debrep/dists/sid/non-free/binary-amd64/Packages.xz
(equal to
debrep/dists/sid/non-free/binary-amd64/by-hash/SHA256/f3bd71a2ea115fc0ebc7c0ce8642fdf584b1646c59cdb82bec997fc4be0aed50)<br>
Restoring virtual file debrep/dists/sid/non-free/i18n/Translation-en.bz2 (equal
to
debrep/dists/sid/non-free/i18n/by-hash/SHA256/4c054c92c5d9d3e4a99c2595a34af733d27d6633a40f701aa7fa8d52fe9091c6)<br>
Restoring virtual file debrep/dists/sid/non-free/source/Sources.diff/Index
(equal to )<br>
Couldn't read/var/cache/apt-cacher-ng/.head<br>Restoring virtual file
debrep/dists/sid/non-free/source/Sources.xz (equal to
debrep/dists/sid/non-free/source/by-hash/SHA256/86b9a947ff12cff4f3fae5a6d60d7b2764682b86b24cdced9d2ad198f9635f5b)<br>
There were error(s) processing debrep/dists/sid/121503199304556538597,
ignoring...<br>
<span class="ERROR">Found errors during processing, aborting as
requested.</span>
<!--
41d_a6aeb8-26dfa2Errors found, aborting expiration...
-->
</body></html>>From c59679cf75938a69aeb40eab2ead5c7ad42da0a1 Mon Sep 17 00:00:00 2001
From: Carlos Maddela <e7ap...@gmail.com>
Date: Mon, 21 Aug 2017 22:19:42 +1000
Subject: [PATCH 1/2] Allow build with DEBUGIDX defined without DEBUG.
---
include/cacheman.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/cacheman.h b/include/cacheman.h
index 7d81e9e..95f5c2b 100644
--- a/include/cacheman.h
+++ b/include/cacheman.h
@@ -76,7 +76,7 @@ public:
{
bro = this;
};
-#ifdef DEBUG
+#if defined(DEBUG) || defined(DEBUGIDX)
inline tSS toString() const
{
return tSS() << alreadyparsed << "|"
--
2.14.1
>From 81c07bc0008ebb24bab35e763c965f625d308e6b Mon Sep 17 00:00:00 2001
From: Carlos Maddela <e7ap...@gmail.com>
Date: Mon, 21 Aug 2017 23:46:29 +1000
Subject: [PATCH 2/2] Fix restoration of index files by hash.
Fix the condition for lazy construction of 'solidPathRel' and 'solidPathAbs'
to ensure that they are available when required.
---
source/cacheman.cc | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/source/cacheman.cc b/source/cacheman.cc
index 1f5f831..5dd617e 100644
--- a/source/cacheman.cc
+++ b/source/cacheman.cc
@@ -2382,7 +2382,7 @@ bool
cacheman::ProcessByHashReleaseFileRestoreFiles(cmstring& releasePathRel, cm
Cstat wantedState(wantedPathAbs);
string solidPathRel, solidPathAbs;
// lazy construction for the check below
- if(wantedState.st_size != entry.fpr.size)
+ if(!wantedState || wantedState.st_size != entry.fpr.size)
{
solidPathRel =
entry.sDirectory.substr(stripPrefix.size()) + "by-hash/" +
GetCsNameReleaseFile(entry.fpr.csType) + '/' + hexname;
@@ -2407,7 +2407,7 @@ bool
cacheman::ProcessByHashReleaseFileRestoreFiles(cmstring& releasePathRel, cm
if(!h.LoadFromFile(SABSPATH(solidPathRel) + ".head") ||
! h.h[header::XORIG])
{
if(m_bVerbose)
- SendFmt << "Couldn't read" <<
SABSPATH(solidPathRel) << ".head<br>";
+ SendFmt << "Couldn't read " <<
SABSPATH(solidPathRel) << ".head<br>";
return;
}
string origin(h.h[header::XORIG]);
--
2.14.1
