On Mon, Aug 17, 2009 at 06:20:20PM -0400, Daniel Richard G. wrote: > On Mon, 2009 Aug 17 21:35+0100, Mark Hindley wrote: > > > > Actually, there is one more bit that concerns me. That is the checksum > > code. Do you use checksumming? > > I don't use checksumming, but aren't the checksum databases built off > the index files, not the packages?
Yes, as a new index file is cached, it is parsed and the checksums stored in the database so that .debs can be verified against it later. > > At the moment there is a set of checksum data in a DB database for > > each distro. On my systems it uses about 25-30MB using the Concurrent > > Data Store. If we make this change, we are going to end up with the > > same amount of data for each hostname that is fetched through the > > cache. That could be huge! I simply don't know if the database > > throughput will scale the required amount. And the disk space usage > > could be really quite big. > > I'm not understanding how the checksum databases relate to the cached > package files (aside from storing package checksums), and why the new > package-filename scheme would lead to multiple databases being created. It isn't that there would be multiple databases, just that the size would/could increase dramatically. Currently each .deb has it's checksums in the database. About 30MB for the Debian distro. No hosts or paths are stored. Filenames are considered unique. If we change to the long naming scheme, when each index file is cached and parsed the contained checksums will have to be stored against the relevant .debs but with the hostname and path prepended. So we will end up with a complete set of checksums for each host that is cached in the server. > Is it off the mark to think of the checksum database(s) as a sort of > binary representation of the package indices? (I can't make much of the > code in apt-cacher-lib-cs.pl...) Yes, it just saves parsing the index files each time a cached file is served to find the checksum to verify which would be very slow. Mark -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
