debdelta , Re: effectiveness of rsync and apt

2006-05-20 Thread A Mennucc1
hi I recollected my earlier experiments on computing package diffs, and prepared a package called 'debdelta'. It is available at http://tonelli.sns.it/pub/mennucc1/debdelta and also uploaded into experimental. This package contains both a command 'debdelta', to compute deltas of Debian packag

Re: effectiveness of rsync and apt

2006-05-10 Thread A Mennucc
hi I had the same idea some time ago if you ever decide to work on that, I may help Goswin von Brederlow wrote: > I actualy have a little hack how one could implement patch debs now to > test this out: > > 1. Create an archive mirror with rsync batch files (or xdelta or > whatever) between the

Re: effectiveness of rsync and apt

2006-05-04 Thread Goswin von Brederlow
Marc Haber <[EMAIL PROTECTED]> writes: > On Mon, 01 May 2006 09:30:55 +0200, Florian Weimer <[EMAIL PROTECTED]> > wrote: >>The downside is that anything that doesn't work on entire .debs is >>very likely to change them at the byte stream level (you only need to >>use slightly different zlib versio

Re: effectiveness of rsync and apt

2006-05-04 Thread Marc Haber
On Mon, 01 May 2006 09:30:55 +0200, Florian Weimer <[EMAIL PROTECTED]> wrote: >The downside is that anything that doesn't work on entire .debs is >very likely to change them at the byte stream level (you only need to >use slightly different zlib versions or parameters). This means that >the chain

Re: effectiveness of rsync and apt

2006-05-02 Thread Andreas Barth
* Darren Salt ([EMAIL PROTECTED]) [060502 19:03]: > I demand that Andreas Barth may or may not have written... > > * Brian Eaton ([EMAIL PROTECTED]) [060501 19:21]: > >> On 5/1/06, Andreas Barth <[EMAIL PROTECTED]> wrote: > >>> Or you could create the diffdebs before upload or on ftp-master, and >

Re: effectiveness of rsync and apt

2006-05-02 Thread Darren Salt
I demand that Andreas Barth may or may not have written... > * Brian Eaton ([EMAIL PROTECTED]) [060501 19:21]: >> On 5/1/06, Andreas Barth <[EMAIL PROTECTED]> wrote: >>> Or you could create the diffdebs before upload or on ftp-master, and >>> include the diffdebs somehow in the Packages file (so t

Re: effectiveness of rsync and apt

2006-05-02 Thread Darren Salt
I demand that Pierre Habouzit may or may not have written... [snip; delta packages?] > The real question is: do people clean their apt cache or not? I do, because > after a full X.org/kde/openoffice upgrade, it takes quite a lot of disk in > /var (that is small on my computers). And with that cach

Re: effectiveness of rsync and apt

2006-05-02 Thread Goswin von Brederlow
Pierre Habouzit <[EMAIL PROTECTED]> writes: > Le Lun 1 Mai 2006 15:31, Brian Eaton a écrit : >> On 4/30/06, Goswin von Brederlow wrote: >> > Look at zsync and help develope it far enough so it can look into >> > debs. Without that the gain is practicaly 0 or less. >> >> It's entirely possible tha

Re: effectiveness of rsync and apt

2006-05-02 Thread Goswin von Brederlow
Peter Samuelson <[EMAIL PROTECTED]> writes: >> * Goswin von Brederlow: >> > Look at zsync and help develope it far enough so it can look into >> > debs. Without that the gain is practicaly 0 or less. > > The other thing to do would be to lobby for dpkg-deb and dpkg-source to > use 'gzip --rsyncabl

Re: effectiveness of rsync and apt

2006-05-02 Thread Goswin von Brederlow
Tyler MacDonald <[EMAIL PROTECTED]> writes: > Goswin von Brederlow <[EMAIL PROTECTED]> wrote: >> Bittorrent has a per chunk hash so it can validate each chunk when it >> recieves it instead of waiting for the full file. It won't see if a >> chunk is present at some other position in the file, not

Re: effectiveness of rsync and apt

2006-05-02 Thread A Mennucc
hi I did a similar thing some time ago; I used 'xdelta' on two versions of kernel and of tetex; the results were impressive; I could prepare a 'debdiff' that was < 10% (AFAICR) of the size, and that would recreate an exact copy of the new version of the package, given the previous version of the p

Re: effectiveness of rsync and apt

2006-05-01 Thread Andreas Barth
* Brian Eaton ([EMAIL PROTECTED]) [060501 19:21]: > On 5/1/06, Andreas Barth <[EMAIL PROTECTED]> wrote: > >Or you could create the diffdebs before upload or on ftp-master, and > >include the diffdebs somehow in the Packages file (so they're signed as > >well by the usual mechanismn). > My initial

Re: effectiveness of rsync and apt

2006-05-01 Thread Brian Eaton
On 5/1/06, Andreas Barth <[EMAIL PROTECTED]> wrote: Or you could create the diffdebs before upload or on ftp-master, and include the diffdebs somehow in the Packages file (so they're signed as well by the usual mechanismn). My initial view is that any delta package system that doesn't reproduce

Re: effectiveness of rsync and apt

2006-05-01 Thread Brian Eaton
On 5/1/06, Andreas Barth <[EMAIL PROTECTED]> wrote: * Brian Eaton ([EMAIL PROTECTED]) [060501 16:42]: > On 5/1/06, Andreas Barth <[EMAIL PROTECTED]> wrote: > > > >If one does it right, it might be enough if the original package is > >*installed*. And that happens quite often, e.g. even for securi

Re: effectiveness of rsync and apt

2006-05-01 Thread Andreas Barth
* Brian Eaton ([EMAIL PROTECTED]) [060501 17:49]: > On 5/1/06, Andreas Barth <[EMAIL PROTECTED]> wrote: > >* Brian Eaton ([EMAIL PROTECTED]) [060501 16:42]: > >> On 5/1/06, Andreas Barth <[EMAIL PROTECTED]> wrote: > >> > > >> >If one does it right, it might be enough if the original package is > >>

Re: effectiveness of rsync and apt

2006-05-01 Thread Andreas Barth
* Brian Eaton ([EMAIL PROTECTED]) [060501 16:42]: > On 5/1/06, Andreas Barth <[EMAIL PROTECTED]> wrote: > >* Brian Eaton ([EMAIL PROTECTED]) [060501 15:51]: > >> The only time delta packages will be a win is for upgrades where the > >> client has the original package cached. > > > >If one does it r

Re: effectiveness of rsync and apt

2006-05-01 Thread Pierre Habouzit
Le Lun 1 Mai 2006 16:35, Brian Eaton a écrit : > On 5/1/06, Andreas Barth <[EMAIL PROTECTED]> wrote: > > * Brian Eaton ([EMAIL PROTECTED]) [060501 15:51]: > > > The only time delta packages will be a win is for upgrades where > > > the client has the original package cached. > > > > If one does it

Re: effectiveness of rsync and apt

2006-05-01 Thread Brian Eaton
On 5/1/06, Andreas Barth <[EMAIL PROTECTED]> wrote: * Brian Eaton ([EMAIL PROTECTED]) [060501 15:51]: > The only time delta packages will be a win is for upgrades where the > client has the original package cached. If one does it right, it might be enough if the original package is *installed*.

Re: effectiveness of rsync and apt

2006-05-01 Thread Pierre Habouzit
Le Lun 1 Mai 2006 15:31, Brian Eaton a écrit : > On 4/30/06, Goswin von Brederlow wrote: > > Look at zsync and help develope it far enough so it can look into > > debs. Without that the gain is practicaly 0 or less. > > It's entirely possible that the gain will be nothing no matter what > algorithm

Re: effectiveness of rsync and apt

2006-05-01 Thread Andreas Barth
* Brian Eaton ([EMAIL PROTECTED]) [060501 15:51]: > The only time delta packages will be a win is for upgrades where the > client has the original package cached. If one does it right, it might be enough if the original package is *installed*. And that happens quite often, e.g. even for security

Re: effectiveness of rsync and apt

2006-05-01 Thread Brian Eaton
On 4/30/06, Goswin von Brederlow <[EMAIL PROTECTED]> wrote: > Brian Eaton <[EMAIL PROTECTED]> wrote: >> http://rsync.samba.org/rsync-and-debian/rsync-and-debian.html >> >> Has anyone ever done some log file analysis to figure out how much >> bandwidth would be saved by transferring package deltas

Re: effectiveness of rsync and apt

2006-05-01 Thread Martijn van Oosterhout
On 5/1/06, Brian Eaton <[EMAIL PROTECTED]> wrote: Hello all - Regarding the ideas discussed here: http://rsync.samba.org/rsync-and-debian/rsync-and-debian.html A few comments: 3.2 rsync is too hard on servers That document claims it's an avoidable cost. It's not really, because the client p

Re: effectiveness of rsync and apt

2006-05-01 Thread Peter Samuelson
> * Goswin von Brederlow: > > Look at zsync and help develope it far enough so it can look into > > debs. Without that the gain is practicaly 0 or less. The other thing to do would be to lobby for dpkg-deb and dpkg-source to use 'gzip --rsyncable' when building stuff. (That, or sneak "--rsyncabl

Re: effectiveness of rsync and apt

2006-05-01 Thread Florian Weimer
* Goswin von Brederlow: > Tyler MacDonald <[EMAIL PROTECTED]> writes: > >> Brian Eaton <[EMAIL PROTECTED]> wrote: >>> http://rsync.samba.org/rsync-and-debian/rsync-and-debian.html >>> >>> Has anyone ever done some log file analysis to figure out how much >>> bandwidth would be saved by transferri

Re: effectiveness of rsync and apt

2006-04-30 Thread Tyler MacDonald
Goswin von Brederlow <[EMAIL PROTECTED]> wrote: > Bittorrent has a per chunk hash so it can validate each chunk when it > recieves it instead of waiting for the full file. It won't see if a > chunk is present at some other position in the file, not even if that > position is also on chunk boundarie

Re: effectiveness of rsync and apt

2006-04-30 Thread Goswin von Brederlow
Tyler MacDonald <[EMAIL PROTECTED]> writes: > Brian Eaton <[EMAIL PROTECTED]> wrote: >> http://rsync.samba.org/rsync-and-debian/rsync-and-debian.html >> >> Has anyone ever done some log file analysis to figure out how much >> bandwidth would be saved by transferring package deltas instead of >> e

Re: effectiveness of rsync and apt

2006-04-30 Thread Tyler MacDonald
Brian Eaton <[EMAIL PROTECTED]> wrote: > http://rsync.samba.org/rsync-and-debian/rsync-and-debian.html > > Has anyone ever done some log file analysis to figure out how much > bandwidth would be saved by transferring package deltas instead of > entire new packages? Slightly off-topic, but

effectiveness of rsync and apt

2006-04-30 Thread Brian Eaton
Hello all - Regarding the ideas discussed here: http://rsync.samba.org/rsync-and-debian/rsync-and-debian.html Has anyone ever done some log file analysis to figure out how much bandwidth would be saved by transferring package deltas instead of entire new packages? Assuming someone hasn't done