I build Debian Live images each month, and upload them to a remote site.
As you might expect, most of the files are the same.

Using a Debian 9 stack, this is what I observed this month:

  server images:

    Total file size: 116.29M bytes
    Total bytes sent: 68.51M

    Total file size: 224.01M bytes
    Total bytes sent: 149.05M

  desktop (xfce) image:

    Total file size: 487.74M bytes
    Total bytes sent: 151.98M

  desktop (xfce) image w/ lots of big game assets:

    Total file size: 1.76G bytes
    Total bytes sent: 137.67M

    Total file size: 1.77G bytes
    Total bytes sent: 134.26M

You can see that there seems to be an initial flagfall of ~100MB,
after which the large, unchanging game assets mostly "settle" and are
highly rsyncable, even after passing through mksquashfs -comp xz.


Note that:

  1. These numbers include Debian 9 initrd.img files with COMPRESS=xz.
     These are around 20MB, and are *NOT* rsyncable,
     i.e. rsync has to transmit the whole ramdisk every time.
     In Debian 10 buster, the #845034 fix in initramfs-tools 0.131+ fixes this.

  2. These numbers are from mksquashfs 4.3.
     In Debian 11 bullseye, mksquashfs 4.4 should improve
     "rsyncability" due to several changes related to reproducible builds.

Therefore we can reasonably expect to shave a good 40MB+ off that 100MB 
flagfall.

On this basis, I think an explicit --rsyncable is NOT NEEDED.
It is sufficient to simply upgrade to squashfs 4.4+ and initramfs-tools 0.131+.

It MIGHT further help to set SOURCE_DATE_EPOCH to the same time for both images
(I set mine to TZ=UTC date +%s -d '1 Jan', which is kinda naughty).

It MIGHT further help to use mksquashfs -sort to force a particular packing, but
my initial tests were not promising, so I abandoned this idea.

Reply via email to