Hello all, TL;DR: casync is an interesting piece of technology, but doesn't work for our current test images.
As I just merged another fedora-26 test image refresh, I took some time to evaluate casync [1]. On my laptop I still have the old image (c75882fd), and on my colo server (which has a really fast internet connection) I downloaded the new image (554546f5). The main tradeoff is the chunk size: smaller sizes increase the likelihood of a particular chunk already being present locally, but increase the number chunks that need to be downloaded. As this usually happens through HTTP, lots of small files kill performance completely (mostly due to the usual TCP slow start behaviour, but also due to the extra effort of making the connection itself). The reference time on my VDSL-25: $ time bots/image-download fedora-26 real 9m52.313s This is still bearable, but it would be even more interesting to improve image uploads (they take about 45 mins here). First I created a casync index from the fedora 26 qcow image on my server, with the the default option (64 kB chunk size): server$ cd /path/to/testdir server$ casync make fedora-26.caibx images/fedora-26-*.qcow2 Now on my laptop I use casync to get the new image, using as many chunks from the previous one as a basis, plus the new chunks from my server: $ casync extract --seed images/fedora-26-c75882*.qcow2 http://piware.de/tmp/cockpit-images-casync/fedora-26.caibx images/fedora-26-554546f5c12141c3bc2e92ba53564ad490f6974aea5f7df831ebcab563bf1b62.qcow2 This has to download thousands of chunks, each using one HTTP request: It took slightly over 22 minutes. Oddly enough this didn't save the downloaded chunks anywhere, so I'm unable to tell whether this actually saved any bandwidth. Next I changed the average chunk size to 8 MB, and produced a new chunk store: server$ casync --store 8m.castr --chunk-size=8M make fedora-26-8mb.caibx images/fedora-26-*.qcow2 Trying to download it fails: $ casync -vv --store http://piware.de/tmp/cockpit-images-casync/8m.castr --seed images/fedora-26-c75882fdee8847229eb89707d41b4488286605fc9f150811a5d950b8a1399825.qcow2 extract http://piware.de/tmp/cockpit-images-casync/fedora-26-8mb.caibx images/fedora-26-554546f5c12141c3bc2e92ba53564ad490f6974aea5f7df831ebcab563bf1b62.qcow2 [...] Chunk too large Failed to acquire http://piware.de/tmp/cockpit-images-casync/8m.castr/6265/626515b391d445efc75b8d9b0ec1e5ffdc93e13444e6b1e10b4309d0a488e538.xz Failed to run synchronizer: Broken pipe OK, fair enough. Next attempt with 1MB worked, but took pretty much exactly the same time as bots/image-download. Apparently there wasn't any actual saving, the qcow images seem to differ too much in between image rebuilds. Cross-checking with rsync over ssh (against the old image): | 1,515,291,648 100% 2.68MB/s 0:09:00 (xfr#1, to-chk=0/1) | | sent 282,439 bytes received 1,479,233,072 bytes 2,712,219.09 bytes/sec | total size is 1,515,291,648 speedup is 1.02 Maybe there is some trick ("sort" the blocks on the file system or so) to make qcow files rsyncable better. But in conclusion, right now this delta approach doesn't buy us much. Martin [1] http://0pointer.net/blog/casync-a-tool-for-distributing-file-system-images.html _______________________________________________ cockpit-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
