I noticed after a recent change that `make src.txz' is very slow. It turns out that `bsdtar -J' uses only *one* CPU, and the thread option has no effect. Either multithreaded xz is not enabled in bsdtar, or the xz block size is too small.
How to repeat: # as a test directory use /usr/bin - ca. 21MByte cd /; tar cf - usr/bin | wc -c 21585920 # bsdtar -J: only one CPU used time -p tar -J --options xz:threads=0 -cf - usr/bin | wc -c 5765120 real 9.38 user 9.28 sys 0.08 # tar | xz: only one CPU used time -p tar -cf - usr/bin | xz -T0 | wc -c 5759376 real 10.22 user 9.77 sys 0.54 # tar | xz + block size 1MB, seven times faster time -p tar -cf - usr/bin | xz -T0 --block-size=1MiB| wc -c 6372068 real 1.55 user 10.93 sys 13.01 # now running with 2*21MB, tar | xz => two CPUs are used time -p tar -cf - usr/bin usr/bin | xz -T0 | wc -c 11515212 real 11.97 user 20.13 sys 0.37 I guess this is due the default xz block size of ca. 24MB on FreeBSD (3 × 8 MB dict size = 24 MB) Any thoughts on how to use more than one CPU with tar -J? PS: on debian bsdtar -J works fine. Strange. PPS: found this very old bug report: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233543 In lib/libarchive/Makefile.inc there is no setup for multithreaded xz compression (-DHAVE_LZMA_STREAM_ENCODER_MT=1). If I add it, bsdtar uses more than one CPU for data > 20MB. Is there still a reason not to enable multithreaded xz compression in libarchive? -Wolfram -- Wolfram Schneider <[email protected]> https://wolfram.schneider.org
