Hi Laszlo, > I'm playing with the different patched versions for hours now. > Indeed, it seems with your latest patch the memory leak is bearable.
Nice to hear. I compressed a 8GB tree of toolchains and rootfs with valgrind (took > 10h ;) without any noticable leaks. > I've never tested CPU usage before. I've realized it's rhapsodical, > sometimes all my CPU cores utilized on 100%, sometimes one on ~15%. > Should I wait for other fix(es)? No. I'ven't looked into it. But this is a long standing issue. First of all, as squashfs-tools wasn't written in the last years, when reproducible builds became more famous. So it's not written with reproducible building in mind. For me is reproducible builds more important than using all cpu cores. But I don't use it with gigabytes images. It got a bit worse with removing the frag_deflator thread, as there is one workload thread less. The old frag_deflator thread has been spawnd $cpus times. fragments are small files. The cpu usage will also depend on the compression algorithm. If you like to profile mksquashfs-tools, I'm happy to help to understand the code base. Maybe there is a different approach to do it reproducible. Create first an index over all files, ensure a proper order through multiple queues via an index. But I'm not sure, if this would be really faster than it's now. Btw: even with `-processors 1` it will spawn multiple threads which should use more than 1 core. Best, lynxis
pgpZwUTI6Rfzp.pgp
Description: OpenPGP digital signature
