Hi, On 22/09/17 07:36, Andreas Tille wrote: > control: tags -1 help > > Hi Graham, > > since I have no idea how to deal with this I'm forwarding the issue > to debian-mips list asking desperately for help. [...] >> /*stdin*\ : 9.40% (6000000000 => 563851359 bytes, /*stdout*\) >> roundTripTest: ./datagen -g1500000000 -P97 | ../programs/zstd -v1 >> -T999 | ../programs/zstd -d >> >> *** zstd command line interface 32-bits v1.3.1, by Yann Collet *** >> zstd: error 11 : Allocation error : not enough memory
I think the problem is using too many threads with big stacks. When you use "zstd -T999", zstd will try to create 256 threads (the maximum amount). Since the default stack size is 8M, this will try to allocate 2G of virtual memory. On most 32-bit architectures which have 3G address space boundaries, there is just enough memory to do this. On mips and mipsel, the boundary is at 2G so there will not be enough space to allocate all these stacks along with any existing memory. To fix this you could: Use "ulimit -s 4096" to half the default stack size. This is only a workaround for the Debian packaging though - you can't expect users to do this. Reduce the value of ZSTDMT_NBTHREADS_MAX. In my testing, using 253 threads worked so you may not need to reduce it much (but maybe to a more rounder number). You could limit it to mips if you want, but please only limit it to 32-bit mips: #if defined(__mips__) && !defined(__mips64) Thanks, James
signature.asc
Description: OpenPGP digital signature
