On Sat, Jan 06, 2018 at 01:42:25AM +0200, Alfred Baroti wrote:
> Hi,
> I found this from long long time ago.
> Is this a serious bug?
[...]
This is not a serious bug at all. It's just a memory allocation failure.
> [root@n1x ~]#su nix
> nix@n1x:/root$ printf "%s\n"
> {{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}
> bash: xmalloc: .././braces.c:793: cannot allocate 7329062664 bytes
> (614682624 bytes allocated)
You are performing a brace expansion which generates 62^5 combinations and
requires bash to allocate 7329062664 bytes (6.83 GiB) from the operating
system. The allocation fails at 614682624 bytes (586 MiB).
Once bash is unable to allocate memory, it will crash (xmalloc is a wrapper
around malloc which terminates execution on allocation errors).
Looking at a program execution trace for bash 4.4.12(1)-release (as distributed
by Ubuntu):
dualbus@ubuntu:~$ LC_ALL=C strace -fo /dev/stdout bash -c ':
{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}{{a..z},{A..Z},{0..9}}'
| cat -n | tail -n6
bash: xmalloc: .././braces.c:793: cannot allocate 7329062664 bytes
(613113856 bytes allocated)
116508 28672 brk(0x254ad000) = 0x254ad000
116509 28672 brk(0x254ae000) = 0x254ae000
116510 28672 brk(0x254af000) = 0x254af000
116511 28672 write(2, "bash: xmalloc: .././braces.c:793"..., 95) = 95
116512 28672 exit_group(2) = ?
116513 28672 +++ exited with 2 +++
You can see that the brk() system call actually succeeds.
So the allocation fails, even if the system has enough memory to handle it.
Upon closer inspection:
dualbus@ubuntu:~/src/gnu/bash/lib/malloc$ cat -n malloc.c | sed -n
'777,781p'
777 /* Silently reject too-large requests. */
778 if (nunits >= NBUCKETS) {
779 abort();
780 return ((PTR_T) NULL);
781 }
The internal_malloc function that bash uses on some systems (instead of the
libc provided malloc), including Linux, has a special provision that rejects
memory requests that are too large (NBUCKETS = 30).
So you have two options:
(1) Compile bash by providing `--without-bash-malloc' to the `configure' script
(2) Find another option to generate all these permutations that doesn't require
requesting >580 MiB at once.
Question for Chet: Is there a reason we use bash-malloc in Linux systems by
default, instead of the typical glibc malloc which is available on most of
them?
