Hello,
today while playing around with brace expansion, I ran into something
for me unexpected. The actual intension was to compare the speed /
system usage of `seq x y' and brace expansion {x..y}. So I took a fairly
large integer (lets say 15755500), and ran the following:
seq 0 15755500
- no problem
printf "%s\n" {0..15755500}
(or for i in {0..15755500}; do echo $i; done)
- cpu and mem usage goes up, system starts swaping hardly, takes a while
- bang - bash process killed - out of memory.
System is Debian etch.
GNU bash, version 3.1.17(1)-release (i486-pc-linux-gnu)
600MHz Celeron, 392MB RAM, 512MB swap
I start thinking about it:
ok, building a 'list', which is too big for my system, may always lead
to that. So I try:
for i in $(seq 0 15755500); do echo $i; done
-bash: xrealloc: ../bash/subst.c:512: cannot reallocate 182209024 bytes
(0 bytes allocated)
ok, thesis looks confirmed...
I'm no C programmer, but I try to think logically about it.
There may be no way around it, as it may be necessary to build the
complete list, before it's possible to work with it.
If this assumption is correct, as building the list needs to be done in
memory -> bang comes sooner or later, depending on the system.
Am I thinking right?
Okay, assuming I'm somehow right, taking that subject to my 'real life
scripting', how should I deal with that?
There may be circumstances, where a 'list' becomes fairly large, and if
the system is for what ever reason low on resources, the process death
is predestined.
I haven't tested any other bash versions, nor other shells (would need
to install).
But, I just guess, it will be the same there?
How do experienced bash (shell) programmers deal with such cases?
Are there any guidelines (I may have missed reading) for 'careful'
scripting, to avoid such problems?
Thank you for any information!
Greets
Mart
