Hi Joel
On Sun, 8 Jun 2025 at 00:08, Joel Ebel via Bug reports for the GNU Bourne
Again SHell <bug-bash@gnu.org> wrote:
> This is mostly an FYI, as I don't think our ridiculous environment needs to
> be supported, but the E2BIG test in exec3.sub never finishes in our testing
> environment due to ARG_MAX being set to a huge number. I'm adding a patch
> to limit the test to 2 MB ARG_MAX, though I don't know what the best value
> limit might be generally. I wanted to share it in case it was useful.
I'm guessing this is blocking your CI/CD pipeline?
Technically this test will finish, but you'd need to be *much* more patient.
The problem with using a small fake value for argmax is that the rest of
the test won't elicit E2BIG, which basically constitutes a test failure.
I would rather fix the test so that it still runs correctly, but in a
reasonable time.
As a baseline, running the current test with a 2 MiB limit over-fills the
environment by about 15%, and takes about 19 seconds to run on my machine.
If I run it with a 16 MiB limit, the overfill is negligible, and it takes
about 85 seconds to run on my machine.
If I were to run it with a 2 GiB limit, by linear extrapolation I would
expect it to take *at least 4 hours*, but possibly much longer, as there is
a quadratic element to the timing that is only measurable above about 500
MiB. (See below)
For a start, *just* lifting the range expansion out of the loop reduces the
time from 4 hours to 5 minutes.
However that's still a long wait, and we can do *much* better:
(a) avoid all range expansions and
(b) avoid a linear loop
The simplest (and fastest) would be more like this:
$ git diff tests
diff --git a/tests/exec3.sub b/tests/exec3.sub
index c8875bef4..01ced690f 100644
--- a/tests/exec3.sub
+++ b/tests/exec3.sub
@@ -62,10 +62,7 @@ argmax=$(getconf ARG_MAX 2>/dev/null)
if (( argmax <= 0 )); then
argmax=1048576
fi
-v=$(echo {1..250000})
-while (( ${#v} < $argmax )); do
- v+=$(echo {250001..350000})
-done
+printf -v v %.*u "$argmax" 0
export v
exec ${THIS_SH} </dev/null
EOF
$ #endpatch
(Running this with a 2 GiB limit takes 11.9 seconds on my machine, or at
least 1200 times faster than the current version.)
However there might be some objections to relying on the printf built-in,
since in theory Bash could be built without it, in which case I suggest:
$ git diff tests
diff --git a/tests/exec3.sub b/tests/exec3.sub
index c8875bef4..af43e09bf 100644
--- a/tests/exec3.sub
+++ b/tests/exec3.sub
@@ -62,9 +62,11 @@ argmax=$(getconf ARG_MAX 2>/dev/null)
if (( argmax <= 0 )); then
argmax=1048576
fi
-v=$(echo {1..250000})
-while (( ${#v} < $argmax )); do
- v+=$(echo {250001..350000})
+for (( m = n = argmax / ${#argmax} + 1 ; o = m & m-1 ; m = o )) do :; done
+v=
+for ((; m ; m >>= 1 )) do
+ v+=$v
+ (( n & m )) && v+=$argmax
done
export v
exec ${THIS_SH} </dev/null
$ #endpatch
(Running this with a 2 GiB limit takes 13.6 seconds on my machine, or about
14% slower than my printf version, but still more than 1000 times faster
than the current version. Running this with a 1 GiB limit takes 5.3 seconds
on my machine, so the quadratic timing is evident. Running it with a 2 MiB
limit takes 36 milliseconds on my machine.)
-Martin
