Re: Change in indirect expansion behavior from 4.4 to 5.0
On Tue, 29 Sep 2020 21:28:59 -0700, Jason Miller wrote:
> Gentoo linux, GNU bash, version 5.0.18(1)-release (x86_64-pc-linux-gnu)
>
> On the above vesion of bash, the following script will not run the echo
> command and print an error. On bash 4.4 it appears to treat the ${!foo}
> the same as expanding an unset variable and thus outputs "bar":
>
> unset foo
> echo ${!foo} bar
>
> "shopt -s compat44" does not seem to make a difference
This is documented, and thus (I suppose) on purpose. See the file CHANGES:
This document details the changes between this version, bash-5.0-alpha, and
the previous version, bash-4.4-release.
...
y. If indirect expansion attempts to indirectly reference through an unset
variable, report an error.
--
D.
Re: Change in indirect expansion behavior from 4.4 to 5.0
On 9/30/20 12:28 AM, Jason Miller wrote:
> Gentoo linux, GNU bash, version 5.0.18(1)-release (x86_64-pc-linux-gnu)
>
> On the above vesion of bash, the following script will not run the echo
> command
> and print an error. On bash 4.4 it appears to treat the ${!foo} the same as
> expanding an unset variable and thus outputs "bar":
>
> unset foo
> echo ${!foo} bar
This is the result of
https://lists.gnu.org/archive/html/bug-bash/2016-11/msg00123.html
I explained the reasoning in
https://lists.gnu.org/archive/html/bug-bash/2016-11/msg00165.html
The basic idea is that indirect expansion is just a string substitution,
so indirecting an unset variable is the logically same thing as ${},
which is an expansion error.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
execve E2BIG (Argument list too long)
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -O2 -g -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions
-fstack-protector-strong -grecord-gcc-switches
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
-Wno-parentheses -Wno-format-security
uname output: Linux h001201355 5.7.10-201.fc32.x86_64 #1 SMP Thu Jul
23 00:58:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-redhat-linux-gnu
Bash Version: 5.0
Patch Level: 17
Release Status: release
Description:
The included short script when run with the following command results
in execve "E2BIG (Argument list too long) errors".
Running "strace -f" on this shows the error, and "Argument list too
long" is printed out to stderr for the calls to external commands.
seq 1 4 | xargs ./bash_bug
BEGIN: bash_bug
--
#!/usr/bin/env bash
. "exists_or_not"
export A123456789012345="${BASH_ARGV[*]}"
mkdir -p /tmp/foobar
perl -E 'say "hello world"' 'foo' 'bar'
--
END bash_bug
"xargs" is only used for the sake of being concise. The issue occurs
with an equivalently long list of arguments.
A number of the parameters can be tuned to make the problem occur or not.
* The number of arguments can be tuned down to "seq 1 23694" and it
still occurs, but any lower and it disappears.
* The sourcing of a file appears to be required, but it doesn't matter
what that file is or even if it exists.
* The variable name length of the exported variable must be 16 or more
characters, reducing this to 15 or less makes the bug disappear. It
also doesn't matter what characters are, any valid variable name of 16
or more characters will do.
* The path used for calling the script seems to affect the issue. In
my tests using the absolute script path sometimes made the bug
disappear, but using a local relative path seemed to reliably produce
it.
This issue was tested on the reporting system and the oldest system I
could find and was present on both systems
* Fedora release 32 (Thirty Two) 5.7.10-201.fc32.x86_64
* Red Hat Enterprise Linux Server release 6.10 (Santiago)
2.6.32-754.3.5.el6.x86_64
Repeat-By:
Running provided script with the given command.
Re: execve E2BIG (Argument list too long)
On Wed, Sep 30, 2020 at 5:53 PM Michael Green wrote: > The included short script when run with the following command results > in execve "E2BIG (Argument list too long) errors". > > * The number of arguments can be tuned down to "seq 1 23694" and it > still occurs, but any lower and it disappears. > That sounds a lot like the 128 kB hard limit Linux puts on the size of a single argument to exec. (I know it counts for command line arguments, I expect it also counts for env variables. They're passed in pretty much the same way anyway.) seq 1 23694 | wc gives 131058, just a bit less than 131072. Add the variable name and it goes over. Workaround: use another OS, or pass big data like that in files.
Re: execve E2BIG (Argument list too long)
On Wed, Sep 30, 2020 at 09:12:30AM +0200, Michael Green wrote:
> seq 1 4 | xargs ./bash_bug
>
> BEGIN: bash_bug
> --
> #!/usr/bin/env bash
> . "exists_or_not"
> export A123456789012345="${BASH_ARGV[*]}"
> mkdir -p /tmp/foobar
> perl -E 'say "hello world"' 'foo' 'bar'
> --
> END bash_bug
Why do you consider this to be a bug in bash? You've asked your operating
system to run a command (mkdir or perl) with an environment that exceeds
the size the kernel will permit. The kernel gives you the error.
Re: execve E2BIG (Argument list too long)
On 9/30/20 3:12 AM, Michael Green wrote: > Bash Version: 5.0 > Patch Level: 17 > Release Status: release > > Description: > > The included short script when run with the following command results > in execve "E2BIG (Argument list too long) errors". The argument list includes the size of the environment, and your script creates a huge environment variable. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: execve E2BIG (Argument list too long)
On September 30, 2020 8:24:01 AM Ilkka Virta wrote: On Wed, Sep 30, 2020 at 5:53 PM Michael Green wrote: The included short script when run with the following command results in execve "E2BIG (Argument list too long) errors". * The number of arguments can be tuned down to "seq 1 23694" and it still occurs, but any lower and it disappears. That sounds a lot like the 128 kB hard limit Linux puts on the size of a single argument to exec. (I know it counts for command line arguments, I expect it also counts for env variables. They're passed in pretty much the same way anyway.) It might be worth asking lkml to lift this restriction seq 1 23694 | wc gives 131058, just a bit less than 131072. Add the variable name and it goes over. Workaround: use another OS, or pass big data like that in files.
Re: execve E2BIG (Argument list too long)
On Sep 30 2020, Daniel Colascione wrote: > It might be worth asking lkml to lift this restriction You will have bad luck with that. The limit has been introduced for security reasons. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Re: Change in indirect expansion behavior from 4.4 to 5.0
On Wed, 30 Sep 2020 09:19:34 -0400 Chet Ramey wrote: ... > I explained the reasoning in > > https://lists.gnu.org/archive/html/bug-bash/2016-11/msg00165.html ... The reasoning is 100% sound, but would it make sense to add the old behavior to a compat option? I'm not at all familiar with the decision process for those.
