bash-4.2(42) on AIX 6.1 has broken command expansion in double-quoted strings
Hi bash gurus - bash appears to be broken on AIX 6.1 - I'd really appreciate some advice : With bash-4.2 patchlevel 42 (the latest available as of today from ftp.gnu.org ), on AIX, the resultant bash built with gcc-4.7.2 and system ld + as, ends up being broken wrt to command output in strings: $ ./bash bash-4.2$ echo "$(echo hello)" bash: command substitution: line 2: syntax error near unexpected token `)' bash: command substitution: line 2: `echo hello)"' bash-4.2$ v="$(date)" bash: command substitution: line 13: syntax error near unexpected token `)' bash: command substitution: line 13: `date)"' I've tried compiling with / without libiconv and either to use AIX /usr/lib/libcurses or latest libncurses (from invisible-island.net), with GCC - 4.7.2 and the latest version of m4 + bison and/or byacc, and with / without all combinations of the '--disable-nls --without-libiconv-prefix --enable-readline --with-installed-readline' configuration options, but with same result. Also, the stock IBM "/opt/freeware" bash 4.1 comes with a bug that disables -completion when the path resolves to an NFS mount point . Does anyone know if there is a way to disable this ? Thanks in advance for any response - Best Regards, Jason Vas Dias
Re: More fun with IFS
Dan Douglas dixit:
>of any reason it should be inserting a '':'' between the two arguments,
>especially for the ''$@'' variants, either quoted or unquoted. It certainly
>can't be because of a word splitting step.
‘:’ is ${IFS::1} and inserted because of the word *concatenation*
(not splitting) that occurs when assigning a vector to a scalar.
>Most script writers treat assignments as identical whether quoted or not.
We had a discussion about that on the Austin ML (when I still
managed to somewhat follow it) and specifically changed it so
shells MUST distinguish between unquoted and quoted $*/$@.
> $ mksh -c 'set one:::two three:::four; IFS=:; cat <<<$@'
>one:::two:three:::four
> $ mksh -c 'set one:::two three:::four; IFS=:; cat <<<"$@"'
>one:::two:three:::four
I think that is completely reasonable and correct here.
>I tend to think AT&T ksh is doing the most reasonable thing here by making the
>concatenated result exactly the same as if expanded as arguments in a quoted
>context, with whitespace separating them.
Why whitespace? $IFS certainly contains none. And the usual
insertion rules all specify the first character of $IFS and
specify what to do if $IFS is empty or unset (which it isn’t
in these examples).
>> In other words, “don’t do that then” (rely on this behaviour).
>
>I wouldn't bother with this language if the only non-random behavior was that
>specified by POSIX. "POSIX doesn't specify it" is a horrible reason to do
>anything.
Right, “POSIX says so” oftentimes also is, after all…
which is why I tried above to argue without it.
>> I think eval is evil anyway ;-)
>
>Eval is frowned upon because it almost always misused. Until shells add first-
>class closures and higher-order functions I'll continue using it.
Yeah, of course, it’s the only way to do some things… I personally
usually abstract everything eval into little functions of their
own and then just use those.
bye,
//mirabilos
--
„nein: BerliOS und Sourceforge sind Plattformen für Projekte, github ist
eine Plattform für Einzelkämpfer“
-- dieses Zitat ist ein Beweis dafür, daß auch ein blindes Huhn
mal ein Korn findet, bzw. – in diesem Fall – Recht haben kann
export in posix mode
I certainly could be doing something wrong, but it looks to me like bash - when in Posix mode - does not suppress the "-n" option for export. The version of bash that I'm looking at is 3.2.25. -jrm
Re: export in posix mode
On 02/27/2013 11:07 AM, James Mason wrote: > I certainly could be doing something wrong, but it looks to me like bash > - when in Posix mode - does not suppress the "-n" option for export. > The version of bash that I'm looking at is 3.2.25. So what? Putting bash in posix mode does not require bash to instantly prohibit extensions. POSIX intentionally allows for implementations to provide extensions, and 'export -n' is one of bash's extensions. There's no bug here, since leaving the extension always enabled does not conflict with subset of behavior required by POSIX. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: bash-4.2(42) on AIX 6.1 has broken command expansion in double-quoted strings
On 2/27/13 1:26 PM, Jason Vas Dias wrote: > Hi bash gurus - bash appears to be broken on AIX 6.1 - I'd really > appreciate some advice : > > With bash-4.2 patchlevel 42 (the latest available as of today from > ftp.gnu.org ), > on AIX, the resultant bash built with gcc-4.7.2 and system ld + as, > ends up being > broken wrt to command output in strings: > $ ./bash > bash-4.2$ echo "$(echo hello)" > bash: command substitution: line 2: syntax error near unexpected token `)' > bash: command substitution: line 2: `echo hello)"' > bash-4.2$ v="$(date)" > bash: command substitution: line 13: syntax error near unexpected token `)' > bash: command substitution: line 13: `date)"' I can't reproduce this, but ISTR something similar when the parser was built with byacc. I think a sufficiently modern version of bison should work. Make sure you delete builtins/evalstring.o and builtins/libbuiltins.a after rebuilding y.tab.[ch]. The dependency there is tricky, and it's best just to do it explicitly. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: export in posix mode
Eric Blake wrote: > James Mason wrote: > > I certainly could be doing something wrong, but it looks to me like bash > > - when in Posix mode - does not suppress the "-n" option for export. > > The version of bash that I'm looking at is 3.2.25. > > So what? Putting bash in posix mode does not require bash to instantly > prohibit extensions. POSIX intentionally allows for implementations to > provide extensions, and 'export -n' is one of bash's extensions. > There's no bug here, since leaving the extension always enabled does not > conflict with subset of behavior required by POSIX. If you are looking to try to detect non-portable constructs then you will probably need to test against various shells including ash. (If on Debian then use dash.) https://en.wikipedia.org/wiki/Almquist_shell The posh shell was constructed specifically to be as strictly conforming to posix as possible. (Making it somewhat less than useful in Real Life but it may be what you are looking for.) It is Debian specific in origin but should work on other systems. http://packages.debian.org/sid/posh http://anonscm.debian.org/gitweb/?p=users/clint/posh.git;a=summary Bob
Re: export in posix mode
On 02/27/2013 04:00 PM, Bob Proulx wrote: Eric Blake wrote: James Mason wrote: I certainly could be doing something wrong, but it looks to me like bash - when in Posix mode - does not suppress the "-n" option for export. The version of bash that I'm looking at is 3.2.25. So what? Putting bash in posix mode does not require bash to instantly prohibit extensions. POSIX intentionally allows for implementations to provide extensions, and 'export -n' is one of bash's extensions. There's no bug here, since leaving the extension always enabled does not conflict with subset of behavior required by POSIX. If you are looking to try to detect non-portable constructs then you will probably need to test against various shells including ash. (If on Debian then use dash.) https://en.wikipedia.org/wiki/Almquist_shell The posh shell was constructed specifically to be as strictly conforming to posix as possible. (Making it somewhat less than useful in Real Life but it may be what you are looking for.) It is Debian specific in origin but should work on other systems. http://packages.debian.org/sid/posh http://anonscm.debian.org/gitweb/?p=users/clint/posh.git;a=summary Bob We considered setting up another shell as the implementation of "/bin/sh", but that's hazardous in the context of vast amounts of boot-time initialization scripting that hasn't been vetted as to avoidance of bash-isms. Changing product script code - just so you can look for these sorts of things - isn't practical (or safe) either. So I guess if you take the view that bash POSIX mode exists only to make bash accept POSIX scripts, and not to preclude/warn about behavior that isn't going to be acceptable elsewhere, then you're right - it's not a bug. If you care about helping people to be able to write scripts that work various places and don't exceed the POSIX specification, you're unhelpfully wrong (and you might contemplate why "bashisms" gives > 50K google hits). -jrm
Re: export in posix mode
On 2/27/13 4:00 PM, Bob Proulx wrote: > The posh shell was constructed specifically to be as strictly > conforming to posix as possible. (Making it somewhat less than useful > in Real Life but it may be what you are looking for.) It is Debian > specific in origin but should work on other systems. Keep in mind that posh is overly strict in some areas (e.g., it throws an error on `exit 1'). It may not be useful in isolation. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: export in posix mode
Am 27.02.2013 22:39, schrieb James Mason: > On 02/27/2013 04:00 PM, Bob Proulx wrote: >> Eric Blake wrote: >>> James Mason wrote: I certainly could be doing something wrong, but it looks to me like bash - when in Posix mode - does not suppress the "-n" option for export. The version of bash that I'm looking at is 3.2.25. >>> So what? Putting bash in posix mode does not require bash to instantly >>> prohibit extensions. POSIX intentionally allows for implementations to >>> provide extensions, and 'export -n' is one of bash's extensions. >>> There's no bug here, since leaving the extension always enabled does >>> not >>> conflict with subset of behavior required by POSIX. >> If you are looking to try to detect non-portable constructs then you >> will probably need to test against various shells including ash. (If >> on Debian then use dash.) >> >>https://en.wikipedia.org/wiki/Almquist_shell >> >> The posh shell was constructed specifically to be as strictly >> conforming to posix as possible. (Making it somewhat less than useful >> in Real Life but it may be what you are looking for.) It is Debian >> specific in origin but should work on other systems. >> >>http://packages.debian.org/sid/posh >>http://anonscm.debian.org/gitweb/?p=users/clint/posh.git;a=summary >> >> Bob > > We considered setting up another shell as the implementation of > "/bin/sh", but that's hazardous in the context of vast amounts of > boot-time initialization scripting that hasn't been vetted as to > avoidance of bash-isms. > > Changing product script code - just so you can look for these sorts of > things - isn't practical (or safe) either. > > So I guess if you take the view that bash POSIX mode exists only to > make bash accept POSIX scripts, and not to preclude/warn about > behavior that isn't going to be acceptable elsewhere, then you're > right - it's not a bug. If you care about helping people to be able > to write scripts that work various places and don't exceed the POSIX > specification, you're unhelpfully wrong (and you might contemplate why > "bashisms" gives > 50K google hits). > > -jrm > > > bash posix more just changes bash behaviour that is incompatible with the posix spec. Nothing more or less. There are other shells for doing what you seem to want as has already been stated. Namely dash and posh.
Re: export in posix mode
On 2/27/13 4:52 PM, John Kearney wrote: > bash posix more just changes bash behaviour that is incompatible with > the posix spec. Nothing more or less. Correct. Limiting behavior to only what Posix specifies or warning about extensions beyond the spec was not in scope. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: export in posix mode
Chet Ramey wrote: > Keep in mind that posh is overly strict in some areas (e.g., it throws > an error on `exit 1'). It may not be useful in isolation. As I did mention I have found that posh is somewhat less than useful in Real Life. But you say it throws an error on exit 1? $ cat >/tmp/trial <<'EOF' #!/bin/posh echo "Hello from posh" exit 1 EOF $ chmod a+x /tmp/trial $ /tmp/trial Hello from posh $ echo $? 1 I see no error when using 'exit 1'. Other than the expected exit code. What am I missing? Bob
Re: export in posix mode
On 2/27/13 5:17 PM, Bob Proulx wrote:
> Chet Ramey wrote:
>> Keep in mind that posh is overly strict in some areas (e.g., it throws
>> an error on `exit 1'). It may not be useful in isolation.
>
> As I did mention I have found that posh is somewhat less than useful
> in Real Life. But you say it throws an error on exit 1?
>
> $ cat >/tmp/trial <<'EOF'
> #!/bin/posh
> echo "Hello from posh"
> exit 1
> EOF
> $ chmod a+x /tmp/trial
> $ /tmp/trial
> Hello from posh
> $ echo $?
> 1
>
> I see no error when using 'exit 1'. Other than the expected exit
> code. What am I missing?
I don't know what version you're using; I have 0.11.
$ ./posh
\[\]${HOST}($SHLVL)\$ exit 1
./posh: exit: bad number
$ ./posh
\[\]${HOST}($SHLVL)\$ exit 10
./posh: exit: bad number
$ echo $?
1
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: export in posix mode
On 02/27/2013 03:22 PM, Chet Ramey wrote:
> I don't know what version you're using; I have 0.11.
>
> $ ./posh
> \[\]${HOST}($SHLVL)\$ exit 1
> ./posh: exit: bad number
>
> $ ./posh
> \[\]${HOST}($SHLVL)\$ exit 10
> ./posh: exit: bad number
> $ echo $?
> 1
I tested with posh 0.10.2; looks like this is an upstream regression in
behavior (although you'd have to ask posh developers whether the
regression is intentional):
$ posh -c 'exit 10'; echo $?
10
--
Eric Blake eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org
signature.asc
Description: OpenPGP digital signature
Re: export in posix mode
James Mason wrote: > We considered setting up another shell as the implementation of > "/bin/sh", but that's hazardous in the context of vast amounts of > boot-time initialization scripting that hasn't been vetted as to > avoidance of bash-isms. You appear to be doing product QA. Awesome! Have you considered setting up a chroot system or a VM in different configurations and running continuous integration testing upon them? Within limits of kernel compatibility chroots are very lightweight. I am a fan due to the simplicity. Or VMs are quite accessible these days for the full system including kernel now that most recent cpus support full virtualization. > Changing product script code - just so you can look for these sorts > of things - isn't practical (or safe) either. Things generally are not that bad. I have tried this type of thing on occasion and usually while there are a few errors it isn't anything bad enough to prevent the system from booting to a useful state. And if it is a test VM without a lot of non-essential stuff then the core functionality is usually fine. > So I guess if you take the view that bash POSIX mode exists only to > make bash accept POSIX scripts, and not to preclude/warn about > behavior that isn't going to be acceptable elsewhere, then you're > right - it's not a bug. If you care about helping people to be > able to write scripts that work various places and don't exceed the > POSIX specification, you're unhelpfully wrong (and you might > contemplate why "bashisms" gives > 50K google hits). It is not a bash issue anymore than it is an issue specific to csh, ksh, pdksh, zsh or any other specific featureful shell. If it is a bug in one then it is a bug in all. While the bash project does not exist for the purpose you describe this does not prevent other projects such as dash and posh from existing and being useful. Whether the operating system as an entity decides to use bash or dash or other as a /bin/sh is a decision for them to make for themselves. If your OS does something different then I say take the issue up with your operating system. Debian for example uses /bin/sh symlinked to dash for just that reason. Works great. I don't see any problem. Scripts that wish to use bash features specify #!/bin/bash as the interpreter. Scripts that specify #!/bin/sh and use bash specific features will fail. As a local option the local admin may change the /bin/sh symlink from dash to bash and run scripts from other systems that include bash features. But that isn't by default. However it is a useful option for people stuck needing to run code that isn't cleanly written. Bob
Re: export in posix mode
Chet Ramey wrote:
> I don't know what version you're using; I have 0.11.
>
> $ ./posh
> \[\]${HOST}($SHLVL)\$ exit 1
> ./posh: exit: bad number
>
> $ ./posh
> \[\]${HOST}($SHLVL)\$ exit 10
> ./posh: exit: bad number
> $ echo $?
> 1
I am using 0.11 too. I was using the Debian packaged version. Since
this originated with Debian it is packaged using "native" packaging
where the tar.gz file is used without any patches to an upstream since
upstream is Debian. That means 0.11 without a 0.11-number -number on
the end.
I just now pulled the source code and did a build and the locally
compiled copy worked fine.
I can only assume this is some type of portability bug in the sources
compiled on your system. I can't imagine that many people compile
that program on other systems. It can't have gotten that much
exposure.
Bob
Re: Should this be this way?
Chet Ramey wrote: > > Your vendor, which may be SuSE, has changed bash and shipped the modified > version. Sorry for the bother. I reported the problem in the proper venue... (thanks)..
Re: Should this be this way?
Greg Wooledge wrote: > > How often, when at a terminal, do you type #!/bin/bash before every line? > > When I've put the contents into a file? Every. single. time. --- Then when I press 'v' to edit the command line in a text editor -- maybe 'bash' should insert such a line? It's converted your command line into an editable file. But it hasn't put the #!/bin/bash at the front. If that file is saved and run, it doesn't execute the same way as it does interactively. (note, I say the above someone tongue-in-cheek, but only to point out that it's very easy to go from editing a command to a file and if it becomes a file and 'bash' might not execute it the same way w/o the header, maybe it should put a headerline in? ;^/ ... (maybe distro's shouldn't muck w/user's shells...)...
Re: Should this be this way?
On 2013-02-27 20:05, Linda Walsh wrote: > (maybe distro's shouldn't muck w/user's shells...)... You're using SuSE, which is basically guaranteeing that your programs are mangled in unexpected ways. Then again, it's as bad of an offender as Debian, and most other distributions, I guess... Chris
