$OPTIND varibale value is different from sh
if $OPTIND value start from 1 then i think "-b" $OPTIND should be "3" like sh but bash print "2" set -- -a -bc hello world echo $OPTIND# sh, bash = 1 getopts abc opt "$@" echo $opt, $OPTIND # sh, bash = a, 2 getopts abc opt "$@" # bash = b, 2 <-- different from "sh" echo $opt, $OPTIND# sh = b, 3 getopts abc opt "$@" # sh, bash = c, 3 echo $opt, $OPTIND
Re: $OPTIND varibale value is different from sh
On Wed, Jun 20, 2018 at 05:16:48PM +0900, Hyunho Cho wrote: > set -- -a -bc hello world > > echo $OPTIND# sh, bash = 1 > > getopts abc opt "$@" > echo $opt, $OPTIND # sh, bash = a, 2 > > getopts abc opt "$@" # bash = b, 2 <-- > different from "sh" > echo $opt, $OPTIND# sh = b, 3 (There is no "sh". There are just differing implementations of the POSIX standard. If you meant "dash", then yes, I get the same results as you using bash and dash.) POSIX says, Each time it is invoked, the getopts utility shall place the value of the next option in the shell variable specified by the name operand and the index of the next argument to be processed in the shell variable OPTIND. After your second call to getops, the second argument is only half-way processed. Your third call to getopts will still be processing the second argument. So, it's unclear what OPTIND should contain at that point. Since POSIX doesn't say anything about what OPTIND should contain when you're in the middle of processing an argument with multiple concatenated options inside it, I'd say that this falls into the realm of "unspecified behavior", and shells can do anything they want. So, not a bug in bash. Or in dash. Or in ksh, which by the way does the same thing as bash. I really don't understand what you're doing here, either. The only use of OPTIND is after the final call to getopts, when there are no more options to process. At that point, OPTIND tells you how many times you have to "shift" to get rid of all the options that were processed. (And you have to subtract 1, because legacy historical reasons.) Any attempt to use OPTIND in the middle of the loop seems pointless and hazardous to me. If you want the option-argument of an option that takes one, it's in the OPTARG variable. You use that inside the while getopts loop. (Your example didn't have any options with arguments, so I'm still puzzled as to what you're actually trying to do.)
propagating environment variable via internal command
Hi, when comparing strange behaviors of different shell implementations Tomas Janousek found this bug When using BASH as sh shell... $ ls -l /bin/sh lrwxrwxrwx 1 root root 4 Jun 13 16:25 /bin/sh -> bash ...internal shell command `set' propagates environment variable to the process its running: $ /bin/sh sh-4.4$ VARIABLE=value set -o noglob sh-4.4$ env | grep VARIABLE VARIABLE=value sh-4.4$ The problem doesn't occur if BASH is not running in POSIX mode $ /bin/bash bash-4.4$ VARIABLE=value set -o noglob bash-4.4$ env | grep VARIABLE bash-4.4$ This seems to be inconsistent with other shells but mainly it looks wrong. Best regards, S_W signature.asc Description: Digital signature
Re: Variables can’t contain NUL
On Sun, 2018-05-20 at 04:56 +0200, Garreau, Alexandre wrote: > On 2015-11-13 at 07:17, Greg Wooledge wrote: > Actually in the most general case, where those output streams may > contain NUL bytes, it requires two temp files, because you can't store > arbitrary data streams in bash variables at all. > > Why do bash variables use 0-terminated arrays instead of arrays structure > with a length attribute? > This is a question that interests me for various reasons: I don't really favor the idea that a shell shouldn't be considered a "real" programming language or be held to that kind of standard. Though it is difficult to reconcile that with backward-compatibility and POSIX-compatibility a lot of the time. Apart from the reasons already given: shells tend to assume some level of equivalence between facilities the shell language provides, and similar facilities the OS provides. For instance, shell variables are generally assumed to work the same as OS environment variables. These days there are cases where the two diverge (shell variables support arrays and such, while environment variables do not) and so you can't "export" an array variable, for instance. Encoding shell variables as length-prefixed arrays would create another such disparity: the underlying OS mechanisms for environment variables generally assume a NUL terminates an environment variable (for instance execve() or "man 7 environ") - even if the environment could be (mis-?)used to carry data with NUL in it, the program receiving that data would have to follow the same convention for how to use it, or the data would effectively be lost. NUL containment could be provided in shell variables (similar to how shell variables can provide arrays, etc. but can't "export" them) but then there's an additional problem, of what you can do with them. You can't provide a NUL as part of a command-line argument to an external command (because, like the environment, argv[] is by convention assumed to be NUL-terminated and the OS itself may enforce that assumption in some cases) - so you'd be pretty much limited to internal commands and shell functions - creating another disparity. "Disparities" aren't just theoretical problems or aesthetic blemishes, they turn into user frustration and bug reports. (As in "I put a NUL in a variable and it didn't work right") Personally I do think some method of handling arbitrary binary data in the shell would be a welcome addition (and I think zsh provides that - don't remember if ksh does) - it's just hard to resolve against some of the other underlying assumptions of the shell.
Re: propagating environment variable via internal command
On 6/20/18 9:25 AM, Tomáš Čech wrote: > Hi, > > > when comparing strange behaviors of different shell implementations Tomas > Janousek found this bug > > When using BASH as sh shell... > > $ ls -l /bin/sh > lrwxrwxrwx 1 root root 4 Jun 13 16:25 /bin/sh -> bash > > ...internal shell command `set' propagates environment variable to the > process its running: > > $ /bin/sh > sh-4.4$ VARIABLE=value set -o noglob > sh-4.4$ env | grep VARIABLE > VARIABLE=value > sh-4.4$ Posix requires this behavior, which dates back to the Bourne shell, for assignment statements that precede special builtins: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_14 It used to require the same behavior for assignment statements preceding shell function calls, but that was removed in the most recent update. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ signature.asc Description: OpenPGP digital signature
Re: Variables can’t contain NUL
On Wed, Jun 20, 2018 at 10:22 PM, George wrote:
>
> Personally I do think some method of handling arbitrary binary data in the
> shell would be a welcome addition (and I think zsh provides that - don't
> remember if ksh does)
>
Ksh93 has "typeset -b" which defines vars for binary data (actually base64
encoded). E.g.:
[STEP 100] $ echo ${.sh.version}
Version AJM 93u+ 2012-08-01
[STEP 101] $ typeset -b var
[STEP 102] $ var=$( echo hello world | base64 )
[STEP 103] $ echo $var
aGVsbG8gd29ybGQK
[STEP 104] $ printf %B var
hello world
[STEP 105] $
Re: $OPTIND varibale value is different from sh
On 20.6. 15:39, Greg Wooledge wrote: On Wed, Jun 20, 2018 at 05:16:48PM +0900, Hyunho Cho wrote: set -- -a -bc hello world getopts abc opt "$@" getopts abc opt "$@" # bash = b, 2 <-- different from "sh" echo $opt, $OPTIND# sh = b, 3 Since POSIX doesn't say anything about what OPTIND should contain when you're in the middle of processing an argument with multiple concatenated options inside it, I'd say that this falls into the realm of "unspecified behavior", and shells can do anything they want. So, not a bug in bash. Or in dash. Or in ksh, which by the way does the same thing as bash. Even worse, OPTIND doesn't contain all the necessary information. It just points at an argument, but doesn't tell what character within it is being processed (next). Resetting OPTIND to 1 is valid, so assigning to it has to clear that hidden information, so a simple OPTIND=$OPTIND can have some curious effects... I tested a couple of shells out of curiosity. Zsh in particular has a yet another set of values it returns in OPTIND during the loop. Though in the end, the final value is correct in all shells I tried: $ for sh in dash 'busybox sh' bash ksh93 zsh ; do printf "%-10s: " "$sh"; $sh -c 'while getopts abcd opt; do printf "$OPTIND "; done; printf "$OPTIND "; shift $(($OPTIND - 1)); echo "$1" ' sh -a -bcd hello; done dash : 2 3 3 3 3 hello busybox sh: 2 3 3 3 3 hello bash : 2 2 2 3 3 hello ksh93 : 2 2 2 3 3 hello zsh : 1 2 2 2 3 hello -- Ilkka Virta / itvi...@iki.fi
Re: $OPTIND varibale value is different from sh
Op 20-06-18 om 13:39 schreef Greg Wooledge: I really don't understand what you're doing here, either. The only use of OPTIND is after the final call to getopts, when there are no more options to process. At that point, OPTIND tells you how many times you have to "shift" to get rid of all the options that were processed. (And you have to subtract 1, because legacy historical reasons.) In other words, because all the options have now been processed, OPTIND points to the first argument *after* the list of options. I don't see how that is "because legacy historical reasons". It seems both logical and useful to me. Any attempt to use OPTIND in the middle of the loop seems pointless and hazardous to me. Agreed. - M.
Re: $OPTIND varibale value is different from sh
Op 20-06-18 om 17:45 schreef Ilkka Virta: $ for sh in dash 'busybox sh' bash ksh93 zsh ; do printf "%-10s: " "$sh"; $sh -c 'while getopts abcd opt; do printf "$OPTIND "; done; printf "$OPTIND "; shift $(($OPTIND - 1)); echo "$1" ' sh -a -bcd hello; done dash : 2 3 3 3 3 hello busybox sh: 2 3 3 3 3 hello bash : 2 2 2 3 3 hello ksh93 : 2 2 2 3 3 hello zsh : 1 2 2 2 3 hello yash has interesting behaviour here. While other shells use an extra internal (i.e. not exposed to the shell language) state variable to keep track of parsing multiple options combined in a single argument, yash just adds that extra state value to OPTIND. yash : 1:2 2:2 2:3 2:4 3 hello Which demonstrates quite well that the value of OPTIND in the middle of processing really should be considered unspecified, and that only the initial value (1) and the final value can be relied upon. Relevant POSIX text: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/getopts.html#tag_20_54 | If the application sets OPTIND to the value 1, a new set of | parameters can be used: either the current positional parameters or | new arg values. Any other attempt to invoke getopts multiple times in | a single shell execution environment with parameters (positional | parameters or arg operands) that are not the same in all invocations, | or with an OPTIND value modified to be a value other than 1, produces | unspecified results. - M.
Re: $OPTIND varibale value is different from sh
On 6/20/18 2:09 PM, Martijn Dekker wrote: > Op 20-06-18 om 13:39 schreef Greg Wooledge: >> I really don't understand what you're doing here, either. The only >> use of OPTIND is after the final call to getopts, when there are no >> more options to process. At that point, OPTIND tells you how many >> times you have to "shift" to get rid of all the options that were >> processed. (And you have to subtract 1, because legacy historical >> reasons.) > > In other words, because all the options have now been processed, OPTIND > points to the first argument *after* the list of options. I don't see how > that is "because legacy historical reasons". It seems both logical and > useful to me. It enables the universal idiom `shift $(( $OPTIND - 1 ))' to throw away the options and option arguments and deal with the rest of the arguments. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: propagating environment variable via internal command
Hi Chet and Tomáš,
On Wed, Jun 20, 2018 at 10:42:07AM -0400, Chet Ramey wrote:
> On 6/20/18 9:25 AM, Tomáš Čech wrote:
> > $ /bin/sh
> > sh-4.4$ VARIABLE=value set -o noglob
> > sh-4.4$ env | grep VARIABLE
> > VARIABLE=value
> > sh-4.4$
>
> Posix requires this behavior, which dates back to the Bourne shell, for
> assignment statements that precede special builtins:
>
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_14
>
> It used to require the same behavior for assignment statements preceding
> shell function calls, but that was removed in the most recent update.
Thanks for the quick reply!
I followed the link in item 2 and there is:
If the command name is a special built-in utility, variable assignments
shall affect the current execution environment. Unless the set -a option is
on (see set), it is unspecified:
Whether or not the variables gain the export attribute during the
execution of the special built-in utility
Whether or not export attributes gained as a result of the variable
assignments persist after the completion of the special built-in utility
It seems that dash implements this without the export, which means this
"surprising" example we came up with behaves "correctly", but there's still
this one, possibly more subtle:
$ {da,}sh -c 'VAR=val set -o noglob; echo $VAR'
val
So even though this does look a bit like behaviour defined by historic
implementations, I guess the conclusion is that we should just read up on
POSIX shells. :-)
Thank you Chet.
--
Tomáš Janoušek, DEV/SETI/CEP @ GoodData, +420 608 876 277
Bash 5.0-alpha tries to expand exclamation marks in quoted strings
I don't think this is intentional: [bero@c64 ~]$ echo "test!!" echo "testecho "testecho "test!""" testecho testecho test! ttyl bero
Re: Bash 5.0-alpha tries to expand exclamation marks in quoted strings
On 6/20/18 6:29 PM, Bernhard Rosenkraenzer wrote: > I don't think this is intentional: > > [bero@c64 ~]$ echo "test!!" > echo "testecho "testecho "test!""" > testecho testecho test! It's impossible to say whether or not this is correct without knowing what's in your history. Double quotes don't inhibit history expansion, and never have. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
