extglobs in case constructs
Configuration Information [Automatically generated, do not change]:
Machine: i686
OS: linux-gnu
Compiler: i686-pc-linux-gnu-gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i686'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i686-pc-linux-gnu'
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash'
-DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib
-DDEFAULT_PATH_VALUE='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
-DSTANDARD_UTILS_PATH='/bin:/usr/bin:/sbin:/usr/sbin'
-DSYS_BASHRC='/etc/bash/bashrc'
-DSYS_BASH_LOGOUT='/etc/bash/bash_logout' -DNON_INTERACTIVE_LOGIN_SHELLS
-DSSH_SOURCE_BASHRC -march=prescott -O2 -ggdb
uname output: Linux server 2.6.30-gentoo-r5 #1 SMP PREEMPT Tue Aug 18
11:26:16 CEST 2009 i686 Intel(R) Pentium(R) 4 CPU 3.00GHz GenuineIntel
GNU/Linux
Machine Type: i686-pc-linux-gnu
Bash Version: 4.0
Patch Level: 33
Release Status: release
Description:
I've found out that extglobs behave differently in different
constructs. To avoid syntax errors, the always have to be
enabled at parse time.
For comparison constructs like ``[[ $v == a@(a|b|c)c ]]'' this
is enough. For case constructs like ``case $v in a@(a|b|c)c)''
the extglob shopt has to be set at runtime as well. This is kind
of inconsistent.
I've read several messages by Chet on the bug-bash mailing list,
all claiming that the extglob shopt should change the parser
only. In this case one would expect that setting it at the time
a function gets defined should be enough, and that it shouldn't
be necessary when the function gets executed. Case constructs
inside a function render this assumption invalid.
Repeat-By:
-- Sample script ---
#!/bin/bash
shopt -s extglob
echo -n "definition: "
shopt extglob
f() {
v=abc
shopt $1 extglob
echo -n " runtime: "
shopt extglob
echo -n "comparison uses extglob "
if [[ $v == a@(a|b|c)c ]]; then
echo "yes"
else
echo "no"
fi
echo -n "case uses extglob "
case $v in
a@(a|b|c)c)
echo "yes"
;;
*)
echo "no"
;;
esac
}
f -s
f -u
-- Resulting output
definition: extglob on
runtime: extglob on
comparison uses extglob yes
case uses extglob yes
runtime: extglob off
comparison uses extglob yes
case uses extglob no
Re: extglobs in case constructs
On Thu, Oct 01, 2009 at 04:17:33PM +0200, Martin von Gagern wrote:
> #!/bin/bash
>
> shopt -s extglob
> f() {
> if [[ $v == a@(a|b|c)c ]]; then
> case $v in
> a@(a|b|c)c)
You're using extglobs inside a function, and extglob was enabled
at the time the function was delcared. The parser parsed them at that
time, decided that they are extended globs, and therefore that's what
they are.
Most people have the opposite problem -- they try to do this:
f() {
shopt -s extglob
if [[ $x = @(some|extended|glob) ]]; then
This does not work, because the parser parses the functions before the
extended globs are turned on. The solution is to move the
"shopt -s extglob" outside of the function, as you already have.
Re: extglobs in case constructs
Greg Wooledge wrote: > You're using extglobs inside a function, and extglob was enabled > at the time the function was delcared. The parser parsed them at that > time, decided that they are extended globs, and therefore that's what > they are. Then why don't they work as extended globs if the extglob shopt is set to off at runtime, from within the function, after it has been parsed? > Most people have the opposite problem I know, I understand the reason for this, and I even approve of it, because after all, the use of extended globs is something the person defining the function should worry about, not the person using the function. Sadly, currently both have to worry. signature.asc Description: OpenPGP digital signature
Re: preventing pipe reader from existing on writer exiting
On Wed, 2009-09-30 at 23:13 +0200, Andreas Schwab wrote: > > Just make sure the write side of the pipe is not closed prematurely. Hrm. Yes, of course. John's solution of having a null writer keeping it open is one way -- which I might just use. > $ (n=0; while [ $n -lt 10 ]; do cat /dev/zero; let n=$n+1; done) > /tmp/fifo & But this is where (simplified) my example using cat went sideways. :-( In my real world use, the first cat is actually mplayer and doesn't have the option of writing to stdout instead of a named file for this particular use of it. So other than John's solution, I suppose I could still follow your advice of not having the fifo write close prematurely and insulate the writer from the fifo. I can't think of another way other than using two fifos with a cat between them. Can anyone else? FWIW, the actual process which the second cat is standing in for is oggenc and it can take stdin, if that makes it any easier. So to recap: mplayer ... /tmp/fifo & oggenc ... /tmp/fifo The mplayer may terminate prematurely and need to be restarted, ideally without oggenc knowing this. Cheers and much thanx! b. signature.asc Description: This is a digitally signed message part
Re: preventing pipe reader from existing on writer exiting
"Brian J. Murrell" writes: > But this is where (simplified) my example using cat went sideways. :-( > In my real world use, the first cat is actually mplayer and doesn't have > the option of writing to stdout instead of a named file for this > particular use of it. Perhaps you can use /dev/stdout instead. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Re: Strange compgen behaviour
I decided to make this into a blog post to in some way contribute back to the community for all the help I got. http://muublog.blogspot.com/2009/10/bash-completion-for-playing-movies.html Enjoy! /Mathias
Re: extglobs in case constructs
Martin von Gagern wrote:
> I've found out that extglobs behave differently in different
> constructs. To avoid syntax errors, the always have to be
> enabled at parse time.
> For comparison constructs like ``[[ $v == a@(a|b|c)c ]]'' this
> is enough. For case constructs like ``case $v in a@(a|b|c)c)''
> the extglob shopt has to be set at runtime as well. This is kind
> of inconsistent.
OK, I guess this is due to the following difference:
execute_cmd.c:3225-3231 in execute_cond_node:
> int oe;
> oe = extended_glob;
> extended_glob = 1;
> result = binary_test (cond->op->word, arg1, arg2,
> TEST_PATMATCH|TEST_ARITHEXP)
> ? EXECUTION_SUCCESS
> : EXECUTION_FAILURE;
> extended_glob = oe;
execute_cmd.c:2931 in execute_case_command:
> match = strmatch (pattern, word, FNMATCH_EXTFLAG|FNMATCH_IGNCASE) !=
> FNM_NOMATCH;
What's interesting is what you don't see here: the case command doesn't
temporarily set extended_glob to 1 as the cond node does. I guess this
is where the different behaviour at runtime comes from.
Is this difference intentionally, or is this a bug? If it is a bug, the
attached patch should fix it, although it might introduce other issues.
At least for my example script, the patch does its job.
I'm a bit surprised that execute_cond_node seems to unconditionally
enable the extended glob mode. I don't see any check whether the thing
has been parsed as an extended glob in the first place.
Is this because things that look like extended globs will always be
invalid syntax if extglob is off? So that if anything got parsed
successfully and still looks like an extended glob, the code can be sure
it was parsed as an extended glob? In that case, is there any point in
not always enabling extended globs, except if you are testing the
portability of your script? If so, I'll second the request to always
enable them, but I have some doubts that's the case.
As an alternative to my patch, one could maybe move all this temporary
extglob stuff to the function definitions. I.e. a function defined with
extglob on will also have extglob on when executed, and vice versa. This
might break some compatibility, though, especially for subshells. And
simply saving and restoring functins by listing their definition would
become a trouble as well. So it's just a thought so far.
Greetings,
Martin
Allow use of extended globs in case constructs at runtime,
even if the extglob shopt is not set.
2009-10-01 Martin von Gagern
References:
http://thread.gmane.org/gmane.comp.shells.bash.bugs/13518
Index: bash-4.0/execute_cmd.c
===
--- bash-4.0.orig/execute_cmd.c
+++ bash-4.0/execute_cmd.c
@@ -2915,6 +2915,7 @@ execute_case_command (case_command)
QUIT;
for (list = clauses->patterns; list; list = list->next)
{
+ int oe;
es = expand_word_leave_quoted (list->word, 0);
if (es && es->word && es->word->word && *(es->word->word))
@@ -2928,8 +2929,11 @@ execute_case_command (case_command)
/* Since the pattern does not undergo quote removal (as per
Posix.2, section 3.9.4.3), the strmatch () call must be able
to recognize backslashes as escape characters. */
+ oe = extended_glob;
+ extended_glob = 1;
match = strmatch (pattern, word, FNMATCH_EXTFLAG|FNMATCH_IGNCASE) !=
FNM_NOMATCH;
free (pattern);
+ extended_glob = oe;
dispose_words (es);
Buffer underflow in help builtin with narrow window
Configuration Information [Automatically generated, do not change]: Machine: i686 OS: linux-gnu Compiler: i686-pc-linux-gnu-gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i686' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i686-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -DDEFAULT_PATH_VALUE='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' -DSTANDARD_UTILS_PATH='/bin:/usr/bin:/sbin:/usr/sbin' -DSYS_BASHRC='/etc/bash/bashrc' -DSYS_BASH_LOGOUT='/etc/bash/bash_logout' -DNON_INTERACTIVE_LOGIN_SHELLS -DSSH_SOURCE_BASHRC -O3 -march=core2 -fomit-frame-pointer -pipe uname output: Linux koko 2.6.27-gentoo-r8 #1 SMP Wed Mar 18 21:02:33 GMT 2009 i686 Intel(R) Celeron(R) CPU E1200 @ 1.60GHz GenuineIntel GNU/Linux Machine Type: i686-pc-linux-gnu Bash Version: 4.0 Patch Level: 28 Release Status: release Description: When 1 <= $COLUMNS <= 5 the "help" builtin writes to an invalid bit of memory. Repeat-By: COLUMNS=5 help Fix: In builtins/help.def in show_builtin_command_help(), a $COLUMNS value that's <=0 is treated as being 80, but other values are used directly. But values under 6 are unsafe, as ($COLUMNS/2)-3 is used as in index to the local blurb[] array. One fix would thus be to change if (width <= 0) width = 80; to if (width <= 5) width = 80; That said, even slightly bigger widths don't yield very helpful output. "COLUMNS=10 help", for example, is all but useless in practice -- so perhaps a larger threshold should be used.
Re: extglobs in case constructs
Chet Ramey wrote: > In general, if you enable extglob before a function is parsed and leave > it on when the function is executed, you should be ok. I'm thinking about this in terms of bash completion functions. Many of them use extended globs, but they should not force a specific permanent extglob setting on the user. And cleanly restoring the setting can a bit tricky. I believe I have a solution for this, though: https://alioth.debian.org/tracker/index.php?func=detail&aid=311628&group_id=100114&atid=413095 Still, that solution brings a bit of overhead, and if that could have been avoided, I'd been even happier. > The reason that [[ conditional commands don't require extglob to be > enabled while they're executed (and case commands do) lies in how the > commands are defined to behave. case commands are defined by Posix: > the pattern is always a Posix shell pattern, and any extension to that > must be enabled manually. The [[ command, which bash picked up from ksh, > is defined so that the rhs of `==' is an extended pattern (ksh always > uses extended patterns, without requiring a special option). For > compatibility, bash forces the rhs of the [[ command's == and != > operators to be matched as an extended pattern when executed. That's a good explanation, and a valid reason. If you say it's going to stay that way, I'll accept that and stop suggesting alternatives. Still, if there were a way to have the case command remember whether it was defined with or without extglob, then that state could be recreated when the command is executed. You'd still have to enable a manual switch if you wanted to diverge from the Posix standard, only now it would only have to be when you define the function. The benefit would be that the semantics of the function would stay unaffected from later modifications of this shopt. As an alternative, would you consider things like "local shopts"? I'm thinking about a flag like "-l" to shopt which would set the option for the duration of the current function, but neither export it to functions called therein nor leave it in place when the function returns. I believe this would be quite a major task, but might be worthwhile for functions that need to fine-tune these options but don't want to affect their surroundings. If you are interested, I could try writing a patch. > In fact, to be perfectly compatible, bash should temporarily enable > extglob when parsing the rhs of `==' and `!=' inside a conditional > command. I will look at doing that for the next version. I don't know if I speak for a majority of users, but in my daily use I prefer bash to be usable intuitively, and consider compatibility to other shells (which I rarely if ever use) second to that. Things change if I want to write portable scripts, but in this case, I'd take plain sh as the common denominator, and would also invoke bash with that name. So yes, compatibility with other shells is nice, and always having extended globs as the rhs of an == would increase the ease of use. Different behaviour for different constructs on the other hand is quite a bit counterintuitive, and for me personally compatibility with other shells isn't important enough to warrant this quirk. Greetings, Martin signature.asc Description: OpenPGP digital signature
Re: extglobs in case constructs
Martin von Gagern wrote: > Configuration Information [Automatically generated, do not change]: > Machine: i686 > OS: linux-gnu > Compiler: i686-pc-linux-gnu-gcc > Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i686' > -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i686-pc-linux-gnu' > -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' > -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib > -DDEFAULT_PATH_VALUE='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' > -DSTANDARD_UTILS_PATH='/bin:/usr/bin:/sbin:/usr/sbin' > -DSYS_BASHRC='/etc/bash/bashrc' > -DSYS_BASH_LOGOUT='/etc/bash/bash_logout' -DNON_INTERACTIVE_LOGIN_SHELLS > -DSSH_SOURCE_BASHRC -march=prescott -O2 -ggdb > uname output: Linux server 2.6.30-gentoo-r5 #1 SMP PREEMPT Tue Aug 18 > 11:26:16 CEST 2009 i686 Intel(R) Pentium(R) 4 CPU 3.00GHz GenuineIntel > GNU/Linux > Machine Type: i686-pc-linux-gnu > > Bash Version: 4.0 > Patch Level: 33 > Release Status: release > > Description: > > I've found out that extglobs behave differently in different > constructs. To avoid syntax errors, the always have to be > enabled at parse time. > For comparison constructs like ``[[ $v == a@(a|b|c)c ]]'' this > is enough. For case constructs like ``case $v in a@(a|b|c)c)'' > the extglob shopt has to be set at runtime as well. This is kind > of inconsistent. > > I've read several messages by Chet on the bug-bash mailing list, > all claiming that the extglob shopt should change the parser > only. In this case one would expect that setting it at the time > a function gets defined should be enough, and that it shouldn't > be necessary when the function gets executed. Case constructs > inside a function render this assumption invalid. You've mischaracterized the substance of those messages. The responses are always to a question about why extglob also has to be enabled when a function is parsed, instead of just when the function is executed. The extglob option controls all aspects of extended globbing: it must be enabled when executing to pass the correct flags to the matching engine, and it must be enabled when the construct is parsed to turn off the usual `metacharacter' definitions of parens and `|'. In general, if you enable extglob before a function is parsed and leave it on when the function is executed, you should be ok. The reason that [[ conditional commands don't require extglob to be enabled while they're executed (and case commands do) lies in how the commands are defined to behave. case commands are defined by Posix: the pattern is always a Posix shell pattern, and any extension to that must be enabled manually. The [[ command, which bash picked up from ksh, is defined so that the rhs of `==' is an extended pattern (ksh always uses extended patterns, without requiring a special option). For compatibility, bash forces the rhs of the [[ command's == and != operators to be matched as an extended pattern when executed. In fact, to be perfectly compatible, bash should temporarily enable extglob when parsing the rhs of `==' and `!=' inside a conditional command. I will look at doing that for the next version. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
