Re: param expansion with single-character special vars in the environment
I can understand this, but perhaps this discussion is too general? in this particular case there is no point in the assignment. Unless you can convince me there is? pg On 5 May 2016, at 00:31, Dan Douglas wrote: > ...Also remember it isn't feasible to actually validate a "name" in a > script because a name can contain a subscript with a command > substitution that effectively requires parsing the full language. > (there are some tricks like with `set -nv` and reading its output to > shanghai the shell parser into doing your bidding, but that's not very > practical.). Before bash had namerefs, it could ignore aspects of > "invalid" names, like trailing characters after an array subscript, > which makes some valid ksh names at least get partially interpreted in > bash, like `a[foo].bar` or `a[foo][bar]`.
Re: namref dicussion continued: [PATCH] export -r reference
ok, so Dan wants this patch.
I took a look at your script, there are two cases it addresses: exporting
nameref and doing it from inside the function scope, I applied both patches:
nameref masking in scope:
https://lists.gnu.org/archive/html/bug-bash/2016-05/msg00012.html
exporting namerefs:
https://lists.gnu.org/archive/html/bug-bash/2016-05/msg00024.html
and with this modification to your script:
diff --git a/exportscope b/exportscope
index 05bae73..0125681 100644
--- a/exportscope
+++ b/exportscope
@@ -37,7 +37,8 @@ function h {
;;
2)
typeset -n _x=$2
- typeset -x _x='in h2'
+ export -r _x;
+ _x='in h2';
eval "$showEnv" h2
esac
}
@@ -75,4 +76,4 @@ for sh in bash ksh mksh zsh; do
printf '%s\n\n' "$("$sh" -c "$code" -- "$sh" 2>&1)"
done
You can get (difference in test 3):
#./exportscope
bash test 1:
subprocess f: declare -x x="in f"
local f: declare -x x="in f"
global: declare -x x="global"
bash test 2:
subprocess g: declare -x x="in g"
subprocess f: declare -x x="in f"
local f: declare -x x="in f"
local g: declare -x x="in g"
global: declare -- x="unset"
bash test 3:
subprocess h2: declare -x x="in h2"
subprocess h1: declare -x x="in h2"
local h1: declare -x x="in h2"
Maybe you can comment on wether the patches are valid.
cheers,
pg
On 4 May 2016, at 22:40, Dan Douglas wrote:
> Yeah I was just looking for this old script last night and just found it :)
>
> https://gist.github.com/ormaaj/04923e11e8bdc27688ad
>
> If you scroll down to the output for "test 3" where "h" gets called
> and passes a local "x" to a function that creates a reference to it
> and exports the reference you can see that bash calls "x" unset in
> both the first and second scope. As I recall we were discussing the
> way exported locals interact between scopes at the time, not namerefs,
> but I added the case for namerefs since I wasn't sure how this should
> work. I should probably run it again with the current devel branch.
>
> Even though bash doesn't yet support references for parameters like
> ksh93, neither does mksh, which also shows differences from bash.
Re: namref dicussion continued: [PATCH] export -r reference
ok, a small correction, the problem lies in exportscope:40, but none of the
patches mentioned solve it, the same behaviour of the test can be recovered
with just export instead of declare on this line (at least proves that the
patches dont break it ;-)). some other make_local_variable issue I guess.
pg
On 5 May 2016, at 09:46, Piotr Grzybowski wrote:
>
> ok, so Dan wants this patch.
> I took a look at your script, there are two cases it addresses: exporting
> nameref and doing it from inside the function scope, I applied both patches:
>
> nameref masking in scope:
> https://lists.gnu.org/archive/html/bug-bash/2016-05/msg00012.html
>
> exporting namerefs:
> https://lists.gnu.org/archive/html/bug-bash/2016-05/msg00024.html
>
> and with this modification to your script:
>
> diff --git a/exportscope b/exportscope
> index 05bae73..0125681 100644
> --- a/exportscope
> +++ b/exportscope
> @@ -37,7 +37,8 @@ function h {
>;;
>2)
>typeset -n _x=$2
> - typeset -x _x='in h2'
> + export -r _x;
> + _x='in h2';
>eval "$showEnv" h2
>esac
> }
> @@ -75,4 +76,4 @@ for sh in bash ksh mksh zsh; do
>printf '%s\n\n' "$("$sh" -c "$code" -- "$sh" 2>&1)"
> done
>
> You can get (difference in test 3):
>
> #./exportscope
> bash test 1:
>subprocess f: declare -x x="in f"
>local f: declare -x x="in f"
>global: declare -x x="global"
> bash test 2:
>subprocess g: declare -x x="in g"
>subprocess f: declare -x x="in f"
>local f: declare -x x="in f"
>local g: declare -x x="in g"
>global: declare -- x="unset"
> bash test 3:
>subprocess h2: declare -x x="in h2"
>subprocess h1: declare -x x="in h2"
>local h1: declare -x x="in h2"
>
> Maybe you can comment on wether the patches are valid.
>
> cheers,
> pg
>
>
>
> On 4 May 2016, at 22:40, Dan Douglas wrote:
>
>> Yeah I was just looking for this old script last night and just found it :)
>>
>> https://gist.github.com/ormaaj/04923e11e8bdc27688ad
>>
>> If you scroll down to the output for "test 3" where "h" gets called
>> and passes a local "x" to a function that creates a reference to it
>> and exports the reference you can see that bash calls "x" unset in
>> both the first and second scope. As I recall we were discussing the
>> way exported locals interact between scopes at the time, not namerefs,
>> but I added the case for namerefs since I wasn't sure how this should
>> work. I should probably run it again with the current devel branch.
>>
>> Even though bash doesn't yet support references for parameters like
>> ksh93, neither does mksh, which also shows differences from bash.
>
Re: set isearch-terminators "\r" in .inputrc causes malloc assert fails/death
hi, I cannot replicate it in anyway on mac os x or linux. Could you please define "sometimes", and supply the exact version of bash that shows this behavior, and the version you refer to as "my old bash"? I have a distinct feeling that it is something different than my old bash. cheers, pg On 4 May 2016, at 19:43, Britton Kerin wrote: > I tried with export INPUTRC=test_inputrc, where test_inputrc contains > just this: > > set isearch-terminators "\r" > > this causes bash to sometimes spit out things like this: > > bkerin@debian:~$ ls > > malloc: .././variables.c:2497: assertion botched > malloc: block on free list clobbered > last command: ls > Aborting...Aborted > > also sometimes it just dies and takes the terminal with it. > > I'm not sure if this is specific to using \r as an isearch terminator but I > suspect it is since hitting return normaly causes bash to fire off the command > you're searching for. Not sure though. > > My old bash didn't do this, if anyone cares enough to ask I can bring > that computer back up and check the bash/readline version to narrow > down the search where this bug crept in. > > I'm really hoping for a fix as reverse search followed by enter to > bring a command back onto the command line for subsequent editing is > deeply ingrained for me, I keep re-executing old commands. > > Here is my debian system info: > > -- System Information: > Debian Release: 8.3 > APT prefers stable-updates > APT policy: (500, 'stable-updates'), (500, 'stable') > Architecture: amd64 (x86_64) > > Kernel: Linux 4.4.5.emp3 (SMP w/8 CPU cores) > Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8) > Shell: /bin/sh linked to /bin/bash > Init: systemd (via /run/systemd/system) > > Versions of packages bash depends on: > ii base-files 8+deb8u3 > ii dash 0.5.7-4+b1 > ii debianutils 4.4+b1 > ii libc62.19-18+deb8u3 > ii libncurses5 5.9+20140913-1+b1 > ii libtinfo55.9+20140913-1+b1 > > Versions of packages bash recommends: > ii bash-completion 1:2.1-4 > > Versions of packages bash suggests: > pn bash-doc > > -- no debconf information > > > Thanks, > Britton >
Re: set isearch-terminators "\r" in .inputrc causes malloc assert fails/death
On Thu, May 05, 2016 at 10:30:04AM +0200, Piotr Grzybowski wrote: > hi, > > I cannot replicate it in anyway on mac os x or linux. > Could you please define "sometimes", and supply the exact version of bash > that shows this behavior, and the version you refer to as "my old bash"? I > have a distinct feeling that it is something different than my old bash. > > cheers, > pg > I'm able to replicate with the master branch. | dualbus@yaqui ...src/gnu/bash % INPUTRC=/dev/fd/3 gdb --batch ./bash -ex r 3<<<'set isearch-terminators "\r"' | dualbus@yaqui:~/local/src/gnu/bash$ ls | | malloc: unknown:0: assertion botched | malloc: block on free list clobbered | last command: ls | Aborting... | Program received signal SIGABRT, Aborted. | 0x7763a107 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 | 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. | | | Program received signal SIGABRT, Aborted. | 0x7763a107 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 | 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. | (gdb) bt | #0 0x7763a107 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 | #1 0x7763b4e8 in __GI_abort () at abort.c:89 | #2 0x00447470 in programming_error (format=0x4f9bf0 "malloc: block on free list clobbered") at error.c:176 | #3 0x004d8bc2 in xbotch (mem=0x8a2e08, e=0, s=0x4f9bf0 "malloc: block on free list clobbered", file=0x4dcd68 "make_cmd.c", line=100) at malloc.c:319 | #4 0x004d9516 in internal_malloc (n=3, file=0x4dcd68 "make_cmd.c", line=100, flags=1) at malloc.c:801 | #5 0x004d9ee2 in sh_malloc (bytes=3, file=0x4dcd68 "make_cmd.c", line=100) at malloc.c:1187 | #6 0x004882e6 in sh_xmalloc (bytes=3, file=0x4dcd68 "make_cmd.c", line=100) at xmalloc.c:183 | #7 0x0042fd5a in make_bare_word (string=0x72b0a8 "ls") at make_cmd.c:100 | #8 0x004464b7 in copy_word (w=0x731d28) at copy_cmd.c:61 | #9 0x004464f5 in copy_word_list (list=0x731788) at copy_cmd.c:75 | #10 0x0046543e in expand_word_list_internal (list=0x731788, eflags=31) at subst.c:9656 | #11 0x00464a5e in expand_words (list=0x731788) at subst.c:9280 | #12 0x0043b3cc in execute_simple_command (simple_command=0x733a88, pipe_in=-1, pipe_out=-1, async=0, fds_to_close=0x731d08) at execute_cmd.c:4000 | #13 0x00435829 in execute_command_internal (command=0x733a48, asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x731d08) at execute_cmd.c:787 | #14 0x00434e8a in execute_command (command=0x733a48) at execute_cmd.c:390 | #15 0x00420d00 in reader_loop () at eval.c:160 | #16 0x0041eb21 in main (argc=1, argv=0x7fffe528, env=0x7fffe538) at shell.c:756 | | dualbus@yaqui ...src/gnu/bash % INPUTRC=/dev/fd/3 gdb --batch ./bash -ex 'b ' -ex r 3<<<'set isearch-terminators "\n"' | No default breakpoint address now. | dualbus@yaqui:~/local/src/gnu/bash$ ls | | malloc: make_cmd.c:100: assertion botched | malloc: block on free list clobbered | last command: ls | Aborting... | Program received signal SIGABRT, Aborted. | 0x7763a107 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 | 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. | dualbus@yaqui ...src/gnu/bash % INPUTRC=/dev/fd/3 gdb --batch ./bash -ex 'b ' -ex r 3<<<'set isearch-terminators "a"' | No default breakpoint address now. | dualbus@yaqui:~/local/src/gnu/bash$ ls | | malloc: /usr/src/local/bash/bash-4.3-patched/parse.y:6226: assertion botched | malloc: block on free list clobbered | last command: ls | Aborting... | Program received signal SIGABRT, Aborted. | 0x7763a107 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 | 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. | dualbus@yaqui ...src/gnu/bash % INPUTRC=/dev/fd/3 gdb --batch ./bash -ex 'b ' -ex r 3<<<'set isearch-terminators "123"' | No default breakpoint address now. | dualbus@yaqui:~/local/src/gnu/bash$ ls | | malloc: make_cmd.c:100: assertion botched | malloc: block on free list clobbered | last command: ls | Aborting... | Program received signal SIGABRT, Aborted. | 0x7763a107 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 | 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. Forgive the extraneus -ex 'b ' on gdb, I'm lazy and I left it :-D I can't reproduce it in devel. -- Eduardo Bustamante https://dualbus.me/
Re: param expansion with single-character special vars in the environment
On 5/4/16 3:37 PM, Piotr Grzybowski wrote: > > On 4 May 2016, at 17:51, Chet Ramey wrote: > >> The issue I'm thinking about currently is whether or not to allow nameref >> variables to have numeric (integer) values. bash-4.3 didn't allow those >> values to be expanded, but allowed them to be assigned. It doesn't seem >> harmful to change the error to assignment time. > > I vote for error at assignment, whats the point of allowing pointless > assignment? No reason, except for minimal backwards compatibility. I'm leaning towards that position myself. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: History modifier :p ignored prior to another history reference
Hey, confirmed, this is what happens by construction. this one is going to be tricky, expansion happens on the shell line, way before it comes to execution. I do not see reparsing again at execute_command even if we had means, which we dont, to store the information from history expansion to know what to execute and what not. Re-tokenizing by rules of history expansion anywhere outside histexpand seems out of place as much as implementing shell lexer (again) in histexpand. Maybe splitting in history_expand and returning tokens with options, that are later put together for execution? Either way seems a bit large. Currently the design indicates: :p applies to the whole line. Maybe I am mistaken, correctly me if I am wrong. cheers, pg On 3 May 2016, at 18:15, Dean Stanton wrote: > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: linux-gnu > Compiler: gcc > Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' > -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-redhat-linux-gnu' > -DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' > -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include > -I./lib -D_GNU_SOURCE -DRECYCLES_PIDS -O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector > --param=ssp-buffer-size=4 -m64 -mtune=generic > uname output: Linux dstanton-vm.tintri.com 2.6.32.26-175.fc12.x86_64 #1 SMP > Wed Dec 1 21:39:34 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux > Machine Type: x86_64-redhat-linux-gnu > > Bash Version: 4.0 > Patch Level: 38 > Release Status: release > > Description: > History modifier :p is not honored when it appears prior to another history > reference on the same command line. > > I tried to find a website with known bash bugs (to search to see if > this issue was already discussed), but didn't find such a site. > > Repeat-By: > $ echo one two three four > one two three four > $ echo !:2:p ; echo !$ # :p was ignored! > echo two ; echo four > two > four > > $ echo one two three four > one two three four > $ echo !:p !! # :p was ignored! > echo one two three four echo one two three four > one two three four echo one two three four > > > I expected the entire command line to be only printed and not > executed. The manpage says >p Print the new command but do not execute it. > Instead, the :p was ignored and the entire command line was (printed > and then) executed. > > $ echo one two three four > one two three four > $ echo !:2 ; echo !$:p # :p was heeded. > echo two ; echo four > $ echo one two three four > $ echo !:1:p !$ > # :p was ignored > echo one four > one four > >
Re: declare [-+]n behavior on existing (chained) namerefs
On 4/29/16 12:50 PM, Grisha Levit wrote:
> The only buggy behavior when inside of functions (ignoring scope issues)
> seems to be that `declare -n ref’ unsets the value of $ref if it previously
> pointed to a variable that is unset.
>
> |$ f() { unset var; declare -n ref=var; declare -n ref; declare -p ref; }; f
> declare -n ref # why did the value of ref disappear? $ f() { local var;
> declare -n ref=var; declare -n ref; declare -p ref; }; f declare -n ref="var"
> |
I fixed this as part of making other changes prompted by the discussion.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: declare [-+]n behavior on existing (chained) namerefs
On 5/2/16 1:55 PM, Piotr Grzybowski wrote:
> Hi,
>
> I hope the attached patch does not break to much; it addresses the masking:
>
>> declare -nt r=a; f() { declare a; declare -n r=a; declare -p a r; }; f
>
> as per Grisha's report; it tries to enlighten make_local_variable to the
> existence of nameref by returning existing local nameref in scope (declare -n
> r=PATH; declare -n r; returns r with value of PATH) and by (hopefully)
> correctly making existing nameref take precedence over the dereferenced value.
>
> Would you please consider it?
Thanks, this is a good fix.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: History modifier :p ignored prior to another history reference
On 5 May 2016, at 20:08, Piotr Grzybowski wrote: > [..] > Currently the design indicates: :p applies to the whole line. As Dean just has pointed out to me, this sentence is obviously wrong. it should be: Currently the design indicates: the last :p applies to the whole line, the information about existence of others before it, is overwritten. sorry, pg > On 3 May 2016, at 18:15, Dean Stanton wrote: > >> Configuration Information [Automatically generated, do not change]: >> Machine: x86_64 >> OS: linux-gnu >> Compiler: gcc >> Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' >> -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-redhat-linux-gnu' >> -DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' >> -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include >> -I./lib -D_GNU_SOURCE -DRECYCLES_PIDS -O2 -g -pipe -Wall >> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector >> --param=ssp-buffer-size=4 -m64 -mtune=generic >> uname output: Linux dstanton-vm.tintri.com 2.6.32.26-175.fc12.x86_64 #1 SMP >> Wed Dec 1 21:39:34 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux >> Machine Type: x86_64-redhat-linux-gnu >> >> Bash Version: 4.0 >> Patch Level: 38 >> Release Status: release >> >> Description: >> History modifier :p is not honored when it appears prior to another history >> reference on the same command line. >> >>I tried to find a website with known bash bugs (to search to see if >> this issue was already discussed), but didn't find such a site. >> >> Repeat-By: >> $ echo one two three four >>one two three four >>$ echo !:2:p ; echo !$ # :p was ignored! >>echo two ; echo four >>two >>four >> >> $ echo one two three four >>one two three four >>$ echo !:p !! # :p was ignored! >> echo one two three four echo one two three four >> one two three four echo one two three four >> >> >>I expected the entire command line to be only printed and not >> executed. The manpage says >> p Print the new command but do not execute it. >>Instead, the :p was ignored and the entire command line was (printed >> and then) executed. >> >>$ echo one two three four >>one two three four >>$ echo !:2 ; echo !$:p # :p was heeded. >>echo two ; echo four >> $ echo one two three four >> $ echo !:1:p !$ >> # :p was ignored >> echo one four >> one four >> >> >
Re: declare [-+]n behavior on existing (chained) namerefs
On 4/28/16 9:49 PM, Grisha Levit wrote:
> In a slightly different version, with `declare -n r; r=a', the function
> exits with code 1 after the `r=a' statement:
>
> $ declare -nt r=a; f() { declare a; declare -n r; r=a; declare -p a r; };
> f; echo $?
> 1
In this case, you create a self-referencing local nameref, which ends up
resolving to nothing, which causes an assignment error, which results in
function execution being aborted. You create the self-referencing local
nameref because bash follows the nameref chain for `r', and resolves it
to `a'. It's as Piotr surmised: as if you ran `typeset -n a; a=a'.
When asked to create a local variable that shadows a nameref from a
different context, declare needs to throw away the results of that nameref
chain search and just create a local variable.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: namref dicussion continued: [PATCH] export -r reference
On Thu, May 5, 2016 at 2:46 AM, Piotr Grzybowski wrote: > ok, so Dan wants this patch. Yes I think your bash patch is probably an improvement at least in this particular case. > Maybe you can comment on wether the patches are valid. The posix export and readonly don't produce locally scoped variables at all, so it's expected that e.g. `typeset -x` and `export` will differ. This is yet another problem that's better solved by immutable C++-like references. At least, the problem goes away when variables are passed explicitly. The exact mechanism isn't really documented. When ksh determines that a nameref assignment is to a positional parameter during evaluation, you get a completely different type of reference. That is, if `$1` expands to `x`, then `typeset -n ref=$1` and `typeset -n ref=x` aren't equivalent like they are in bash. In the former case, any use of `ref` except for `typeset +n` and `unset -n` applies to the exact object that `$1` referred to at the time the reference was created, even if it's another nameref in which case that reference is followed. The latter gives you a bash-like dynamic reference that points to somewhere within either the current scope or the global scope (due to static scoping) unless it happens to point to another reference parameter of course. When you changed that `typeset` line to `export -r _x`, things become less clear. Ksh sees that _x is a reference parameter and because of the rules it does the only thing that makes sense which is apply the export attribute to whatever was passed to the function for both `export` or `typeset -x`, though there may be some other differences between those. Bash on the other hand never does what you'd want for either `export` or `typeset -x`. For typeset, it tears down the reference and redefines it as a new local with the export attribute. It does that indiscriminately for any usage of typeset, which means you can't correctly modify an attribute by reference in bash. If you actually wanted that result, the correct way would be to first use `typeset +n _x` followed by `typeset -x _x`. If you choose `export`, the correct behavior is even less clear thanks to dynamic scope. Your patch is a partial solution that makes `export` modify the attribute in the way you would expect in the absence of the reference parameter feature. Ignoring namerefs, the only other way for bash to modify the attribute of the currently visible dynamically scoped local without declaring a new local is to use export, which is why bash behaves as it did. This solution doesn't solve the same problem for attributes other than export and readonly. Bash can't use e.g. `typeset -u` by reference. It also doesn't solve the many other problems caused by this like name conflicts.
Re: namref dicussion continued: [PATCH] export -r reference
Oh I see you talked about some of this already in the "chained nameref" thread. I haven't read that one yet... I wonder why `export -r` requires a different solution because it's pretty much the same underlying problem.
