Re: Sourcing a script from a pipe is not reliable
interesting. it looks like somehow the ". /dev/stdin yes" side isn't waiting for the "echo echo '$1'" side to write to the pipe. in theory, you should be able to delay the echo and it should still work... (sleep 1; echo echo '$1') | . /dev/stdin yes but on my mac, adding the sleep makes it fail reliably on the first iteration. on my linux machine, it seems to succeed reliably even with the sleep - as expected. On Wed, Jan 9, 2019 at 10:54 PM Jeremy wrote: > Configuration Information [Automatically generated, do not change]: > > Machine: Mac > > OS: Darwin > > Compiler: gcc > > Compilation CFLAGS: Xcode > > uname output: Darwin Octo.local 15.6.0 Darwin Kernel Version 15.6.0: Thu > Jun 21\ > > 20:07:40 PDT 2018; root:xnu-3248.73.11~1/RELEASE_X86_64 x86_64 > > Machine Type: x86_64-Apple-Darwin > > > Bash Version: 3.2 > > Patch Level: 48 > > Release Status: relase > > > Although bashbug listed the Patch Level as 48 (and misspelled "release") > the version string is > > 3.2.57(1)-release (x86_64-apple-darwin15) > > > This is on MacOS 10.11.6. If there is a better place for me to report this > bug, please let me know. > > Description: > > Sourcing a script from a pipe is not reliable. > > > Repeat-By: > > This command line should run forever: > > > i=0; while [ "_$(echo echo '$1' | . /dev/stdin yes)" = "_yes" ]; \ > > do echo -n .; ((i++)); done; printf "\n%s\n" $i > > > When I run it, it usually terminates with $i much less than 1,000. >
Re: Sourcing a script from a pipe is not reliable
On Thu, Jan 10, 2019 at 12:01:10AM -0800, don fong wrote: > but on my mac, adding the sleep makes it fail reliably on the first > iteration. > > on my linux machine, it seems to succeed reliably even with the sleep - as > expected. That's because the version of bash on Mac OS X is 3.2. Bash 3.2 does not support sourcing from a process substitution. wooledg:~$ bash-3.2 -c 'source <(echo echo hi)' wooledg:~$ bash-4.0 -c 'source <(echo echo hi)' hi Like Eduardo, I believe there is not going to be a fix for this on your vendor's version of bash, due to its age. You could install a newer version in /usr/local/bin (or another directory of your choice) and use that instead. Otherwise, you'll just have to code around the bugs and missing features of your vendor's bash.
Re: bash5: "progcomp_alias" and "complete -D"
On 1/9/19 10:51 PM, Clark Wang wrote: > Seems like if there's "complete -D" defined then "progcomp_alias" would > never work. So does it make more sense to make "progcomp_alias" has higher > priority than "complete -D"? Maybe. The other side is that `progcomp_alias' only comes into play if you can't find a valid completion for a particular command, and `complete -D' ensures that you always will. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Sourcing a script from a pipe is not reliable
Agreed there is no likelihood of a patch to 3.2. However, this version of bash still has a significant presence in the wild and this bug is breaking an installation script, so I am looking for a POSIX-compliant (and works under Cygwin) alternative that avoids this bug so the script can be modified. As a step towards getting there, and out of plain curiosity, I would also like to know if this is due to a known bug and if so, what the underlying cause is and in what versions it has been fixed. On Wed, Jan 9, 2019 at 11:09 PM Eduardo Bustamante wrote: > On Wed, Jan 9, 2019 at 10:54 PM Jeremy wrote: > (...) > > 3.2.57(1)-release (x86_64-apple-darwin15) > > > > > > This is on MacOS 10.11.6. If there is a better place for me to report > this > > bug, please let me know. > > Have you tried the latest version? Bash 5.0 was just released: > https://lists.gnu.org/archive/html/bug-bash/2019-01/msg00063.html > > I don't think you'll be seeing any updates in the OS X provided > version of bash, so there's no point in even trying to back-port any > fixes to 3.2 > >
Re: Sourcing a script from a pipe is not reliable
On 1/10/19 2:36 PM, Jeremy wrote: > Agreed there is no likelihood of a patch to 3.2. However, this version of > bash still has a significant presence in the wild and this bug is breaking > an installation script, so I am looking for a POSIX-compliant (and works > under Cygwin) alternative that avoids this bug so the script can be > modified. Have you tried a newer version of bash? -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Sourcing a script from a pipe is not reliable
On Thu, Jan 10, 2019 at 11:36:03AM -0800, Jeremy wrote: > Agreed there is no likelihood of a patch to 3.2. However, this version of > bash still has a significant presence in the wild and this bug is breaking > an installation script, so I am looking for a POSIX-compliant (and works > under Cygwin) alternative that avoids this bug so the script can be > modified. Use a temporary file.
Re: Sourcing a script from a pipe is not reliable
On Thu, Jan 10, 2019 at 11:38 AM Chet Ramey wrote: > On 1/10/19 2:36 PM, Jeremy wrote: > > Agreed there is no likelihood of a patch to 3.2. However, this version of > > bash still has a significant presence in the wild and this bug is > breaking > > an installation script, so I am looking for a POSIX-compliant (and works > > under Cygwin) alternative that avoids this bug so the script can be > > modified. > > Have you tried a newer version of bash? > > This bug seems not to be in current versions of bash 4.x, but I have not made an exhaustive study. The thing is I do not have control over which versions of bash people use to run the script, so I need a workaround. I was hoping this was a known bug with a known cause and a known workaround that would continue to be very portable. Alternatively, if we can narrow down which versions of bash are affected, then we can consider doing a specific check for that. So far we have been able to limit this script to read operations so that it can run unprivileged on a read-only file system as it is read from a stream. That is why we are generating the script inline like this, and why I am not happy with the obvious workaround of sourcing a file instead of /dev/stdin.
Re: Sourcing a script from a pipe is not reliable
On 1/10/19 2:52 PM, Jeremy wrote: > This bug seems not to be in current versions of bash 4.x, but I have not > made an exhaustive study. The thing is I do not have control over which > versions of bash people use to run the script, so I need a workaround. I > was hoping this was a known bug with a known cause and a known workaround > that would continue to be very portable. Alternatively, if we can narrow > down which versions of bash are affected, then we can consider doing a > specific check for that. > > So far we have been able to limit this script to read operations so that it > can run unprivileged on a read-only file system as it is read from a > stream. That is why we are generating the script inline like this, and why > I am not happy with the obvious workaround of sourcing a file instead of > /dev/stdin. Have you considered reading stdin into a string (if it's one line) or an array (if it's more) and using `eval' on it? That obviously works better if it's one line, but could be made to work on multiple lines. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Sourcing a script from a pipe is not reliable
On Thu, Jan 10, 2019 at 11:59 AM Chet Ramey wrote: > On 1/10/19 2:52 PM, Jeremy wrote: > >> This command line should run forever: > > >> i=0; while [ "_$(echo echo '$1' | . /dev/stdin yes)" = "_yes" ]; \ > > do echo -n .; ((i++)); done; printf "\n%s\n" $i > > >> When I run it, it usually terminates with $i much less than 1,000. > > > >... The thing is I do not have control over which > > versions of bash people use to run the script, so I need a workaround. > > > > ... > > I am not happy with the obvious workaround of sourcing a file instead of > > /dev/stdin. > > Have you considered reading stdin into a string (if it's one line) or an > array (if it's more) and using `eval' on it? That obviously works better > if it's one line, but could be made to work on multiple lines. > > We are trying to determine if the current shell supports passing positional arguments to a script sourced by dot (.), and we are trying to do it in a way that will work under pretty much any shell. If you can show me how to do that with eval, then that would be great, even though in general I hate to ship a script that uses eval. The best workaround I have come up with so far is to generate a more complex script that outputs either "yes" or "no" and then retry if it gets no output, but then I have to put a limit on retries to avoid a possible infinite loop and handle the case where the loop terminates without an answer. My hope is that if we understand the source of the bug we can create a more straightforward work around.
Re: Sourcing a script from a pipe is not reliable
Date:Thu, 10 Jan 2019 13:52:59 -0800
From:Jeremy
Message-ID:
| We are trying to determine if the current shell supports passing positional
| arguments to a script sourced by dot (.), and we are trying to do it in a
| way that will work under pretty much any shell.
The only real way to do that is to actually write such a script, and test it.
If you cannot write the script, you're basically out of luck, tricks with
/dev/stdin are not the way forward.
But the real question is why?
If you're prepared for a "no" answer, then why not simply assume the answer
is always going to be "no" for all shells ?
If you really need "." scripts with args, you could use
rundot()
{
__F="$1"
shift
. "$__F"
}
or something similar (probably with some error checking) to
simulate processing a "." script with args. The extra cost of
calling that function
rundot script arg1 arg2 ...
rather than just
. script arg1 arg2
isn't worth worrying about, the cost of the search for script,
and opening it, etc, dwarfs everything else that would be
happening, and that is the same both ways.
kre
Re: Sourcing a script from a pipe is not reliable
Date:Thu, 10 Jan 2019 13:52:59 -0800 From:Jeremy Message-ID: ps: (intended to add this in the previous reply and forgot...) | even though in general I hate | to ship a script that uses eval. Why?eval is a prefectly proper and useful part of the shell, and should be used when appropriate. You shouldn't really be using it on arbitrary user data (or not without careful precautions) but on strings created by the script itself it is perfectly safe. kre
Re: "return" should not continue script execution, even if used inappropriately
Date:Tue, 8 Jan 2019 23:40:40 -0800 From:don fong Message-ID: | to me, your suggested wrapper script pattern seems unnatural. i don't | always want users to have to carry around 2 files (the dottable library and | the wrapper to dot it in). Then use the " sh -c '. script' " version instead. | speaking of breakage, i'd also note that your suggested pattern has a bug, | it assumes that run-dotscr will only be run from the directory where the | file lives. yes, the bug can be easily fixed. but fixing it will make | your script a bit less "simple and natural". Not really a bug, simply a simpliciation for the purposes of this e-mail exchange ... obviously you're expected to be smart enough to use the actual form that is needed to work in the environment in which the script is to be tested - whether that means using a full path name, or a PATH search, or whatever is appropriate for the situation (that is, the "./dotscr" part of the example should be the same thing you'd expect the user to type if they were to run it as an independent script. The purpose of the example was to illustrate the echnique, not to provide a ready made (complete) solution. kre
Re: Sourcing a script from a pipe is not reliable
A more reliable way to demonstrate the bug: cat <(echo echo '$1' 2) | . /dev/stdin yes Should output "yes 2", and does in bash 4.4.23, but in bash 3.2.57(1)-release it outputs nothing: the output of cat does not get sourced. Reminder, I'm seeking a workaround rather than a patch, to reliably test if the current shell supports passing positional parameters to a sourced script, without creating a temporary file. On Wed, Jan 9, 2019 at 8:11 PM Jeremy wrote: > Configuration Information [Automatically generated, do not change]: > > Machine: Mac > > OS: Darwin > > Compiler: gcc > > Compilation CFLAGS: Xcode > > uname output: Darwin Octo.local 15.6.0 Darwin Kernel Version 15.6.0: Thu > Jun 21\ > > 20:07:40 PDT 2018; root:xnu-3248.73.11~1/RELEASE_X86_64 x86_64 > > Machine Type: x86_64-Apple-Darwin > > > Bash Version: 3.2 > > Patch Level: 48 > > Release Status: relase > > > Although bashbug listed the Patch Level as 48 (and misspelled "release") > the version string is > > 3.2.57(1)-release (x86_64-apple-darwin15) > > > This is on MacOS 10.11.6. If there is a better place for me to report this > bug, please let me know. > > Description: > > Sourcing a script from a pipe is not reliable. > > > Repeat-By: > > This command line should run forever: > > > i=0; while [ "_$(echo echo '$1' | . /dev/stdin yes)" = "_yes" ]; \ > > do echo -n .; ((i++)); done; printf "\n%s\n" $i > > > When I run it, it usually terminates with $i much less than 1,000. >
Re: Sourcing a script from a pipe is not reliable
Op 10-01-19 om 22:52 schreef Jeremy:
> We are trying to determine if the current shell supports passing positional
> arguments to a script sourced by dot (.), and we are trying to do it in a
> way that will work under pretty much any shell. If you can show me how to
> do that with eval, then that would be great, even though in general I hate
> to ship a script that uses eval.
While not all shells support passing positional parameters to dot
scripts, it is very simple to make work for POSIX sh: shell functions
provide positional parameters, so wrap the dot command in one of those.
dot() {
_myscript=$1
shift
. "$_myscript"
}
# use like:
dot ./somescript.sh one two three
"Sourcing" a script read from standard input without relying on the
non-standard /dev/stdin is also very simple:
eval "$(cat)"
This does load the entire script in memory before beginning execution,
but I can't imagine that being a problem in 2019, unless the script is
generated by some infinite loop.
To get your local positional parameters for a script read from standard
input, you can combine both techniques:
dotstdin() {
eval "$(cat)"
}
The following then works as expected:
dotstdin one two three <<-'EOF'
printf "ok: %s\\n" "$@"
EOF
Note that double-quoting the "$(cat)" command substitution is essential
to avoid globally active split and glob; among other things, this makes
multiple-line scripts work.
This is all perfectly POSIX and should also work fine on bash 3.2.
Hope this helps.
- M.
Re: Sourcing a script from a pipe is not reliable
On 1/10/19 4:52 PM, Jeremy wrote: > Have you considered reading stdin into a string (if it's one line) or an > array (if it's more) and using `eval' on it? That obviously works better > if it's one line, but could be made to work on multiple lines. > > > We are trying to determine if the current shell supports passing positional > arguments to a script sourced by dot (.), and we are trying to do it in a > way that will work under pretty much any shell. Bash does, but it's not portable. > If you can show me how to > do that with eval, then that would be great, even though in general I hate > to ship a script that uses eval. You have to do it yourself the old-fashioned way, using `set', if you want to restrict yourself to using `eval'. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Sourcing a script from a pipe is not reliable
I appreciate the help, Martijn, and I suspect I will find your suggestions useful in the future, but they do not completely work for us in the present instance. We are, in part, trying to determine if our script may have lost its positional parameters because it was invoked by someone else using ". ourscript --options" in a shell that does not support that, so it does not help to know they could have done it better some other way. On Thu, Jan 10, 2019 at 3:29 PM Martijn Dekker wrote: > Op 10-01-19 om 22:52 schreef Jeremy: > > We are trying to determine if the current shell supports passing > positional > > arguments to a script sourced by dot (.), and we are trying to do it in a > > way that will work under pretty much any shell. If you can show me how > to > > do that with eval, then that would be great, even though in general I > hate > > to ship a script that uses eval. > > While not all shells support passing positional parameters to dot > scripts, it is very simple to make work for POSIX sh: shell functions > provide positional parameters, so wrap the dot command in one of those. > ... >
