Re: posix command search and execution
On Wed, Nov 08, 2023 at 10:42:20AM -0500, Chet Ramey wrote: > On 11/7/23 5:37 PM, Mike Jonkmans wrote: > > On Tue, Nov 07, 2023 at 11:49:25AM -0500, Chet Ramey wrote: > > > On 11/7/23 8:54 AM, Mike Jonkmans wrote: > > So the discussion is hidden. Hmm. > Not hidden; if you look at the open group's mailing list archives, you > can find the messages. I just don't consider the interface usable. It's > barely searchable. Ah, thanks. I'll have a look later. > > > > Then again, is there a requirement for the standard utilities to be > > > > found in the current PATH? Or do they just need to be present somewhere. > > > They have to be findable using the value returned by `getconf PATH'. If > > > the user modifies PATH to, say, prepend directories before that standard > > > PATH, then all bets are off. > > I see. Weirdly on Ubuntu 22.04, with /bin symlinked to /usr/bin, > > `getconf PATH' produces `/bin:/usr/bin'. > > That looks like a recipe for redundant `stats'. > Does that matter? The value getconf returns is static, and is guaranteed > to find all the standard utilities regardless of what the file system > looks like. Maybe it matters. If I am not mistaken, for POSIX compliance, both /bin and /usr/bin have to be in PATH (see quote from Robert). A naïve implementation, doing a PATH search, might be calling `stat' for both directories, whilst they are symlinked. This might cost some performance. [ Just checked: bash stats both. ] Symlinking also meddles with the description below. Relevant quote from Robert's mail: It is actually messier that it first seems, if you have a builtin command, it isn't simply finding that command's name in a PATH search that is required, but that the directory (from PATH) in which it was found be the one "associated" with the builtin command (how exactly that is determined, or what it really even means is not specified anywhere). The effect is that if a shell with a builtin "test" believes that the associated directory is /bin and someone's path is /usr/bin:/bin - and there is a test command in /usr/bin (even if /usr/bin and /bin are the same directory, or /bin/name and /usr/bin/name are linked and so invoke the same thing) then the filesystem command is supposed to be invoked rather than the builtin.There are reasons for that, but they're really fairly stupid. This is more complex than it needs to be (and should be). For a user it is very hard to have any confidence in the posix-ness of their scripts. PS. Remember this `nn'/`Pnews' message? "Your message will cost the net hundreds if not thousands of dollars to send everywhere." What would the cost of all these extra `stat' calls be? ;) -- Regards, Mike Jonkmans
Re: posix command search and execution
Date:Thu, 9 Nov 2023 14:21:35 +0100 From:Mike Jonkmans Message-ID: <20231109132135.ga208...@jonkmans.nl> | If I am not mistaken, for POSIX compliance, both /bin and /usr/bin have | to be in PATH (see quote from Robert). No, I didn't say that, there are no particular required directory names, just that PATH needs to include whatever directories contain all the standard utilities ... getconf() returns that path. I used those names just as an example. kre
Re: posix command search and execution
On Thu, Nov 09, 2023 at 10:12:06PM +0700, Robert Elz wrote: > Date:Thu, 9 Nov 2023 14:21:35 +0100 > From:Mike Jonkmans > Message-ID: <20231109132135.ga208...@jonkmans.nl> > > | If I am not mistaken, for POSIX compliance, both /bin and /usr/bin have > | to be in PATH (see quote from Robert). > > No, I didn't say that, there are no particular required directory > names, just that PATH needs to include whatever directories contain > all the standard utilities ... getconf() returns that path. > > I used those names just as an example. Ah, that is clear then. On Ubuntu 22.04 `getconf path' returns /bin:/usr/bin and these are symlinked. -- Regards, Mike Jonkmans
Re: Idea: jobs(1) -i to print only :%ID:s
Steffen Nurpmeso wrote in
<20231109011212.tc9hj%stef...@sdaoden.eu>:
...
Something like this that would be, adding JLIST_SPEC_ONLY and
jobs(1) -j. Just in case of interest.
diff --git a/builtins/jobs.def b/builtins/jobs.def
index 1ce098d08b..989a78079e 100644
--- a/builtins/jobs.def
+++ b/builtins/jobs.def
@@ -23,13 +23,14 @@ $PRODUCES jobs.c
$BUILTIN jobs
$FUNCTION jobs_builtin
$DEPENDS_ON JOB_CONTROL
-$SHORT_DOC jobs [-lnprs] [jobspec ...] or jobs -x command [args]
+$SHORT_DOC jobs [-jlnprs] [jobspec ...] or jobs -x command [args]
Display status of jobs.
Lists the active jobs. JOBSPEC restricts output to that job.
Without options, the status of all active jobs is displayed.
Options:
+ -j lists only jobspecs
-l lists process IDs in addition to the normal information
-n lists only processes that have changed status since the last
notification
@@ -90,10 +91,13 @@ jobs_builtin (list)
state = JSTATE_ANY;
reset_internal_getopt ();
- while ((opt = internal_getopt (list, "lpnxrs")) != -1)
+ while ((opt = internal_getopt (list, "jlpnxrs")) != -1)
{
switch (opt)
{
+ case 'j':
+ form = JLIST_SPEC_ONLY;
+ break;
case 'l':
form = JLIST_LONG;
break;
diff --git a/doc/bash.1 b/doc/bash.1
index 55c562208a..8dc34a00d7 100644
--- a/doc/bash.1
+++ b/doc/bash.1
@@ -9150,7 +9150,7 @@ error occurs while reading or writing the history file,
an invalid
history expansion supplied as an argument to \fB\-p\fP fails.
.RE
.TP
-\fBjobs\fP [\fB\-lnprs\fP] [ \fIjobspec\fP ... ]
+\fBjobs\fP [\fB\-jlnprs\fP] [ \fIjobspec\fP ... ]
.PD 0
.TP
\fBjobs\fP \fB\-x\fP \fIcommand\fP [ \fIargs\fP ... ]
@@ -9160,6 +9160,9 @@ meanings:
.RS
.PD 0
.TP
+.B \-j
+List only the \fIjobspec\fPs.
+.TP
.B \-l
List process IDs
in addition to the normal information.
diff --git a/jobs.c b/jobs.c
index 45869dd819..b56cecccb3 100644
--- a/jobs.c
+++ b/jobs.c
@@ -2037,6 +2037,13 @@ pretty_print_job (job_index, format, stream)
{
register PROCESS *p;
+ /* Format only jobspec? */
+ if (format == JLIST_SPEC_ONLY)
+{
+ fprintf (stream, "%%%d\n", job_index + 1);
+ return;
+}
+
/* Format only pid information about the process group leader? */
if (format == JLIST_PID_ONLY)
{
diff --git a/jobs.h b/jobs.h
index 276204f0c7..5d1cc43d1f 100644
--- a/jobs.h
+++ b/jobs.h
@@ -32,8 +32,9 @@
#define JLIST_STANDARD 0
#define JLIST_LONG 1
#define JLIST_PID_ONLY 2
-#define JLIST_CHANGED_ONLY 3
-#define JLIST_NONINTERACTIVE 4
+#define JLIST_SPEC_ONLY 3
+#define JLIST_CHANGED_ONLY 4
+#define JLIST_NONINTERACTIVE 5
/* I looked it up. For pretty_print_job (). The real answer is 24. */
#define LONGEST_SIGNAL_DESC 24
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: Idea: jobs(1) -i to print only :%ID:s
Steffen Nurpmeso wrote in <20231109181107.bj0wl%stef...@sdaoden.eu>: |Steffen Nurpmeso wrote in | <20231109011212.tc9hj%stef...@sdaoden.eu>: | ... |Something like this that would be, adding JLIST_SPEC_ONLY and |jobs(1) -j. Just in case of interest. I mean some scripting on "jobs | wc -l" would do that, though. :( Maybe i should just write a function that builds the string necessary to do what i wanted with %* or "%1-2 %4" etc. Eh. Forget about it. Ciao! --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Idea: jobs(1) -i to print only :%ID:s
On Thu, Nov 9, 2023, 7:17 PM Steffen Nurpmeso wrote:
> Steffen Nurpmeso wrote in
> <20231109181107.bj0wl%stef...@sdaoden.eu>:
> |Steffen Nurpmeso wrote in
> | <20231109011212.tc9hj%stef...@sdaoden.eu>:
> | ...
> |Something like this that would be, adding JLIST_SPEC_ONLY and
> |jobs(1) -j. Just in case of interest.
>
> I mean some scripting on "jobs | wc -l" would do that, though. :(
> Maybe i should just write a function that builds the string
> necessary to do what i wanted with %* or "%1-2 %4" etc.
> Eh. Forget about it.
>
> Ciao!
>
jobs wc is
j=( $( jobs -p ) )
or ${ ..
i think its with ' kinds ' cooperative
greets
-nlists only processes that have changed status since the last
notification
-plists process IDs only
-rrestrict output to running jobs
-srestrict output to stopped jobs
--steffen
> |
> |Der Kragenbaer,The moon bear,
> |der holt sich munter he cheerfully and one by one
> |einen nach dem anderen runter wa.ks himself off
> |(By Robert Gernhardt)
>
>
Re: Idea: jobs(1) -i to print only :%ID:s
On Thu, Nov 9, 2023, 7:21 PM alex xmb sw ratchev wrote:
>
>
> On Thu, Nov 9, 2023, 7:17 PM Steffen Nurpmeso wrote:
>
>> Steffen Nurpmeso wrote in
>> <20231109181107.bj0wl%stef...@sdaoden.eu>:
>> |Steffen Nurpmeso wrote in
>> | <20231109011212.tc9hj%stef...@sdaoden.eu>:
>> | ...
>> |Something like this that would be, adding JLIST_SPEC_ONLY and
>> |jobs(1) -j. Just in case of interest.
>>
>> I mean some scripting on "jobs | wc -l" would do that, though. :(
>> Maybe i should just write a function that builds the string
>> necessary to do what i wanted with %* or "%1-2 %4" etc.
>> Eh. Forget about it.
>>
>> Ciao!
>>
>
> jobs wc is
>
> j=( $( jobs -p ) )
>
jc=${#j[*]}
or ${ ..
>
> i think its with ' kinds ' cooperative
>
> greets
>
> -nlists only processes that have changed status since the
> last
> notification
> -plists process IDs only
> -rrestrict output to running jobs
> -srestrict output to stopped jobs
>
> --steffen
>> |
>> |Der Kragenbaer,The moon bear,
>> |der holt sich munter he cheerfully and one by one
>> |einen nach dem anderen runter wa.ks himself off
>> |(By Robert Gernhardt)
>>
>>
>>
Re: Idea: jobs(1) -i to print only :%ID:s
Steffen Nurpmeso wrote in
<20231109181645.bocyg%stef...@sdaoden.eu>:
|Steffen Nurpmeso wrote in
| <20231109181107.bj0wl%stef...@sdaoden.eu>:
||Steffen Nurpmeso wrote in
|| <20231109011212.tc9hj%stef...@sdaoden.eu>:
|| ...
||Something like this that would be, adding JLIST_SPEC_ONLY and
||jobs(1) -j. Just in case of interest.
|
|I mean some scripting on "jobs | wc -l" would do that, though. :(
Nah, that would be wrong.
|Maybe i should just write a function that builds the string
|necessary to do what i wanted with %* or "%1-2 %4" etc.
So i did that (what a mess -- does anyone know how i can create an
awk regular expression where parts of the expression is a variable
that should be expanded? ugh! what do i know??):
j() {
local j= a=${AWK:-awk}
[ $# -gt 0 ] && j='&& $2 !~ /(^| )('$(echo "$@" | tr ' ' '|')')( |$)/'
j=$(jobs -l | $a -F '[][]' '/^[[]/'"$j"'{print "%" $2}{next}')
echo $j
}
|Eh. Forget about it.
Sorry for the noise.
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: Idea: jobs(1) -i to print only :%ID:s
On Thu, Nov 09, 2023 at 08:09:23PM +0100, Steffen Nurpmeso wrote:
> j() {
> local j= a=${AWK:-awk}
> [ $# -gt 0 ] && j='&& $2 !~ /(^| )('$(echo "$@" | tr ' ' '|')')( |$)/'
> j=$(jobs -l | $a -F '[][]' '/^[[]/'"$j"'{print "%" $2}{next}')
> echo $j
> }
Classic code injection vulnerability.
What are we even parsing? Start with the input:
unicorn:~$ sleep 5 &
[1] 849028
unicorn:~$ jobs -l
[1]+ 849028 Running sleep 5 &
OK, so you wanted to strip the "1" from "[1]" and turn that into "%1",
yes? That shouldn't be terribly hard in pure bash.
re='^\[([0-9]+)\]'
jobspecs=()
while IFS= read -r line; do
if [[ $line =~ $re ]]; then
jobspecs+=( "%${BASH_REMATCH[1]}" )
fi
done < <(jobs -l)
Wrap that in a function with local declarations, etc.
Re: Idea: jobs(1) -i to print only :%ID:s
On Thu, Nov 9, 2023, 8:10 PM Steffen Nurpmeso wrote:
> Steffen Nurpmeso wrote in
> <20231109181645.bocyg%stef...@sdaoden.eu>:
> |Steffen Nurpmeso wrote in
> | <20231109181107.bj0wl%stef...@sdaoden.eu>:
> ||Steffen Nurpmeso wrote in
> || <20231109011212.tc9hj%stef...@sdaoden.eu>:
> || ...
> ||Something like this that would be, adding JLIST_SPEC_ONLY and
> ||jobs(1) -j. Just in case of interest.
> |
> |I mean some scripting on "jobs | wc -l" would do that, though. :(
>
> Nah, that would be wrong.
>
> |Maybe i should just write a function that builds the string
> |necessary to do what i wanted with %* or "%1-2 %4" etc.
>
> So i did that (what a mess -- does anyone know how i can create an
> awk regular expression where parts of the expression is a variable
> that should be expanded? ugh! what do i know??):
>
awk -v var1='cont ent' -vv2="$other' ' { code } '
keep in mind that mass spawning of any bin is , when its done massley ,
very out ..
also var so awk processes some \esc caracters
to cleanly insert data , either use gawk ENVIRON["varname']
or via alternative fd or as part of file parse things
my='a b c'
# one line string
awk 3<<<"$my thing' BEGIN { f = "/dev/fd/3" ; getline v local j= a=${AWK:-awk}
> [ $# -gt 0 ] && j='&& $2 !~ /(^| )('$(echo "$@" | tr ' ' '|')')( |$)/'
> j=$(jobs -l | $a -F '[][]' '/^[[]/'"$j"'{print "%" $2}{next}')
> echo $j
> }
>
> |Eh. Forget about it.
>
> Sorry for the noise.
>
> --steffen
> |
> |Der Kragenbaer,The moon bear,
> |der holt sich munter he cheerfully and one by one
> |einen nach dem anderen runter wa.ks himself off
> |(By Robert Gernhardt)
>
>
Re: Idea: jobs(1) -i to print only :%ID:s
alex xmb sw ratchev wrote in
:
...
|> So i did that (what a mess -- does anyone know how i can create an
|> awk regular expression where parts of the expression is a variable
|> that should be expanded? ugh! what do i know??):
|
|awk -v var1='cont ent' -vv2="$other' ' { code } '
|
|keep in mind that mass spawning of any bin is , when its done massley ,
|very out ..
i fail to see what this has to do with my problem.
|also var so awk processes some \esc caracters
|to cleanly insert data , either use gawk ENVIRON["varname']
|or via alternative fd or as part of file parse things
|my='a b c'
|# one line string
|awk 3<<<"$my thing' BEGIN { f = "/dev/fd/3" ; getline v
Re: Idea: jobs(1) -i to print only :%ID:s
On Thu, Nov 9, 2023, 8:36 PM Steffen Nurpmeso wrote:
> alex xmb sw ratchev wrote in
> :
> ...
> |> So i did that (what a mess -- does anyone know how i can create an
> |> awk regular expression where parts of the expression is a variable
> |> that should be expanded? ugh! what do i know??):
> |
> |awk -v var1='cont ent' -vv2="$other' ' { code } '
> |
> |keep in mind that mass spawning of any bin is , when its done massley ,
> |very out ..
>
> i fail to see what this has to do with my problem.
>
> |also var so awk processes some \esc caracters
> |to cleanly insert data , either use gawk ENVIRON["varname']
> |or via alternative fd or as part of file parse things
>
> |my='a b c'
> |# one line string
> |awk 3<<<"$my thing' BEGIN { f = "/dev/fd/3" ; getline v |other main code here '
>
> i am still wondering.
>
> |foo=content
> |export foo
> |or foo=$foo gawk ..
> |gawk -vv=foo ' BEGIN { v = ENVIRON[ v ] } other code '
> |same as v = ENVIRON[ "v" ] or "varname"
>
> wow.
>
> |to match in string , thers two ways , one is exact text match , the other
> |is regex
> |
> |if u /foo/ thats a regex
>
> really?
>
> |$0 ~ "foo"
> |
> |tx=" my text "
> |if ( $0 ~ "pre" tx "stu" ) { ye }
>
> this does not work the way you think maybe?
> I surely was there myself.
>
> |thers additional stuff to say containing substr() and match() , and some
>
> even with match() not (in a global match line).
>
> |others
> |
> |#exact
> |etx = " exact "
> |if ( index( $0 , etx ) ) { ye }
> |if ( index( $0 , "pre" etx "stu" ) ) { ye }
>
> i am really impressed.
> But that did not work out.
>
can u detail ur case some
i suggest using index
if u match or ~ u regex
means ur input s gotta get regex char escaped
unless u have a very special reason , use index
and not care about awk var \esc seq expandment
at least when have simple strings
eg
m=bar txt=' foo
bar
bla '
awk -vm="$m" ' index( $0 , m ) '
should return the bar line
' bar ' would also match
' bar ' of course not
--steffen
> |
> |Der Kragenbaer,The moon bear,
> |der holt sich munter he cheerfully and one by one
> |einen nach dem anderen runter wa.ks himself off
> |(By Robert Gernhardt)
>
Re: Idea: jobs(1) -i to print only :%ID:s
On Thu, Nov 9, 2023, 8:41 PM alex xmb sw ratchev wrote:
>
>
> On Thu, Nov 9, 2023, 8:36 PM Steffen Nurpmeso wrote:
>
>> alex xmb sw ratchev wrote in
>> :
>> ...
>> |> So i did that (what a mess -- does anyone know how i can create an
>> |> awk regular expression where parts of the expression is a variable
>> |> that should be expanded? ugh! what do i know??):
>> |
>> |awk -v var1='cont ent' -vv2="$other' ' { code } '
>> |
>> |keep in mind that mass spawning of any bin is , when its done massley ,
>> |very out ..
>>
>> i fail to see what this has to do with my problem.
>>
>> |also var so awk processes some \esc caracters
>> |to cleanly insert data , either use gawk ENVIRON["varname']
>> |or via alternative fd or as part of file parse things
>>
>> |my='a b c'
>> |# one line string
>> |awk 3<<<"$my thing' BEGIN { f = "/dev/fd/3" ; getline v > }
>> |other main code here '
>>
>> i am still wondering.
>>
>> |foo=content
>> |export foo
>> |or foo=$foo gawk ..
>> |gawk -vv=foo ' BEGIN { v = ENVIRON[ v ] } other code '
>> |same as v = ENVIRON[ "v" ] or "varname"
>>
>> wow.
>>
>> |to match in string , thers two ways , one is exact text match , the
>> other
>> |is regex
>> |
>> |if u /foo/ thats a regex
>>
>> really?
>>
>> |$0 ~ "foo"
>> |
>> |tx=" my text "
>> |if ( $0 ~ "pre" tx "stu" ) { ye }
>>
>> this does not work the way you think maybe?
>> I surely was there myself.
>>
>> |thers additional stuff to say containing substr() and match() , and some
>>
>> even with match() not (in a global match line).
>>
>> |others
>> |
>> |#exact
>> |etx = " exact "
>> |if ( index( $0 , etx ) ) { ye }
>> |if ( index( $0 , "pre" etx "stu" ) ) { ye }
>>
>> i am really impressed.
>> But that did not work out.
>>
>
> can u detail ur case some
> i suggest using index
> if u match or ~ u regex
> means ur input s gotta get regex char escaped
> unless u have a very special reason , use index
> and not care about awk var \esc seq expandment
> at least when have simple strings
>
> eg
>
> m=bar txt=' foo
> bar
> bla '
> awk -vm="$m" ' index( $0 , m ) '
>
sorry missed again a detail
awk -vm="$m" ' index( $0 , m ) ' <<<"$txt"
should return the bar line
> ' bar ' would also match
> ' bar ' of course not
>
> --steffen
>> |
>> |Der Kragenbaer,The moon bear,
>> |der holt sich munter he cheerfully and one by one
>> |einen nach dem anderen runter wa.ks himself off
>> |(By Robert Gernhardt)
>>
>
Re: Idea: jobs(1) -i to print only :%ID:s
On Thu, Nov 9, 2023, 8:24 PM Greg Wooledge wrote:
> On Thu, Nov 09, 2023 at 08:09:23PM +0100, Steffen Nurpmeso wrote:
> > j() {
> > local j= a=${AWK:-awk}
> > [ $# -gt 0 ] && j='&& $2 !~ /(^| )('$(echo "$@" | tr ' ' '|')')(
> |$)/'
> > j=$(jobs -l | $a -F '[][]' '/^[[]/'"$j"'{print "%" $2}{next}')
> > echo $j
> > }
>
> Classic code injection vulnerability.
>
> What are we even parsing? Start with the input:
>
> unicorn:~$ sleep 5 &
> [1] 849028
> unicorn:~$ jobs -l
> [1]+ 849028 Running sleep 5 &
>
> OK, so you wanted to strip the "1" from "[1]" and turn that into "%1",
> yes? That shouldn't be terribly hard in pure bash.
>
> re='^\[([0-9]+)\]'
> jobspecs=()
> while IFS= read -r line; do
> if [[ $line =~ $re ]]; then
> jobspecs+=( "%${BASH_REMATCH[1]}" )
> fi
> done < <(jobs -l)
>
sleep 500h & sleep 400h &
~ $ s=7s jid=( ) jpid=( ) ; while IFS=$'\n' j=( $( jobs -l ) ) je=${#j[*]}
IFS=$' \t\n' ; do ji=-1 ; while (( ++ji < je )) ; do t=( ${j[ji]} ) jid+=(
"${t//[^0-9]}" ) jpid+=( "${t[1]}" ) ; done ; declare -p j{e,{,p}id} ;
sleep "$s" ; done
declare -- je="2"
declare -a jid=([0]="1" [1]="2")
declare -a jpid=([0]="21003" [1]="21009")
Wrap that in a function with local declarations, etc.
>
>
Re: Idea: jobs(1) -i to print only :%ID:s
Greg Wooledge wrote in
:
|On Thu, Nov 09, 2023 at 08:09:23PM +0100, Steffen Nurpmeso wrote:
|> j() {
|> local j= a=${AWK:-awk}
|> [ $# -gt 0 ] && j='&& $2 !~ /(^| )('$(echo "$@" | tr ' ' '|')')( \
|> |$)/'
|> j=$(jobs -l | $a -F '[][]' '/^[[]/'"$j"'{print "%" $2}{next}')
|> echo $j
|>}
|
|Classic code injection vulnerability.
Well this is for me on my command line. ash(1) and busybox ash(1)
do not even support jobs -l in a function (yet). (Where??)
|What are we even parsing? Start with the input:
|
|unicorn:~$ sleep 5 &
|[1] 849028
|unicorn:~$ jobs -l
|[1]+ 849028 Running sleep 5 &
|
|OK, so you wanted to strip the "1" from "[1]" and turn that into "%1",
|yes? That shouldn't be terribly hard in pure bash.
|
|re='^\[([0-9]+)\]'
|jobspecs=()
|while IFS= read -r line; do
|if [[ $line =~ $re ]]; then
This is indeed a nice approach. I have now spend over an hour
stupid me, and have no more time to extend this to filter out the
unwanted jobs, but i copy your thing over to my ~/.profile for
a future full implementation! (Likely selectively after testing
which shells actually do support this syntax, ie ksh and such.)
|jobspecs+=( "%${BASH_REMATCH[1]}" )
|fi
|done < <(jobs -l)
|
|Wrap that in a function with local declarations, etc.
Thanks!
Ciao,
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: Idea: jobs(1) -i to print only :%ID:s
On Nov 09 2023, Greg Wooledge wrote:
> re='^\[([0-9]+)\]'
> jobspecs=()
> while IFS= read -r line; do
> if [[ $line =~ $re ]]; then
> jobspecs+=( "%${BASH_REMATCH[1]}" )
> fi
> done < <(jobs -l)
That fails for multi-line commands that happen to contain matches for
re.
$ (sleep 100; printf $'\n[100]\n') &
--
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
"And now for something completely different."
Re: Idea: jobs(1) -i to print only :%ID:s
Date:Thu, 09 Nov 2023 21:04:28 +0100 From:Steffen Nurpmeso Message-ID: <20231109200428.8g9lz%stef...@sdaoden.eu> | ash(1) and busybox ash(1) do not even support jobs -l in a function (yet). That seems unlikely, ash based shells mostly don't care whether the commands they're running are in a function or not, except for those few commands for which that is important (like return, local, etc). Are you sure that you're not seeing the effects of running jobs in a subshell environment (in a command substitution) - where no jobs have yet been created in that environment. Modern shells allow jobs (like trap) there, as a way to get the parent shell's jobs (or traps), but even now (in the latest drafts) POSIX does not require that jobs works that way (it does for trap, in very limited cases). kre
Re: Idea: jobs(1) -i to print only :%ID:s
On Thu, Nov 9, 2023, 10:18 PM Andreas Schwab wrote:
> On Nov 09 2023, Greg Wooledge wrote:
>
> > re='^\[([0-9]+)\]'
> > jobspecs=()
> > while IFS= read -r line; do
> > if [[ $line =~ $re ]]; then
> > jobspecs+=( "%${BASH_REMATCH[1]}" )
> > fi
> > done < <(jobs -l)
>
> That fails for multi-line commands that happen to contain matches for
> re.
>
> $ (sleep 100; printf $'\n[100]\n') &
>
a suggestion for the future , that bash coding keywords have own isf osf
var following , so i can have null or other separated , by code builtins ..
--
> Andreas Schwab, sch...@linux-m68k.org
> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
> "And now for something completely different."
>
>
Bash-5.2 patch 16
BASH PATCH REPORT = Bash-Release: 5.2 Patch-ID: bash52-016 Bug-Reported-by:F G Bug-Reference-ID: Bug-Reference-URL: Bug-Description: If an expression in an arithmetic for loop expands to NULL, the shell would crash. Patch (apply with `patch -p0'): *** ../bash-5.2-patched/execute_cmd.c Thu Feb 23 14:15:05 2023 --- execute_cmd.c Mon Feb 27 17:53:08 2023 *** *** 3051,3055 if (l->next) free (expr); ! new = make_word_list (make_word (temp), (WORD_LIST *)NULL); free (temp); --- 3051,3055 if (l->next) free (expr); ! new = make_word_list (make_word (temp ? temp : ""), (WORD_LIST *)NULL); free (temp); *** ../bash-5.2/patchlevel.h2020-06-22 14:51:03.0 -0400 --- patchlevel.h2020-10-01 11:01:28.0 -0400 *** *** 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 15 #endif /* _PATCHLEVEL_H_ */ --- 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 16 #endif /* _PATCHLEVEL_H_ */ -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Bash-5.2 patch 17
BASH PATCH REPORT = Bash-Release: 5.2 Patch-ID: bash52-017 Bug-Reported-by:Dan Church Bug-Reference-ID: <1a8fd1d6-a3ac-9a67-78eb-b9a743530...@gmx.com> Bug-Reference-URL: https://lists.gnu.org/archive/html/bug-bash/2022-12/msg00076.html Bug-Description: In certain cases, using the `.' builtin in a subshell would optimize away the rest of the commands in the subshell. Patch (apply with `patch -p0'): *** ../bash-5.2-patched/builtins/evalfile.c 2019-07-20 16:16:08.0 -0400 --- builtins/evalfile.c 2022-12-22 12:13:08.0 -0500 *** *** 267,271 /* set the flags to be passed to parse_and_execute */ ! pflags = SEVAL_RESETLINE; pflags |= (flags & FEVAL_HISTORY) ? 0 : SEVAL_NOHIST; --- 267,271 /* set the flags to be passed to parse_and_execute */ ! pflags = SEVAL_RESETLINE|SEVAL_NOOPTIMIZE; pflags |= (flags & FEVAL_HISTORY) ? 0 : SEVAL_NOHIST; *** ../bash-5.2/patchlevel.h2020-06-22 14:51:03.0 -0400 --- patchlevel.h2020-10-01 11:01:28.0 -0400 *** *** 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 16 #endif /* _PATCHLEVEL_H_ */ --- 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 17 #endif /* _PATCHLEVEL_H_ */ -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Bash-5.2 patch 19
BASH PATCH REPORT = Bash-Release: 5.2 Patch-ID: bash52-019 Bug-Reported-by:Steffen Nurpmeso Bug-Reference-ID: <20230116233547.2jfxl%stef...@sdaoden.eu> Bug-Reference-URL: https://lists.gnu.org/archive/html/bug-bash/2023-01/msg00057.html Bug-Description: There are some cases where the shell reaped a background (asynchronous) job and would incorrectly try to set the terminal's process group back to the shell's. In these cases it never set the terminal process group to that jobs's process group initially, so resetting it is incorrect. Patch (apply with `patch -p0'): *** ../bash-5.2-patched/jobs.c 2022-12-13 12:09:02.0 -0500 --- jobs.c 2023-10-26 12:12:10.0 -0400 *** *** 3078,3084 subshell. Make sure subst.c:command_substitute uses the same conditions to determine whether or not it should undo this and !give the terminal to pipeline_pgrp. */ ! if ((flags & JWAIT_NOTERM) == 0 && running_in_background == 0 && (subshell_environment & (SUBSHELL_ASYNC|SUBSHELL_PIPE)) == 0) give_terminal_to (shell_pgrp, 0); --- 3036,3046 subshell. Make sure subst.c:command_substitute uses the same conditions to determine whether or not it should undo this and !give the terminal to pipeline_pgrp. We don't give the terminal !back to shell_pgrp if an async job in the background exits because !we never gave it to that job in the first place. An async job in !the foreground is one we started in the background and foregrounded !with `fg', and gave it the terminal. */ if ((flags & JWAIT_NOTERM) == 0 && running_in_background == 0 && + (job == NO_JOB || IS_ASYNC (job) == 0 || IS_FOREGROUND (job)) && (subshell_environment & (SUBSHELL_ASYNC|SUBSHELL_PIPE)) == 0) give_terminal_to (shell_pgrp, 0); *** *** 3624,3627 --- 3599,3603 get_tty_state (); save_stty = shell_tty_info; + jobs[job]->flags &= ~J_ASYNC; /* no longer async */ /* Give the terminal to this job. */ if (IS_JOBCONTROL (job)) *** ../bash-5.2/patchlevel.h2020-06-22 14:51:03.0 -0400 --- patchlevel.h2020-10-01 11:01:28.0 -0400 *** *** 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 18 #endif /* _PATCHLEVEL_H_ */ --- 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 19 #endif /* _PATCHLEVEL_H_ */ -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Bash-5.2 patch 20
BASH PATCH REPORT = Bash-Release: 5.2 Patch-ID: bash52-020 Bug-Reported-by:Dima Korobskiy Bug-Reference-ID: <16664c2d-40ec-df33-b932-83db06e39...@gmail.com> Bug-Reference-URL: https://lists.gnu.org/archive/html/bug-bash/2023-08/msg00125.html Bug-Description: The parser did not allow `time' to appear as the first reserved word in a command substitution. Patch (apply with `patch -p0'): *** ../bash-5.2-patched/parse.y Tue Dec 13 12:53:21 2022 --- parse.y Fri Sep 1 10:36:28 2023 *** *** 3151,3154 --- 3151,3155 case TIMEOPT: /* time -p time pipeline */ case TIMEIGN: /* time -p -- ... */ + case DOLPAREN: return 1; default: *** ../bash-5.2-patched/y.tab.c Tue Dec 13 12:53:21 2022 --- y.tab.c Fri Sep 1 10:36:44 2023 *** *** 5466,5469 --- 5466,5470 case TIMEOPT: /* time -p time pipeline */ case TIMEIGN: /* time -p -- ... */ + case DOLPAREN: return 1; default: *** ../bash-5.2/patchlevel.h2020-06-22 14:51:03.0 -0400 --- patchlevel.h2020-10-01 11:01:28.0 -0400 *** *** 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 19 #endif /* _PATCHLEVEL_H_ */ --- 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 20 #endif /* _PATCHLEVEL_H_ */ -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Bash-5.2 patch 21
BASH PATCH REPORT = Bash-Release: 5.2 Patch-ID: bash52-021 Bug-Reported-by:Norbert Lange Bug-Reference-ID: Bug-Reference-URL: https://lists.gnu.org/archive/html/bug-bash/2022-12/msg00046.html Bug-Description: There is an off-by-one error that causes command substitutions to fail when they appear in a word expansion inside a here-document. Patch (apply with `patch -p0'): *** ../bash-5.2-patched/subst.c 2022-12-13 12:08:58.0 -0500 --- subst.c 2022-12-14 09:09:53.0 -0500 *** *** 1694,1698 CHECK_STRING_OVERRUN (i, si, slen, c); ! tlen = si - i - 1; RESIZE_MALLOCED_BUFFER (result, result_index, tlen + 4, result_size, 64); result[result_index++] = c; --- 1699,1703 CHECK_STRING_OVERRUN (i, si, slen, c); ! tlen = si - i - 2; RESIZE_MALLOCED_BUFFER (result, result_index, tlen + 4, result_size, 64); result[result_index++] = c; *** *** 1714,1718 CHECK_STRING_OVERRUN (i, si, slen, c); ! tlen = si - i - 1; RESIZE_MALLOCED_BUFFER (result, result_index, tlen + 4, result_size, 64); result[result_index++] = c; --- 1719,1723 CHECK_STRING_OVERRUN (i, si, slen, c); ! tlen = si - i - 2; RESIZE_MALLOCED_BUFFER (result, result_index, tlen + 4, result_size, 64); result[result_index++] = c; *** ../bash-5.2/patchlevel.h2020-06-22 14:51:03.0 -0400 --- patchlevel.h2020-10-01 11:01:28.0 -0400 *** *** 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 20 #endif /* _PATCHLEVEL_H_ */ --- 26,30 looks for to find the patch level (for the sccs version string). */ ! #define PATCHLEVEL 21 #endif /* _PATCHLEVEL_H_ */ -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Idea: jobs(1) -i to print only :%ID:s
On Thu, Nov 09, 2023 at 10:17:35PM +0100, Andreas Schwab wrote:
> On Nov 09 2023, Greg Wooledge wrote:
>
> > re='^\[([0-9]+)\]'
> > jobspecs=()
> > while IFS= read -r line; do
> > if [[ $line =~ $re ]]; then
> > jobspecs+=( "%${BASH_REMATCH[1]}" )
> > fi
> > done < <(jobs -l)
>
> That fails for multi-line commands that happen to contain matches for
> re.
>
> $ (sleep 100; printf $'\n[100]\n') &
Fair enough. I believe *nothing* would work in that case; the output of
"jobs -l" cannot be safely parsed in its current form.
Adding a NUL-delimited output option might work around that, but if we're
modifying the jobs builtin, we might as well just add the -i or -j option
instead.
Re: Idea: jobs(1) -i to print only :%ID:s
On Thu, Nov 9, 2023, 10:56 PM Greg Wooledge wrote:
> On Thu, Nov 09, 2023 at 10:17:35PM +0100, Andreas Schwab wrote:
> > On Nov 09 2023, Greg Wooledge wrote:
> >
> > > re='^\[([0-9]+)\]'
> > > jobspecs=()
> > > while IFS= read -r line; do
> > > if [[ $line =~ $re ]]; then
> > > jobspecs+=( "%${BASH_REMATCH[1]}" )
> > > fi
> > > done < <(jobs -l)
> >
> > That fails for multi-line commands that happen to contain matches for
> > re.
> >
> > $ (sleep 100; printf $'\n[100]\n') &
>
> Fair enough. I believe *nothing* would work in that case; the output of
> "jobs -l" cannot be safely parsed in its current form.
>
> Adding a NUL-delimited output option might work around that, but if we're
> modifying the jobs builtin, we might as well just add the -i or -j option
> instead.
>
ther'd be few different values for seps , i think ifs is nearly none , so ,
one is per element , one per record
others may also exist
so just two constant opts to align extend usage
>
Re: Idea: jobs(1) -i to print only :%ID:s
On Thu, Nov 9, 2023, 11:08 PM alex xmb sw ratchev wrote:
>
>
> On Thu, Nov 9, 2023, 10:56 PM Greg Wooledge wrote:
>
>> On Thu, Nov 09, 2023 at 10:17:35PM +0100, Andreas Schwab wrote:
>> > On Nov 09 2023, Greg Wooledge wrote:
>> >
>> > > re='^\[([0-9]+)\]'
>> > > jobspecs=()
>> > > while IFS= read -r line; do
>> > > if [[ $line =~ $re ]]; then
>> > > jobspecs+=( "%${BASH_REMATCH[1]}" )
>> > > fi
>> > > done < <(jobs -l)
>> >
>> > That fails for multi-line commands that happen to contain matches for
>> > re.
>> >
>> > $ (sleep 100; printf $'\n[100]\n') &
>>
>> Fair enough. I believe *nothing* would work in that case; the output of
>> "jobs -l" cannot be safely parsed in its current form.
>>
>> Adding a NUL-delimited output option might work around that, but if we're
>> modifying the jobs builtin, we might as well just add the -i or -j option
>> instead.
>>
>
> ther'd be few different values for seps , i think ifs is nearly none , so
> , one is per element , one per record
> others may also exist
> so just two constant opts to align extend usage
>
i d have a half example about such formatting for printf
( pre ) ( sep for 2+ elements ) ( elem pre ( also elem N pre ) ) $elem (
elem stu or and N th ) ( end )
missing some terms also ..
>
Re: Idea: jobs(1) -i to print only :%ID:s
Date:Thu, 9 Nov 2023 16:55:50 -0500 From:Greg Wooledge Message-ID: | I believe *nothing* would work in that case; There's no requirement (in fact, it is probably not a good idea) for newlines in the output to be printed literally - whether entered literally or using $'\n'. Bash could always just output $'\n' for a newline. It could use $' instead of ' for everything intended to be quoted (and then escape other non-printing characters as well, so output which might lock the terminal wouldn't appear literally, etc). | the output of "jobs -l" cannot be safely parsed in its current form. It probably isn't a good idea, regardless of processing the output, as jobs -l (when it reports process termination, which it can) is supposed to remove the job from the jobs table, and forget it - which generally isn't what is wanted when the output is being parsed. How that operates when the jobs -l is being executed in a subshell but reporting the status of its parent is less clear (and it most probably simply doesn't happen - but there are no guarantees). That is if the jobs command works that way at all in a particular shell. | but if we're modifying the jobs builtin, we might as well just add | the -i or -j option instead. ash based shells have a jobid builtin, which takes anything which identifies a job, and prints a list of its process ids. The NetBSD variant of that has been extended to be able to instead print job identifiers (%1 etc) the process group id, or the lead process id (what $! would have been). I have just added (not yet committed) the ability to do that for all jobs (or any of a listed set) rather than just for one which used to be the case. This command never alters the jobs table, so is safer to use, and doesn't print anything which needs complex interpretation (not intended for humans). But in general this point is valid - though fixing the output of the command to keep it all on one line would also be a useful thing to do, regardless of anything else. kre
Re: Idea: jobs(1) -i to print only :%ID:s
Andreas Schwab wrote in
<87h6lueids@igel.home>:
|On Nov 09 2023, Greg Wooledge wrote:
|
|> re='^\[([0-9]+)\]'
|> jobspecs=()
|> while IFS= read -r line; do
|> if [[ $line =~ $re ]]; then
|> jobspecs+=( "%${BASH_REMATCH[1]}" )
|> fi
|> done < <(jobs -l)
|
|That fails for multi-line commands that happen to contain matches for
|re.
|
|$ (sleep 100; printf $'\n[100]\n') &
Bummer. Same is true for my thing.
Seems to me bash(1) should quote the output of jobs -l.
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: Idea: jobs(1) -i to print only :%ID:s
Robert Elz wrote in
<7989.1699564...@jacaranda.noi.kre.to>:
|Date:Thu, 09 Nov 2023 21:04:28 +0100
|From:Steffen Nurpmeso
|Message-ID: <20231109200428.8g9lz%stef...@sdaoden.eu>
|
|| ash(1) and busybox ash(1) do not even support jobs -l in a function \
|| (yet).
|
|That seems unlikely, ash based shells mostly don't care whether the \
|commands
|they're running are in a function or not, except for those few commands for
|which that is important (like return, local, etc).
|
|Are you sure that you're not seeing the effects of running jobs in a
|subshell environment (in a command substitution) - where no jobs have
|yet been created in that environment. Modern shells allow jobs (like
|trap) there, as a way to get the parent shell's jobs (or traps), but
|even now (in the latest drafts) POSIX does not require that jobs works
|that way (it does for trap, in very limited cases).
That was taking me too literally. It seems monitor mode and all
its tracked jobs are not available in a subshell, not even within
{ ; }. ... Hmm. Well have i done this afternoon:
#?0|kent:tmp$ dash
#?0|kent:tmp$ LESS= less t.rc &
#?0|kent:tmp$ LESS= less t.sh &
#?0|kent:tmp$ jobs -l
[2] + 18950 Stopped (tty output) less t.sh
[1] - 18948 Stopped less t.rc
#?0|kent:tmp$ { jobs -l; }
[2] + 18950 Stopped (tty output) less t.sh
[1] - 18948 Stopped less t.rc
#?0|kent:tmp$ ( jobs -l; )
#?0|kent:tmp$ listem() {
> jobs -l
> }
#?0|kent:tmp$ listem
[2] + 18950 Stopped (tty output) less t.sh
[1] - 18948 Stopped less t.rc
#?0|kent:tmp$ $( listem )
#?0|kent:tmp$
So i would have to use output redirection in order to avoid
calling listem() in a $() construct.
It seems the simple issue is about to become a monster.
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: Idea: jobs(1) -i to print only :%ID:s
Robert Elz wrote in <15529.1699569...@jacaranda.noi.kre.to>: |Date:Thu, 9 Nov 2023 16:55:50 -0500 |From:Greg Wooledge |Message-ID: | || I believe *nothing* would work in that case; | |There's no requirement (in fact, it is probably not a good idea) for |newlines in the output to be printed literally - whether entered literally |or using $'\n'. Bash could always just output $'\n' for a newline. I'd only wish tools would start to create quoted ranges of maximum length, instead of these atomar quote orgies. No normal human being can grasp that. (In my opinion. I mean my bank account has a four digit number, and that is tough stuff to change. :)) ... --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Idea: jobs(1) -i to print only :%ID:s
Greg Wooledge wrote in : |On Thu, Nov 09, 2023 at 10:17:35PM +0100, Andreas Schwab wrote: |> On Nov 09 2023, Greg Wooledge wrote: |>> re='^\[([0-9]+)\]' .. |>> while IFS= read -r line; do |>> if [[ $line =~ $re ]]; then ... |> That fails for multi-line commands that happen to contain matches for |> re. |> |> $ (sleep 100; printf $'\n[100]\n') & | |Fair enough. I believe *nothing* would work in that case; the output of |"jobs -l" cannot be safely parsed in its current form. | |Adding a NUL-delimited output option might work around that, but if we're |modifying the jobs builtin, we might as well just add the -i or -j option |instead. As i started this: it should simply quote it so that "i=$(eval $x)" brings back the real value. I use that excessively for the mutilated shell capabilities of the MUA i maintain (am at to at least improve that a bit for ie "eval set \$(($1+1))".). --End of --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: Idea: jobs(1) -i to print only :%ID:s
On Thursday, November 9, 2023, Steffen Nurpmeso wrote:
>
> I mean some scripting on "jobs | wc -l" would do that, though. :(
> Maybe i should just write a function that builds the string
> necessary to do what i wanted with %* or "%1-2 %4" etc.
> Eh. Forget about it.
>
Can't you abuse jobs -x somehow? Like this perhaps
jobs -x printf '%s\n' %{1..100} | awk '!/%/{print "%"NR}'
--
Oğuz
