Re: 'wait -n' with and without id arguments

[Chet Ramey]

On 9/25/24 12:50 PM, Robert Elz wrote:

 Date:Wed, 25 Sep 2024 11:06:10 -0400
 From:Chet Ramey 
 Message-ID:  

   | So you are saying that prompt notifications and `jobs' have the same
   | effect. POSIX implies but does not require this, and there is differing
   | behavior among current implementatations.

What it does say in XCU 2.9.3.1 (Issue 8) is:

� If the shell is interactive and the asynchronous AND-OR list became
   a background job: a message indicating completion of the corresponding
   job is written to standard error. If set -b is enabled, it is unspecified
   whether the process ID is removed from the list of known process IDs when
   the message is written or immediately prior to when the shell writes the
   next prompt for input.

which certainly implies (at the least) to me that the job needs to be
deleted by the time the next prompt is issued, either as a result of
set -b notification (if that is enabled) or by the changed status
notification that precedes each prompt (and that it is one of those
two which does it.)


It also says, before that,

"This process ID shall remain known until any one of the following occurs
(and, unless otherwise specified, may continue to remain known after it
occurs)."

Do you think the text you quoted satisfies the "unless otherwise specified?"
It's not clear.

This is why there's differing behavior.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: 'wait -n' with and without id arguments

[Robert Elz]

Date:Wed, 25 Sep 2024 16:15:07 -0400
From:Chet Ramey 
Message-ID:  <20daab70-09e2-4dfa-a98a-4ba29b622...@case.edu>

  | Do you think the text you quoted satisfies the "unless otherwise specified?"

I think "unspecified whether it is removed at A or B" is fairly clear
that it is intended to be removed.

I don't disagree that the wording about all of this could be better,
but the best way to make it better would be to first agree on what
it should say, or there will just be more "unspecified whether..."

kre

Re: compgen -I (programmable completion as if initial word)

[Chet Ramey]

On 9/24/24 3:15 PM, Stephen Gildea wrote:

I would like these two partially entered commands to complete the
same way:

$ ca
$ sudo ca

The first completion nicely takes into account any previous
"complete -I" setting (because it is completing the initial
word of a command).

But the second completion relies on a programmable completion,
and in Bash 5.2 it is hard to program the same behavior.  In
particular, the programmed completion cannot easily tell Bash
to obey my -I setting.

I can get _almost_ the same effect by having the "sudo" compspec
run "compgen -c" (generate completions over command names), but
that would ignore my -I setting.

I would like to be able to say "compgen -I" in my compspec and
have Bash generate completions as if this were the initial
word of a command, applying whatever settings it would to an
actual initial word.


Thanks for the suggestion. I will add it to the requested feature list
for a future bash version.

Chet


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

[sr #111125] compgen does not complete paths starting with environment variables when called indirectly

[anonymous]

URL:
  

 Summary: compgen does not complete paths starting with
environment variables when called indirectly
   Group: The GNU Bourne-Again SHell
   Submitter: None
   Submitted: Wed 25 Sep 2024 07:18:59 AM UTC
Category: None
Priority: 5 - Normal
Severity: 3 - Normal
  Status: None
 Privacy: Public
 Assigned to: None
Originator Email: florian.pommeren...@unibas.ch
 Open/Closed: Open
 Discussion Lock: Any
Operating System: GNU/Linux


___

Follow-up Comments:


---
Date: Wed 25 Sep 2024 07:18:59 AM UTC By: Anonymous
[I also posted this on Stackoverflow but this might be the better place for
it. Original post: https://stackoverflow.com/questions/78978949]

With compgen I can complete paths inside a directory and this can include
environment variables if they are not resolved before calling compgen.


> cd /tmp
> mkdir files
> touch files/aa files/ab
> export MY_FILES=/tmp/files
> echo "$MY_FILES/"
/tmp/files
> compgen -v -A file -- "$MY_FILES/"  # not what I want
/tmp/files/ab
/tmp/files/aa
> echo "\$MY_FILES/"
$MY_FILES/
> compgen -v -A file -- "\$MY_FILES/" # this is what I want
$MY_FILES/ab
$MY_FILES/aa


This works fine when I execute the command but doesn't work when the command
is called indirectly (not even in a login shell)


> bash -ilc 'echo "\$MY_FILES/"; compgen -v -A file -- "\$MY_FILES/"'
$MY_FILES/
> echo $?
1


You can tell from the echo command that the escaping works as intended, so
this is not an issue with how the string is escaped.

Likewise, putting the line in a script and calling the script doesn't work:


> cat run-compgen
#!/bin/bash -il

echo "\$MY_FILES/"
compgen -v -A file -- "\$MY_FILES/"
echo $?

> ./run-compgen
$MY_FILES/
1


I also tried calling env indirectly to ensure that the variable is defined in
the nested shell and I tried the above example in a clean ubuntu docker
container to exclude any influence of my setup. Any idea how to get this to
work?








___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/


signature.asc
Description: PGP signature

Re: 'wait -n' with and without id arguments

[Robert Elz]

Date:Wed, 25 Sep 2024 11:06:10 -0400
From:Chet Ramey 
Message-ID:  

  | So you are saying that prompt notifications and `jobs' have the same
  | effect. POSIX implies but does not require this, and there is differing
  | behavior among current implementatations.

What it does say in XCU 2.9.3.1 (Issue 8) is:

� If the shell is interactive and the asynchronous AND-OR list became
  a background job: a message indicating completion of the corresponding
  job is written to standard error. If set -b is enabled, it is unspecified
  whether the process ID is removed from the list of known process IDs when
  the message is written or immediately prior to when the shell writes the
  next prompt for input.

which certainly implies (at the least) to me that the job needs to be
deleted by the time the next prompt is issued, either as a result of
set -b notification (if that is enabled) or by the changed status
notification that precedes each prompt (and that it is one of those
two which does it.)

Apart from the jobs & wait utilities (and XCU 2.11 which is what requires
writing the message mentioned in 2.9.3.1, but doesn't say anything
about removing jobs from the jobs table - except in the case of
converting a bg job to fg which isn't relevant here) from a quick
look I can't see anywhere else where anything is either required to
be, or even permitted to be, deleted (too many child processes, and
such excepted).

kre

Re: 'wait -n' with and without id arguments

[Chet Ramey]

On 9/8/24 8:35 PM, Zachary Santer wrote:

This is still a discussion about interactive shell behavior only.


What behavior do you want from the command lists that differs from what I
described above? Since shell functions are essentially lists, you should
get the same behavior from both.


You'd have to restrict job status notification to only ever occur
immediately prior to a prompt, in both posix and default mode, and
then you'd still need a blurb in the BUGS section of the manual saying
that 'set -b' has a potentially surprising impact on 'wait -n' in the
interactive shell.


This is pretty much posix mode, and there's no reason to list this as a
bug when it's not.



For this to work, you'd have to choose one of the following new behaviors:

1) Background jobs that are both forked and cleared from the jobs
table by a call to 'wait' in the time between an accept-line and the
following prompt would never receive a job id or be notified to the
user in any way. 


That's not how job control works. Jobs are created and job numbers assigned
when the background process is created.


2) Job ids assigned to background jobs continue to increase
monotonically, between accept-line and prompt, even as some of those
jobs are removed from the jobs table by calls to 'wait'.


No shell works this way, and there's not a good reason to adopt it.



If the 'jobs' builtin is called in the midst of a command list being
run with either behavior, this would cause the same updates to the
jobs table and list of saved pids and statuses as would occur
immediately prior to a prompt.


So you are saying that prompt notifications and `jobs' have the same
effect. POSIX implies but does not require this, and there is differing
behavior among current implementatations.



The user would have to know that
calling the 'jobs' builtin would have an impact on what processes
'wait -n' without id arguments will return the termination status of.
That would have to be documented in the man page.


This is posix mode.




However, I think the benefit to
consistent behavior far outweighs the hardship caused to whoever would
write a script intended for use within the interactive shell that depends
on 'wait -n' without id arguments ignoring background processes that the
user has already been notified of via the 'jobs' output.


Consider programmable completion frameworks, commands executed via
`bind -x', or traps (e.g., DEBUG) intended to provide enhancements to
the standard behavior, all of which exist and have generated reports or
requests for features.

People put pretty complicated stuff into PROMPT_COMMAND and other prompts,
too.

We don't know how the existing uses would be affected by changes until I
make them.


Could changes to job status notifications cause issues for these users as well?


You tell me.



Generally speaking, I would expect most functions defined in any of
the above that call 'wait' or 'wait -n' from an interactive
environment to track and use explicit pid arguments, to avoid waiting
on other background jobs the user forked themselves. 


It's an assumption. It might be valid.


In that case, the
behavior they would see, using 'wait -n', has already changed for the
better. The use of 'wait -n' without pid arguments in an interactive
environment is more likely to be something that a user just typed on
the command line themselves.


Why would a user do this? What's the use case for doing that in an
interactive shell? Not that it really matters.


If the behavior here isn't modified, the man page really should note that
'wait -n' without id arguments won't return the termination status of a
child process that has already been notified through the 'jobs' output.


That is exactly the behavior posix seems to require (`wait -n' aside, but
see below): once you notify the user, you delete the job and it disappears
forever.


Should still be in the man page. Very few shell programmers are
reading the POSIX standard.


There is a posix mode section in bash.info.

Does POSIX provide a rationale for this requirement? 


Look at the descriptions of `jobs' and `wait', which say this explicitly.


I'd be curious to know if the Austin Group
people have considered the implications of a feature like 'wait -n'.


Doubtful they considered it explicitly, since the role of a standard is
to standardize existing implementations.



If you go the route of changing job notification behavior, would that
be the end of the list of saved pids and statuses? Maintaining that
list is more useful than simply following POSIX to a tee. What would
be the benefit to the user of making the termination status of
notified jobs unavailable to the 'wait' builtin?


I'm not talking about changing default mode, just making bash conform
to the new requirements in the latest POSIX revision. These requirements
happen to align well with your use case.

For instance, with the latest devel branch build:

$ set -o posix
$ sleep 2 &
[1] 20565
$
[1]+  Done