shell-expand-line working strangely without shebangs
Configuration Information [Automatically generated, do not change]: Machine: aarch64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -g -O2 uname output: Linux nixos 6.1.77 #1-NixOS SMP Mon Feb 5 20:13:03 UTC 2024 aarch64 GNU/Linux Machine Type: aarch64-unknown-linux-gnu Bash Version: 5.2 Patch Level: 15 Release Status: release Description: In bash, the very useful shell-expand-line (`M-C-e`) expands command substitutions to their contents, on the current line. However, this works very strangely for commands without shebangs. Symptoms further described in "Repeat-By" section. Replicated on kitty and mac Terminal on NixOS and macOS with 5.2.15(1)-release (built by Nix). Replicated on kitty and mac Terminal on macOS with 5.2.32(1)-release (built from source). Does NOT replicate on kitty and mac Terminal on macOS with 3.2.57(1)-release (provided by macOS 14.4.1 (23E224)` (Sonoma)). Previously posted on https://unix.stackexchange.com/questions/781355/shell-expand-line-and-shebangs Repeat-By: 1. Create a simple executable file with and without shebang. echo echo hi >> test-no-shebang chmod +x test-no-shebang echo '#!/bin/sh' >> test-shebang echo echo hi >> test-shebang chmod +x test-shebang 2. Enter $(./test-shebang), then M-C-e. This works as expected, expanding the line to "hi". C-_ works normally, undoing the expansion. 3. Enter $(./test-no-shebang), then M-C-e. This works strangely, indeed expanding the line to "hi", but removing the prompt in front of the line. C-_ does not work, printing the character literally instead, and nor do most bindings, but C-u kind of works, successfully clearing the line (but not clearing the "hi").
Re: waiting for process substitutions
On 7/31/24 11:48 AM, Zachary Santer wrote: On Fri, Jul 26, 2024 at 10:19 AM Chet Ramey wrote: You could have looked at the actual commit, which contains the change log, which says - wait_for_any_job: check for any terminated procsubs as well as any terminated background jobs wait_for_any_job is the function that backs `wait -n' without arguments. Thanks for the clarification. $ wait -n > >( cat ) would hang, in the event that there are no other un-waited-for child processes, right? Yes, it will wait for the next job or procsub to terminate. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ OpenPGP_signature.asc Description: OpenPGP digital signature
Re: Bogus (intptr_t) casts
On 8/1/24 4:12 AM, Martin D Kealey wrote:
It follows that the following assertions are allowed to fail:
intptr_t i = 0;
assert(*(void*)i == (void*)0*);
void *p = 0;
assert(*(intptr_t)p == 0*);
Accordingly I provide the following patch:
I'm wondering why you chose these two cases, since there are other very
similar uses of intptr_t casts.
diff --git a/*subst.c* b/subst.c
index 37e0bfa7..140a3a92 100644
--- a/subst.c
+++ b/subst.c
@@ -6875,7 +6875,7 @@ uw_restore_pipeline (void *discard)
static void
uw_restore_errexit (void *eflag)
{
*- change_flag ('e', (intptr_t) eflag ? FLAG_ON : FLAG_OFF);+ change_flag
('e', eflag ? FLAG_ON : FLAG_OFF);*
set_shellopts ();
}
diff --git a/*variables.c* b/variables.c
index cd336c85..d44453fe 100644
--- a/variables.c
+++ b/variables.c
@@ -5444,7 +5444,7 @@ pop_scope (void *is_special)
FREE (vcxt->name);
if (vcxt->table)
{
*- if ((intptr_t) is_special)+ if (is_special)*
hash_flush (vcxt->table, push_builtin_var);
else
hash_flush (vcxt->table, push_exported_var);
You might want to look at the unwind-protect implementation, which doesn't
use assignments. It uses byte copies, so instead of using an assignment of,
say, 0, where the compiler can assign whatever it wants to denote a NULL
pointer, it copies 4-8 bytes, depending on the size of an integer. The cast
of that memory back to an intptr_t should be transparent on all reasonably
common systems.
Of course, if you can provide an example where it fails, I'll look at it
and fix it.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: shell-expand-line working strangely without shebangs
On 8/5/24 4:57 AM, Jae Juang wrote: Bash Version: 5.2 Patch Level: 15 Release Status: release Description: In bash, the very useful shell-expand-line (`M-C-e`) expands command substitutions to their contents, on the current line. However, this works very strangely for commands without shebangs. Symptoms further described in "Repeat-By" section. Hi. Thanks for the report. This is the same issue as reported in https://lists.gnu.org/archive/html/bug-bash/2024-01/msg00095.html The fix is described in https://lists.gnu.org/archive/html/bug-bash/2024-01/msg00101.html and has been in the devel git branch since late January. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ OpenPGP_signature.asc Description: OpenPGP digital signature
Re: waiting for process substitutions
On Mon, Aug 5, 2024 at 9:54 AM Chet Ramey wrote: > > On 7/31/24 11:48 AM, Zachary Santer wrote: > > > > $ wait -n > >( cat ) > > would hang, in the event that there are no other un-waited-for child > > processes, right? > > Yes, it will wait for the next job or procsub to terminate. Basically the same situation as Oğuz's bug here: On Fri, Jul 5, 2024 at 2:38 PM Chet Ramey wrote: > > On 6/29/24 10:51 PM, Zachary Santer wrote: > > > > 1) When all child processes are process substitutions: > > a. wait without arguments actually appears to wait for all of them, > > not just the last-executed one, contradicting the man page. > > This is how it is in bash-5.2. It does contradict the man page, and I > reverted it to match the man page in October, 2022, after > > https://lists.gnu.org/archive/html/bug-bash/2022-10/msg00107.html > > The whole business is kind of a mess. Let's see if we can figure out the > best behavior. So $! is only set to the pid for a procsub that's being redirected to/from after the command (or compound command) whose output/input is being redirected to/from it has completed? If that's the case, maybe the ideal solution for both 'wait' and 'wait -n' would be to wait for all background jobs and all procsubs that have ever been assigned to $! at that point. Bash already knows the pids for the procsubs before they are assigned to $!, right? During that time, those pids can be kept in a separate list, and then the list of all procsub pids that have ever been assigned to $! can be waited for. This still removes the potential for hanging, I think, while being more useful.
Re: shell-expand-line working strangely without shebangs
Thank you! Indeed a build from branch bash-5.3-testing shows the issue to be fixed (devel did not build for me). I look forward to the release of the fix. On Tue, Aug 6, 2024, at 1:41 AM, Chet Ramey wrote: > On 8/5/24 4:57 AM, Jae Juang wrote: > > > Bash Version: 5.2 > > Patch Level: 15 > > Release Status: release > > > > > > > > Description: > > In bash, the very useful shell-expand-line (`M-C-e`) expands command > > substitutions to their contents, on the current line. However, this works > > very strangely for commands without shebangs. Symptoms further described in > > "Repeat-By" section. > > Hi. Thanks for the report. This is the same issue as reported in > > https://lists.gnu.org/archive/html/bug-bash/2024-01/msg00095.html > > The fix is described in > > https://lists.gnu.org/archive/html/bug-bash/2024-01/msg00101.html > > and has been in the devel git branch since late January. > > Chet > > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ > > > > *Attachments:* > • OpenPGP_signature.asc
Re: waiting for process substitutions
On 8/5/24 2:21 PM, Zachary Santer wrote: On Mon, Aug 5, 2024 at 9:54 AM Chet Ramey wrote: On 7/31/24 11:48 AM, Zachary Santer wrote: $ wait -n > >( cat ) would hang, in the event that there are no other un-waited-for child processes, right? Yes, it will wait for the next job or procsub to terminate. Basically the same situation as Oğuz's bug here: You mean the one I referenced below, right? https://lists.gnu.org/archive/html/bug-bash/2022-10/msg00107.html Sort of, but they're not the same. The issue with Oguz's bug is that $! had changed before `wait', and was no longer the same as the last process substitution -- the `wait' command should not have waited for it. The other issue was a bookkeeping problem -- even though the process substitution was performed by the same shell that executed the subshell command (because that's how bash optimizes forks), that command shouldn't inherit the process substitutions, since they're logically performed in a different execution context. But in the end, if you're waiting for a process that isn't going to terminate, you're going to be waiting for a long time. So $! is only set to the pid for a procsub that's being redirected to/from after the command (or compound command) whose output/input is being redirected to/from it has completed? Wow, that's a really complex sentence. And the answer is not really, and it's hard to tell. Since bash performs word expansions before redirections when executing simple commands, all the words in a simple command have been expanded before the process substitution changes $!. So it's not after it's completed, but it's after word expansions have been performed, so it's hard to tell. But if you ran something like this: echo $! <(echo $BASHPID > /dev/tty ) $! the second $! would be expanded to the pid of the procsub because the word expansion changed it. So in your example, the redirection sets $! (or its internal equivalent) to the pid of the procsub, and wait -n waits for it. These operations are performed by the same shell process. If that's the case, maybe the ideal solution for both 'wait' and 'wait -n' would be to wait for all background jobs and all procsubs that have ever been assigned to $! at that point. That's all procsubs. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ OpenPGP_signature.asc Description: OpenPGP digital signature
Re: waiting for process substitutions
On Mon, Aug 5, 2024 at 5:10 PM Chet Ramey wrote: > > But in the end, if you're waiting for a process that isn't going to > terminate, you're going to be waiting for a long time. So, even with wait without id arguments restricted to the final procsub, if its process id is the same as $!, it's still trivial to create a scenario where the call to wait will hang. Given that, I don't see the benefit over simply waiting for all process substitutions. > So in your example, the redirection sets $! (or its internal equivalent) > to the pid of the procsub, and wait -n waits for it. These operations are > performed by the same shell process.
Re: waiting for process substitutions
On Tuesday, August 6, 2024, Zachary Santer wrote: > I > don't see the benefit over simply waiting for all process > substitutions. > The benefit is they're separate from async jobs and don't get in your way. `wait' waiting for the last procsub is acceptable, `wait' waiting for a procsub that I forgot about and that won't be listed by `jobs' is not. Procsubs occupying one slot in the shell's internal list of job statuses is acceptable, them filling up that list and causing data loss/oom errors is not. -- Oğuz
Re: waiting for process substitutions
imo have multiple $! at once set is to me a hard must , when more are invoked .. this is a programmical must .. or do only i see these schematics .. a no is brainless .. .. in the way not supporting floating math .. so i should not care so much =pp =) On Tue, Aug 6, 2024, 07:19 Oğuz wrote: > On Tuesday, August 6, 2024, Zachary Santer wrote: > > > I > > don't see the benefit over simply waiting for all process > > substitutions. > > > > The benefit is they're separate from async jobs and don't get in your way. > `wait' waiting for the last procsub is acceptable, `wait' waiting for a > procsub that I forgot about and that won't be listed by `jobs' is not. > Procsubs occupying one slot in the shell's internal list of job statuses is > acceptable, them filling up that list and causing data loss/oom errors is > not. > > > -- > Oğuz >
