Re: [bug #65651] argument reference inconsistency

[alex xmb sw ratchev]

On Sun, Apr 28, 2024, 08:52 anonymous  wrote:

> URL:
>   
>
>  Summary: argument reference inconsistency
>Group: The GNU Bourne-Again SHell
>Submitter: None
>Submitted: Sun 28 Apr 2024 06:51:44 AM UTC
> Category: None
> Severity: 3 - Normal
>   Item Group: None
>   Status: None
>  Privacy: Public
>  Assigned to: None
>  Open/Closed: Open
>  Discussion Lock: Any
>
>
> ___
>
> Follow-up Comments:
>
>
> ---
> Date: Sun 28 Apr 2024 06:51:44 AM UTC By: Anonymous
> Try the following script w/ various sets of arguments and tell me if you
> see
> something weird!
>
> #!/bin/bash
> echo "script has $# arguments"
> echo "cmdline \$* = $*"
> echo "first 3 args: [$1] [$2] [$3]"
>
> for a in $*
> do
>echo "arg: $a"
> done
>

did u mean ${!a}
or for a ; or for a in "$@"

Argument set:
> A)   alpha beta gamma
> B)   "alpha" "beta" "gamma"
> C)   "alp ha" "be ta" "gam ma"
>
>
>
>
>
>
>
> ___
>
> Reply to this item at:
>
>   
>
> ___
> Message sent via Savannah
> https://savannah.gnu.org/
>
>
>

Re: [bug #65651] argument reference inconsistency

[Emanuele Torre]

I don't know why this savannah ticket has been crossposted to @bug-bash,
but its replies have not been crossposted here: it is very confusing.

Anyway, they already got a response on savannah.

o/
 emanuele6

Re: Examples of concurrent coproc usage?

[Zachary Santer]

On Sat, Apr 27, 2024 at 1:01 PM Carl Edquist  wrote:
>
> On Mon, 22 Apr 2024, Martin D Kealey wrote:
>
> > On Mon, 22 Apr 2024, 09:17 Carl Edquist,  wrote:
> >
> >> But yeah currently a pipe with a series of records and multiple
> >> cooperating/competing readers perhaps only works if the records have a
> >> fixed size. A new readd[elim] system call like you're talking about
> >> would allow safely reading a single variable-length record at a time.
> >
> > There are other options, such as length-prefixed records, or tagged
> > (typed) records, but of course those aren't POSIX text files.
>
> That'd work for "cooperating" readers (as you put it) where they are not
> attempting to read at the same time.  Though reading a single byte at a
> time also works in that case.
>
> My thought is that (performance considerations aside), the real functional
> improvement with a new "readd" call would be with _competing_ readers
> (more than one read call waiting on the same pipe at the same time).
>
> In that case a length-prefixed or type-tagged record wouldn't seem to work
> with the regular read(2), because a single reader would not be able to
> read the length/type _and_ the corresponding record together.  You can't
> work around this by reading a byte at a time either.  That's why I said it
> would only seem to work (with read(2)) if the records have a fixed size.
> (In order to grab a whole record atomically.)
>
> But a new "readd" call would allow multiple competing readers to read,
> say, a stream of filenames from a pipe, without having to pad each one to
> PATH_MAX bytes.
>
> It seems that if there is only one reader at a given time though
> ("cooperating"), then it's just a matter of performance between
> read(2)'ing one byte at a time vs using a new readd call.
>
> ...
>
> I'm not trying to advocate for or against you contacting the kernel folks
> with your idea; it just seems to me that the scenario with multiple
> competing readers might be the strongest argument for it.

Where it's available, you'd probably want to do this with a POSIX
message queue with all the messages sent with the same priority.

If you really want a bash competing-process job-server arrangement by
passing messages through a single pipe, you might be able to use
flock(1) to ensure that only one process is allowed to read from the
pipe at a time.

If you take out this use case, a potential readd[elim] call does sound
like it's kind of just there to have a more efficient read command in
the shell.

Re: [bug #65651] argument reference inconsistency

[alex xmb sw ratchev]

On Sun, Apr 28, 2024, 13:59 Emanuele Torre  wrote:

> I don't know why this savannah ticket has been crossposted to @bug-bash,
> but its replies have not been crossposted here: it is very confusing.
>

cool , thanks ... btw got me the link ? thxx && bless

Anyway, they already got a response on savannah.
>
> o/
>  emanuele6
>

Readdelim (was Re: Examples of concurrent coproc usage)

[Martin D Kealey]

On Sun, 28 Apr 2024, 05:03 Carl Edquist,  wrote:

>
> > I would hope that mapfile/readarray could do better, since it's not
> > obligated to leave anything in the input stream.
>
> That is an interesting thought, although mapfile seems to read a byte at a
> time also.
>
> [I'm not suggesting this should be changed though.  And frankly I have no
> strong desire for a faster 'read' builtin in the shell, either.  A byte at
> a time is relatively slow, but that's generally fine in most limited
> contexts where it's actually needed.]
>

I'm encountering contexts where the lack of speed is annoying though not
critical.

My thought is that (performance considerations aside), the real functional
> improvement with a new "readd" call would be with _competing_ readers
> (more than one read call waiting on the same pipe at the same time).
>

That's a very good point; I'll mention that when I write to the Linux
kernel team.

In that case a length-prefixed or type-tagged record wouldn't seem to work
> with the regular read(2), because a single reader would not be able to
> read the length/type _and_ the corresponding record together.  You can't
> work around this by reading a byte at a time either.  That's why I said it
> would only seem to work (with read(2)) if the records have a fixed size.
> (In order to grab a whole record atomically.)
>

This makes me wonder whether readv() could be encouraged to handle
length-prefix simply by targeting the length element of a subsequent
element of the iovec array? iov[0].ptr = &iov[1].len; iov[0].len = sizeof
iov[1].len; iov[1].ptr = malloc(MAX_RECORD_SIZE); I suspect this won't work
today because the kernel probably computes the total read size up front.
But maybe some day?

-Martin

Re: [5.3-alpha] ">& /some/file" would fail when /some/file already exists

[Chet Ramey]

On 4/26/24 3:42 AM, Grisha Levit wrote:


Looks like RX_* values were previously used only to form the `flags'
argument of do_redirections() and related functions, comprising:

     RX_ACTIVE RX_UNDOABLE RX_CLEXEC


Separately, the REDIRECTION struct has a `flags' member holding O_*
flags passed to open(). But the following (other) RX_* values are also
getting stored in the `flags' member of REDIRECTION structs:

     RX_INTERNAL RX_USER RX_SAVCLEXEC RX_SAVEFD RX_EXPANDED


So as Oğuz notes, the issue here is observed when an RX_* value from
the above list has the same value as an O_* flag.


Yes, this overlap is the problem.


Perhaps it would make sense to change the latter set of RX_* values
into new REDIR_* values, which make up the `rflags' REDIRECTION member.


The rflags approach is the right one, you just need to separate the private
RX_ flag vaues from the public REDIR_  values by reserving bits.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: [bug #65651] argument reference inconsistency

[Lawrence Velázquez]

On Sun, Apr 28, 2024, at 7:59 AM, Emanuele Torre wrote:
> I don't know why this savannah ticket has been crossposted to @bug-bash,
> but its replies have not been crossposted here: it is very confusing.

Apparently it's not possible to have the replies crossposted:

https://savannah.gnu.org/support/?111046

> Anyway, they already got a response on savannah.

OP can't be reached from here anyway, since they were not added to
the CC list.

-- 
vq

Re: [[...]] documentation claims tilde expansion and process substitution occur within double quotes

[Chet Ramey]

On 4/26/24 5:36 PM, Lawrence Velázquez wrote:

Both the manual [1] and man page [2] state:

The words between the [[ and ]] do not undergo word splitting
and filename expansion.  The shell performs tilde expansion,
parameter and variable expansion, arithmetic expansion,
command substitution, process substitution, and quote removal
on those words (the expansions that would occur if the words
were enclosed in double quotes).

However, the parenthetical is misleading because tilde expansion
and process substitution don't occur within double quotes.  (I just
helped an IRC user who'd been confused by this text.)


Yes, the parenthesized text is wrong. I can't remember why I added it.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Linux reports memfd_create() being called without MFD_EXEC or MFD_NOEXEC_SEAL set

[Chet Ramey]

On 4/27/24 6:00 AM, Kerin Millar wrote:

Hi,

At some point after upgrading to bash-5.3-alpha, the following message appeared 
in my kernel ring buffer.

[700406.870502] bash[3089019]: memfd_create() called without MFD_EXEC or 
MFD_NOEXEC_SEAL set

Unfortunately, it took me a while to notice the presence of this message. 
Therefore, I am uncertain as to what bash was being tasked with at the time 
that it was logged. The inference of the message seems clear, however. The 
presently running kernel is 6.6.28.


Bash uses memfd_create to create an anonymous file for noexec command
substitution. It's not clear why it would need file sealing, where
available, in this case.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Linux reports memfd_create() being called without MFD_EXEC or MFD_NOEXEC_SEAL set

[Chet Ramey]

On 4/27/24 8:09 AM, Andreas Schwab wrote:

On Apr 27 2024, Kerin Millar wrote:


At some point after upgrading to bash-5.3-alpha, the following message appeared 
in my kernel ring buffer.

[700406.870502] bash[3089019]: memfd_create() called without MFD_EXEC or 
MFD_NOEXEC_SEAL set


This warning has been tuned down in later kernels, but nevertheless,
bash should pass MFD_NOEXEC_SEAL (if defined) when it calls
memfd_create.


OK, I'll bite. What does that buy me?

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: bash: ":?xxx" filename broken on autocomplete

[Chet Ramey]

On 4/27/24 5:28 PM, Gioele Barabucci wrote:

Control: found -1 5.2.21-2

On Tue, 27 Aug 2019 16:36:03 +0200 Philipp Marek  
wrote:
the autocompletion is broken on filenames or directories with ":?" at the 
beginning.


    # mkdir ':?aa'
    # rmdir :

gives me

    # rmdir :\:\?

which doesn't match the filename; I can finish completion by entering 
"aa", but then "rm" rejects this name.


In bash 5.2.21(1) the filename is now fully completed, but the stray ":" at 
the beginning is still produced:


     $ mkdir ':?aa'
     $ rmdir :
     $ rmdir :\:\?aa/


`:' is one of the characters in the default value of COMP_WORDBREAKS, which
is how bash exposes the set of characters readline uses to break words for
completion.

The word being completed here is "". If you want to complete a filename
starting with `:', quote it with a backslash.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: bash: ":?xxx" filename broken on autocomplete

[Chet Ramey]

On 4/27/24 5:46 PM, Kerin Millar wrote:

On Sat, 27 Apr 2024 23:28:49 +0200
Gioele Barabucci  wrote:


Control: found -1 5.2.21-2

On Tue, 27 Aug 2019 16:36:03 +0200 Philipp Marek 
wrote:

the autocompletion is broken on filenames or directories with ":?" at the
beginning.

 # mkdir ':?aa'
 # rmdir :

gives me

 # rmdir :\:\?

which doesn't match the filename; I can finish completion by entering "aa",
but then "rm" rejects this name.


In bash 5.2.21(1) the filename is now fully completed, but the stray ":"
at the beginning is still produced:

  $ mkdir ':?aa'
  $ rmdir :
  $ rmdir :\:\?aa/


In the course of trying this in bash-5.3-alpha, I noticed something else. If ':?aa' is not the only entry in the current working directory, readline behaves as if : is an ambiguous completion. 


The word being completed is "".

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: [bug #65651] argument reference inconsistency

[Chet Ramey]

On 4/28/24 7:59 AM, Emanuele Torre wrote:

I don't know why this savannah ticket has been crossposted to @bug-bash,
but its replies have not been crossposted here: it is very confusing.


Because savannah can't do that.

The original request was to have one place to go to for bug reports. The
best you can do is to have savannah send `notifications' to bug-bash when
an issue is opened. It's not bugzilla; there's no facility to send email
and have it added to an issue as a comment, and follow-up comments on
savannah don't generate notifications.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: [bug #65651] argument reference inconsistency

[Chet Ramey]

On 4/28/24 1:10 PM, Lawrence Velázquez wrote:

On Sun, Apr 28, 2024, at 7:59 AM, Emanuele Torre wrote:

I don't know why this savannah ticket has been crossposted to @bug-bash,
but its replies have not been crossposted here: it is very confusing.


Apparently it's not possible to have the replies crossposted:

https://savannah.gnu.org/support/?111046


Anyway, they already got a response on savannah.


OP can't be reached from here anyway, since they were not added to
the CC list.


The OP is anonymous.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Examples of concurrent coproc usage?

[Chet Ramey]

On 4/27/24 12:56 PM, Carl Edquist wrote:


On Mon, 22 Apr 2024, Chet Ramey wrote:

You might be surprised. The OP was sending thousands of calculations to 
(I think) GNU bc, which had some resource consumption issue that resulted 
in it eventually hanging, unresponsive. The kill was the solution there. 
I imagine there are similar scenarios with other tools.


Ok, you got me!  I take it back.

I hadn't considered bc operations being cpu/memory intensive.  But that 
possibility makes sense - given that it's arbitrary precision I guess you 
can ask for a number to the billionth power and never see the end of it  :)


I'm not sure it was that so much as the long-running nature of the coproc.
A resource leak might never be noticeable except in this (admittedly
uncommon) scenario.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Linux reports memfd_create() being called without MFD_EXEC or MFD_NOEXEC_SEAL set

[Andreas Schwab]

On Apr 28 2024, Chet Ramey wrote:

> On 4/27/24 8:09 AM, Andreas Schwab wrote:
>> On Apr 27 2024, Kerin Millar wrote:
>> 
>>> At some point after upgrading to bash-5.3-alpha, the following message 
>>> appeared in my kernel ring buffer.
>>>
>>> [700406.870502] bash[3089019]: memfd_create() called without MFD_EXEC or 
>>> MFD_NOEXEC_SEAL set
>> This warning has been tuned down in later kernels, but nevertheless,
>> bash should pass MFD_NOEXEC_SEAL (if defined) when it calls
>> memfd_create.
>
> OK, I'll bite. What does that buy me?

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=105ff5339f49

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: Bug#1069978: bash: incorrect value of $BASH for login shells

[Gioele Barabucci]

On 28/04/24 20:01, Chet Ramey wrote:

On 4/27/24 6:23 PM, Gioele Barabucci wrote:
bash 5.0 and 5.2 do not set $BASH to the right value when bash is used 
as the login shell:


 $ apt install bash-static
 $ getent passwd $USER | cut -d: -f 7
 /bin/bash

 $ su $USER -s /bin/bash-static -c 'echo $BASH; readlink 
/proc/$$/exe; true'

 /usr/bin/bash-static
 /usr/bin/bash-static

 $ su -l $USER -s /bin/bash-static -c 'echo $BASH; readlink 
/proc/$$/exe; true'

 /bin/bash
 /usr/bin/bash-static

(bash-static is not a link to bash)


What does `su' pass to bash in argv[0]?



$ su $USER -s /bin/bash-static -c 'echo $BASH; readlink 
/proc/$$/exe; head -1z /proc/$$/cmdline; echo'

/usr/bin/bash-static
/usr/bin/bash-static
bash-static

$ su -l $USER -s /bin/bash-static -c 'echo $BASH; readlink 
/proc/$$/exe; head -1z /proc/$$/cmdline; echo'

/bin/bash
/usr/bin/bash-static
-bash-static

Regards,

--
Gioele Barabucci

Re: Linux reports memfd_create() being called without MFD_EXEC or MFD_NOEXEC_SEAL set

[Chet Ramey]

On 4/28/24 2:59 PM, Andreas Schwab wrote:

On Apr 28 2024, Chet Ramey wrote:


On 4/27/24 8:09 AM, Andreas Schwab wrote:

On Apr 27 2024, Kerin Millar wrote:


At some point after upgrading to bash-5.3-alpha, the following message appeared 
in my kernel ring buffer.

[700406.870502] bash[3089019]: memfd_create() called without MFD_EXEC or 
MFD_NOEXEC_SEAL set

This warning has been tuned down in later kernels, but nevertheless,
bash should pass MFD_NOEXEC_SEAL (if defined) when it calls
memfd_create.


OK, I'll bite. What does that buy me?


https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=105ff5339f49


It's not worth arguing about.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Bug#1069978: bash: incorrect value of $BASH for login shells

[Chet Ramey]

On 4/28/24 3:07 PM, Gioele Barabucci wrote:

     $ su -l $USER -s /bin/bash-static -c 'echo $BASH; readlink 
/proc/$$/exe; head -1z /proc/$$/cmdline; echo'

     /bin/bash
     /usr/bin/bash-static
     -bash-static


So argv[0] == "-bash-static", which causes $0 to be set to -bash-static
and internally sets shell_name to "bash-static" and login_shell to 1
(which notes that bash was executed with argv[0][0] == '-').

Then when you get to setting $BASH, this code gets executed:

  if ((login_shell == 1) && RELPATH(shell_name))
{
  if (current_user.shell == 0)
get_current_user_info ();
  name = savestring (current_user.shell);
}

which has been the way bash has behaved since the bash-1.x days. Is
this enough of an issue to change behavior that dates back that far?

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Bug#1069978: bash: incorrect value of $BASH for login shells

[Gioele Barabucci]

On 28/04/24 22:50, Chet Ramey wrote:

On 4/28/24 3:07 PM, Gioele Barabucci wrote:

 $ su -l $USER -s /bin/bash-static -c 'echo $BASH; readlink 
/proc/$$/exe; head -1z /proc/$$/cmdline; echo'

 /bin/bash
 /usr/bin/bash-static
 -bash-static


So argv[0] == "-bash-static", which causes $0 to be set to -bash-static
and internally sets shell_name to "bash-static" and login_shell to 1
(which notes that bash was executed with argv[0][0] == '-').

Then when you get to setting $BASH, this code gets executed:

   if ((login_shell == 1) && RELPATH(shell_name))
     {
   if (current_user.shell == 0)
     get_current_user_info ();
   name = savestring (current_user.shell);
     }

which has been the way bash has behaved since the bash-1.x days. Is
this enough of an issue to change behavior that dates back that far?


The bash manual defines $BASH as follows:


Expands to the full filename used to invoke this instance of bash.

So, either the code or the manual should be changed.

I see why changing such an old behavior for the sake of correctness may 
not be worth the risk of breaking somebody's workflow. But at the same 
time it is hard to believe that anybody may be relying on "$BASH 
contains the user shell set up in /etc/passwd when read from a login shell".


In my case, I intended to use $BASH to make sure that the right binary 
was picked up by a certain test. I assume I'm not the only one using it 
in that way.


Regards,

--
Gioele Barabucci