Re: ${param@a} should bypass unbound variable check or be processed ahead of it.

[konsolebox]

On Mon, Oct 21, 2024 at 11:08 AM Grisha Levit  wrote:
>
> Maybe just adjusting the latter to also expand to the variable's attributes 
> would suffice.

var[@] refers to all the elements and not the variable itself so I'll
find that painful to use.  The solution to check want_attributes is
simpler.


-- 
konsolebox

Re: posix vs default mode nonsense

[Zachary Santer]

On Mon, Oct 21, 2024 at 11:50 AM Chet Ramey  wrote:
>
> On 10/21/24 12:15 AM, Zachary Santer wrote:
> >
> > The nontrivial stuff I do is still being run by bash 4.2 at the
> > moment, and I'm not willing to give up procsubs.
>
> Probably something to take up with your distro (Red Hat/Centos, right?)

RHEL 7. Something more recent is coming ... eventually.

On Mon, Oct 21, 2024 at 11:53 AM Chet Ramey  wrote:
>
> On 10/21/24 9:54 AM, Robert Elz wrote:
> >  From the earlier message (just replying to this one to get the
> > new Subject: which fits better)...
> >
> > zsan...@gmail.com said:
> >| I can work around function names needing to be valid shell 'name's by
> >
> > That one bash could easily fix if it wanted to, there's never been a
> > POSIX requirement that only "name"s can be used as function names.
>
> No, the original 1992 standard required it:
>
> "The function is named fname; it shall be a name (see 3.1.5)."
>
> That changed to an application requirement in 2001.

Pardon my ignorance, but as an "application requirement," is that
saying what shell scripts are allowed to do, rather than what bash
itself is allowed to do? That's how I would interpret kre's quote from
the standard. Otherwise, it's contradicting itself.

Don't know how much weight the Google Shell Style Guide carries here,
or with the Austin Group, but:
>  If you’re writing a package, separate package names with :: [in function 
> names].
https://google.github.io/styleguide/shellguide.html#function-names

Koichi Murase's ble.sh has a bunch of function names with '/', '-',
'.', etc. in the names.
https://github.com/akinomyoga/ble.sh/

I've got some functions that look like I'm overloading the increment
operator in C++:
abc::++ () { ... }
xyz::++ () { ... }
++ () { ... }

Those functions spent some time as "increment"* before I realized I
could do that, and I couldn't resist the urge.

So my script might violate POSIX, but POSIX allows shell
implementations to work with those function names anyway? I don't mind
my script violating POSIX. It's probably not going to run in any other
shell than bash anyway.

[bug #66363] "test" builtin command, -a arg not working as expected

[anonymous]

URL:
  

 Summary: "test" builtin command, -a arg not working as
expected
   Group: The GNU Bourne-Again SHell
   Submitter: None
   Submitted: Mon 21 Oct 2024 10:18:53 AM UTC
Category: None
Severity: 3 - Normal
  Item Group: None
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any


___

Follow-up Comments:


---
Date: Mon 21 Oct 2024 10:18:53 AM UTC By: Anonymous
$ test -n "" -a x -ge 4
bash: test: x: integer expression expected

In the above shown, output, bash correctly reports that "x" isn't right as it
expects an integer on the left hand side of the -ge arg. However, I do not
expect bash to evaluate the expression on the right side of the -a arg as the
left side of the -a arg should always evaluate to false (therefore the right
side expression should not be evaluated, IMO).

This issue came up in practice when I did something like this:
$ if test -n "$2" -a "$2" -ge 4; then 

I was making sure an 2nd argument was supplied to my script before doing any
integer comparisons on it.

I ended up working around the issue by changing the line to something like:
$ if test -n "$2" && test "$2" -ge 4; then

I think this is the second time I've been caught out by this unexpected
behavior of the test command.

Cheers







___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/


signature.asc
Description: PGP signature

Re: posix vs default mode nonsense

[Robert Elz]

>From the earlier message (just replying to this one to get the
new Subject: which fits better)...

zsan...@gmail.com said:
  | I can work around function names needing to be valid shell 'name's by 

That one bash could easily fix if it wanted to, there's never been a
POSIX requirement that only "name"s can be used as function names.

What it does require is that

name() ...

be a function definition, whereas what

+++() ...

might mean is just unspecified, so applications intending to be
portable should only use a "name" there, but the implementation can
treat the latter as a syntax error, or as defining a function called
"+++" (the sane thing to do), or as something different if they desire.

The standard says:

The function is named fname; the application shall ensure that it
is a name (see XBD Section 3.216) and that it is not the name of a
special built-in utility. An implementation may allow other
characters in a function name as an extension.

and that second sentence has been there since at least Posix Issue 4 (1994)
(the first has been rewritten a few times, but it isn't the relevant one.)

The only thing that really can't appear in a function name is '/' - and it
isn't because it would be an error to define it that way, but because the
execution rules don't allow finding a function with a '/' in its name
(that's also why using the name of a special built-in doesn't work) - so
those only working (if they do at all) in non-posix mode would be
reasonable, rejecting anything else (including things like operators, if
quoted, in both definition and execution) makes no sense - anything that
can be a command name which could be found by a PATH search ought to be
able to be a function name, in posix mode as well as not in it.

kre

Re: IFS whitespace definition

[Chet Ramey]

On 10/21/24 10:13 AM, Robert Elz wrote:


   | The word splitting section wasn't quite
   | as rigorous as the current version's, but it referenced this definition.

Actually, it didn't, which was one of that section's many problems.

What it said was

The term ``IFS white space'' is used to mean any sequence (zero or
more instances) of white-space characters that are in the IFS value

There's no reference to anything in XBD, and the term it uses is
"white-space" not "white space" which the definitions define.   And
yes, that hyphen really makes a difference in things like this.


Sure. Still, a lot of people, including the folks in the austin group,
made that logical jump.



   | However, the conformance suite tests for this.

That has tested for what its developers thought the standard said,
rather than what it actually says, before, and probably will again.


Sure. And you still couldn't get certification without passing the test
suite, so it's the de facto standard. At least the current version has
this clarified.



   | The comment in locale_setblanks explains this: some systems, like macOS,
   | return true from isspace() for characters between 0x80 and 0xff even
   | though they introduce multibyte characters (every locale besides "C"
   | in macOS uses UTF-8 encoding).

I assume that the macos people assume that if you're fetching multi-byte
characters you should be fetching the whole character before testing what
kind of object it is.


Yes, that's what bash does, and it allows multibyte characters in $IFS.

But Greg was asking about the code that sets the `blank' status of
bytes between 0 and CHAR_MAX, and I explained why bash does what it does.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: IFS whitespace definition

[Robert Elz]

Date:Fri, 18 Oct 2024 16:28:06 -0400
From:Chet Ramey 
Message-ID:  

  | At the time (previous edition of the standard), POSIX defined whitespace
  | as "In the POSIX locale, white space consists of one or more  (
  |  and  characters), , , ,
  | and  characters."

Yes, that was from XBD 3.142 (In issue 7), labelled "White Space"

  | The word splitting section wasn't quite
  | as rigorous as the current version's, but it referenced this definition.

Actually, it didn't, which was one of that section's many problems.

What it said was

The term ``IFS white space'' is used to mean any sequence (zero or
more instances) of white-space characters that are in the IFS value

There's no reference to anything in XBD, and the term it uses is
"white-space" not "white space" which the definitions define.   And
yes, that hyphen really makes a difference in things like this.

  | However, the conformance suite tests for this.

That has tested for what its developers thought the standard said,
rather than what it actually says, before, and probably will again.

  | The comment in locale_setblanks explains this: some systems, like macOS,
  | return true from isspace() for characters between 0x80 and 0xff even
  | though they introduce multibyte characters (every locale besides "C"
  | in macOS uses UTF-8 encoding).

I assume that the macos people assume that if you're fetching multi-byte
characters you should be fetching the whole character before testing what
kind of object it is.

That's certainly what the new standard requires of processing IFS - even
though what it is splitting is just treated as bytes, deciding what is
IFS white space needs to use properly decoded characters from IFS, not
just treat it as a byte string.   Then when testing the field being
split (or the line read in the case of the read builtin, if the sequence
of bytes at the current position matches a character in IFS, then that's
a match, if not, one byte gets removed, and try again (that's processing
the input as a byte sequence).   And yes, that's still something of a mess,
but (at least) when using UTF-8 encoding it all ends up working in any
case where it possibly can.   In other multi-byte locales, anything is
possible.

kre

Re: ${param@a} should bypass unbound variable check or be processed ahead of it.

[konsolebox]

On Mon, Oct 14, 2024 at 2:57 AM Chet Ramey  wrote:
>
> I'll consider it, but I'm not eager to carve out more exceptions for
> `set -u'.

Checking for unbound_vars_is_error can be done earliest if you worry
about additional runtime weight.  It makes the code for disabled `set
-u` instances optimal by default even just by theory.  That way the
number of added exceptions would only matter to people who care about
using `set -u`.


--
konsolebox
diff --git a/subst.c b/subst.c
index 251eafaf..5e906c10 100644
--- a/subst.c
+++ b/subst.c
@@ -10142,9 +10142,9 @@ parameter_brace_expand (char *string, size_t *indexp, 
int quoted, int pflags, in
 
   /* All the cases where an expansion can possibly generate an unbound
  variable error. */
-  if (want_substring || want_patsub || want_casemod || c == '@' || c == '#' || 
c == '%' || c == RBRACE)
-{
-  if (var_is_set == 0 && unbound_vars_is_error && ((name[0] != '@' && 
name[0] != '*') || name[1]) && all_element_arrayref == 0)
+  if (unbound_vars_is_error)
+if (want_substring || want_patsub || want_casemod || c == '@' && 
!want_attributes || c == '#' || c == '%' || c == RBRACE)
+  if (var_is_set == 0 && ((name[0] != '@' && name[0] != '*') || name[1]) 
&& all_element_arrayref == 0)
{
  set_exit_status (EXECUTION_FAILURE);
  err_unboundvar (name);
@@ -10153,7 +10153,6 @@ parameter_brace_expand (char *string, size_t *indexp, 
int quoted, int pflags, in
  free (name);
  return (interactive_shell ? &expand_wdesc_error : 
&expand_wdesc_fatal);
}
-}
 
   /* If this is a substring spec, process it and add the result. */
   if (want_substring)

Re: posix vs default mode nonsense

[Chet Ramey]

On 10/21/24 12:15 AM, Zachary Santer wrote:


Item 8 is just odd and is on the verge of being a dealbreaker. Not to
go off on another tangent, but what on Earth?



AFAICT it's the non-POSIX-mode Bash behavior that is unusual.

While all shells will have treat single quotes as literal here:

 $ V=set
 $ echo "${V+'x'}"
 'x'

Pretty much every other shell (and bash in posix mode) will continue to do so 
for:

 $ echo "${V+'x}"
 'x

...while bash will treat this as an unterminated command.  Because in bash, 
this is a weird kind of quoting, where the quotes are preserved but any } 
character within them loses its special meaning:


The original POSIX language dating from 1992, and through three editions,
required an even number of single quotes, which made them semi-special:

"Within the string of characters from an enclosed "${" to the
matching '}', an even number of unescaped double-quotes or
single-quotes, if any, shall occur."

See https://www.austingroupbugs.net/view.php?id=221 for the results of
the discussion that prompted this change. Most of the talking took place
on the mailing list. Good times.

This changed in 2010, by which we had over 20 years of the original
behavior (and the interpretation was approved in 2010, but not in an
issued version of the standard until 2016).



 $ unset U
 $ (set +o posix; echo "${U+'}'y}")

 $ (set -o posix; echo "${U+'}'y}")
 'y}


I misinterpreted Item 8, then.

POSIX file:

  8. When parsing and expanding a ${...} expansion that appears within
 double quotes, single quotes are no longer special and cannot be
 used to quote a closing brace or other special character, unless
 the operator is one of those defined to perform pattern removal.
 In this case, they do not have to appear as matched pairs.


https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html#tag_19_02_03

The current standard language says "substring processing." Bash can
do whatever it wants with pattern substitution.


The nontrivial stuff I do is still being run by bash 4.2 at the
moment, and I'm not willing to give up procsubs.


Probably something to take up with your distro (Red Hat/Centos, right?)

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: posix vs default mode nonsense

[Chet Ramey]

On 10/21/24 9:54 AM, Robert Elz wrote:

 From the earlier message (just replying to this one to get the
new Subject: which fits better)...

zsan...@gmail.com said:
   | I can work around function names needing to be valid shell 'name's by

That one bash could easily fix if it wanted to, there's never been a
POSIX requirement that only "name"s can be used as function names.


No, the original 1992 standard required it:

"The function is named fname; it shall be a name (see 3.1.5)."

That changed to an application requirement in 2001.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: 'wait -n' with and without id arguments

[Chet Ramey]

On 10/20/24 8:51 PM, Zachary Santer wrote:


On item 58:
My external 'kill' is from Cygwin Utilities, naturally, but its -l
does what bash's 'kill -l' does in POSIX mode and its -L does
something similar to what bash's 'kill -l' does in default mode. Could
POSIX-mode bash 'kill -L' still give the table, instead of doing the
same thing as 'kill -l'?


I'll think about it. The `-L' option just exists so it's not an error,
plus it's not POSIX, and having it do the same thing as `-l' seems
reasonable.


Item 3 is also a weird thing to do, but you can turn alias expansion
right back off in a script with 'shopt -u expand_aliases', so
whatever.


POSIX doesn't make any distinction between interactive and non-
interactive shells for aliases.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: posix vs default mode nonsense

[Zachary Santer]

On Mon, Oct 21, 2024 at 5:07 PM Lawrence Velázquez  wrote:
>
> On Mon, Oct 21, 2024, at 4:23 PM, Zachary Santer wrote:
> > On Mon, Oct 21, 2024 at 11:53 AM Chet Ramey  wrote:
> >>
> >> That changed to an application requirement in 2001.
> >
> > Pardon my ignorance, but as an "application requirement," is that
> > saying what shell scripts are allowed to do, rather than what bash
> > itself is allowed to do?
>
> Yes.  A shell is an "implementation", not an "application".
>
> https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap02.html#tag_02_01
> https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap02.html#tag_02_02
>
>
> > Don't know how much weight the Google Shell Style Guide carries here,
> > or with the Austin Group, but:
> >>  If you’re writing a package, separate package names with :: [in function 
> >> names].
> > https://google.github.io/styleguide/shellguide.html#function-names
>
> Google uses default-mode bash almost exclusively, so their guide
> is not relevant to a discussion about POSIX mode.
>
> https://google.github.io/styleguide/shellguide.html#s1.1-which-shell-to-use
>
>
> > Koichi Murase's ble.sh has a bunch of function names with '/', '-',
> > '.', etc. in the names.
> > https://github.com/akinomyoga/ble.sh/
>
> I wouldn't be surprised if POSIX mode broke ble.sh in a number of
> ways.  Using interactive shells in POSIX mode is surely uncommon.
>
>
> > I've got some functions that look like I'm overloading the increment
> > operator in C++:
> > abc::++ () { ... }
> > xyz::++ () { ... }
> > ++ () { ... }
> >
> > Those functions spent some time as "increment"* before I realized I
> > could do that, and I couldn't resist the urge.

I bring up the Google Shell Style Guide, ble.sh, and my own use case
simply to point out that there are uses for function names that aren't
valid "name"s. I'm not trying to claim that POSIX mode must support
everything that Google, Koichi, and I are doing with bash.

> > So my script might violate POSIX [...] I don't mind my script
> > violating POSIX. It's probably not going to run in any other
> > shell than bash anyway.
>
> Yes, a script that uses those functions is nonconformant, but that
> only matters if you're aiming for conformance in the first place.
>
>
> > but POSIX allows shell implementations to work with those
> > function names anyway?
>
> Yes.

So this aspect of POSIX mode is unnecessary, and maybe the biggest
barrier to its wider use. I don't have any other reason not to use it,
at least in bash 5.3.

5.2.37: bg(1)ed job then only runs partially

[Steffen Nurpmeso]

Hello.

So first: this thread must not even start if the answer is "bash
justs sends a signal to the job's entire process group, if the
programs do not properly deal with that it cannot help that".
(ogg123 in particular is known to have signal issues.)
This just in case possible changes brought in possible bugs.

I have to workaround an inconvinience and therefore have a shell
wrapper around music playing over bluetooth

  $ playbt ...ogg

^ran over an hour, but i needed that terminal, so

  ^Z
  [1]+  Stopped playbt ..
  $ bg
  [1]+ playbt .. &

^i move the job to the background.. the music starts playing
again, .. but only for a few seconds..

  $ jobs
  [1]+  Running playbt ... &
  $ kill -CONT %1

I have to -CONT kill the job to make all processes of "playbt"
work again:

  steffen   3111  3106   0  0.0   2551   6940 Ss+  00:00:00 Oct21 pts/5 \_ 
-bash
  steffen  24711  3111   0  0.0662272 S00:00:00 Oct21 pts/5 |   
\_ /bin/sh /home/steffen/usr-kent-crux-linux-x86_64/bin/playbt ...
  steffen  24712 24711   0  0.5  22646   5836 Sl   00:00:25 Oct21 pts/5 |   
\_ ogg123 -q -d wav -f - ...
  steffen  24713 24711   0  1.1   4533   6420 Sl   00:00:54 Oct21 pts/5 |   
\_ sox --no-show-progress --input-buffer 4194304 -t wav - -t alsa bass -20 
treble +8

Ie playbt here actually runs ($i being the current file)

p='ogg123 -q -d wav -f -'

$p "$i" | $sox --no-show-progress --input-buffer 4194304 -t wav - -t 
alsa bass -20 treble +8

Ie i presume playing then stops once the buffer runs empty, and
only a real -CONT can make the pipeline flow again, which makes me
struggle.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: posix vs default mode nonsense

[Lawrence Velázquez]

On Mon, Oct 21, 2024, at 4:23 PM, Zachary Santer wrote:
> On Mon, Oct 21, 2024 at 11:53 AM Chet Ramey  wrote:
>>
>> On 10/21/24 9:54 AM, Robert Elz wrote:
>> >  From the earlier message (just replying to this one to get the
>> > new Subject: which fits better)...
>> >
>> > zsan...@gmail.com said:
>> >| I can work around function names needing to be valid shell 'name's by
>> >
>> > That one bash could easily fix if it wanted to, there's never been a
>> > POSIX requirement that only "name"s can be used as function names.
>>
>> No, the original 1992 standard required it:
>>
>> "The function is named fname; it shall be a name (see 3.1.5)."
>>
>> That changed to an application requirement in 2001.
>
> Pardon my ignorance, but as an "application requirement," is that
> saying what shell scripts are allowed to do, rather than what bash
> itself is allowed to do?

Yes.  A shell is an "implementation", not an "application".

https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap02.html#tag_02_01
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap02.html#tag_02_02


> Don't know how much weight the Google Shell Style Guide carries here,
> or with the Austin Group, but:
>>  If you’re writing a package, separate package names with :: [in function 
>> names].
> https://google.github.io/styleguide/shellguide.html#function-names

Google uses default-mode bash almost exclusively, so their guide
is not relevant to a discussion about POSIX mode.

https://google.github.io/styleguide/shellguide.html#s1.1-which-shell-to-use


> Koichi Murase's ble.sh has a bunch of function names with '/', '-',
> '.', etc. in the names.
> https://github.com/akinomyoga/ble.sh/

I wouldn't be surprised if POSIX mode broke ble.sh in a number of
ways.  Using interactive shells in POSIX mode is surely uncommon.


> I've got some functions that look like I'm overloading the increment
> operator in C++:
> abc::++ () { ... }
> xyz::++ () { ... }
> ++ () { ... }
>
> Those functions spent some time as "increment"* before I realized I
> could do that, and I couldn't resist the urge.
>
> So my script might violate POSIX [...] I don't mind my script
> violating POSIX. It's probably not going to run in any other
> shell than bash anyway.

Yes, a script that uses those functions is nonconformant, but that
only matters if you're aiming for conformance in the first place.


> but POSIX allows shell implementations to work with those
> function names anyway?

Yes.


-- 
vq

Re: [PATCH] isearch: fix with HANDLE_MULTIBYTE && LC_ALL=C

[Chet Ramey]

On 10/18/24 6:50 PM, Grisha Levit wrote:

HISTFILE= INPUTRC=/ LC_ALL=C bash --norc -in <<< $'\cR.'


Thanks for the report.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/


OpenPGP_signature.asc
Description: OpenPGP digital signature