Re: It is possible to remove the readonly attribute from {BASH, SHELL}OPTS

[Robert Elz]

Date:Fri, 23 Feb 2024 09:15:09 +1000
From:Martin D Kealey 
Message-ID:  


  | Yes, that's exactly the point, to *avoid* dynamic scoping. I want the
  | equivalent of Perl's "my", rather than Perl's "local".

Lexical scoping does not, that is, cannot, work with the shell
language as it is defined, if you want that you need to go and
invent a whole new language.

As it is thare are too many places in sh where correct operation
requires access to specific vars, which makes no sense (ie: would
not work as intended with lexically scoped local vars) - consider
PATH CDPATH PWD OPTIND OPTARG REPLY (for shells that have it)
and try to explain how they can be made to work as intended if
made local, with lexical scoping.

Regardless of how much people have tried to attempt to impose
a more programming language model on sh vars, it simply does
not work.   sh was designed to operate using an entirely global
namespace - the best way to think of 'local' is 'save the value
and attributes of this var as they are now, and restore them when
the current function invocation terminates' - and absolutely no
more than that.  All operations continue to use a single global
namespace, and because of that, everything simply works as it
should without issues.  Deviate even a little and there get to be
all kinds of problems.

  | new code can avoid the craziness that comes from "everything is global,

crazy or not, that is the sh model.  If you don't like it, use
something different -- tclsh (or with graphics added, wish) is much
more rigorous, and also much less friendly interactively.

  | even when we claim it's local" and "unset can even mess with the poor
  | protection afforded by 'local'".

I'm guessing you mean bash's bizarre unset definition wrt locals.
I consider that to just be a bug.  Unsetting a 'local' variable
should be identical to unsetting a global, as that is what
actually happens (should happen).  That there is a saved copy that
will be restored later is irrelevant.

kre

Re: Bug: Ligatures are removed as one character

[Chet Ramey]

On 2/19/24 9:26 PM, Avid Seeker wrote:

When pressing backspace on Arabic ligatures (including characters with
diacritics), they are removed as if they are one character.


As you might guess, readline doesn't know much about Arabic, per se. In a
UTF-8 locale, for example, it knows base characters and combining
characters.

The idea is simple: when moving backwards, move one multibyte character at
a time, ignoring combining characters, until you get to a character for
which wcwidth(x) > 0, and move point there. The algorithm for moving
forward is similar.

How should this be modified to support Arabic in a portable way?

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: It is possible to remove the readonly attribute from {BASH, SHELL}OPTS

[Chet Ramey]

On 2/22/24 6:15 PM, Martin D Kealey wrote:


On Wed, 21 Feb 2024 at 08:09, Chet Ramey > wrote:


On 2/20/24 4:11 AM, Martin D Kealey wrote:
 > Ideally each function invocation would have its own variable namespace,
 > only using the global namespace as a fall-back, but that creates
 > complications with exported variables, so let's take baby steps to
get there.

That doesn't work with dynamic scoping at all.


Yes, that's exactly the point, to /avoid/ dynamic scoping. I want the 
equivalent of Perl's "my", rather than Perl's "local".


OK. Implementing static scoping for functions is not on my list.

Code that currently relies on dynamic scoping would continue to work, while 
new code can avoid the craziness that comes from "everything is global, 
even when we claim it's local" and "unset can even mess with the poor 
protection afforded by 'local'".


Do you mean that unset can mark local variables at the same scope as unset,
that unset can remove local variables at a previous scope, or that unset in
a shell function can unset global variables?

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: It is possible to remove the readonly attribute from {BASH, SHELL}OPTS

[Chet Ramey]

On 2/23/24 8:13 AM, Robert Elz wrote:


   | even when we claim it's local" and "unset can even mess with the poor
   | protection afforded by 'local'".

I'm guessing you mean bash's bizarre unset definition wrt locals.
I consider that to just be a bug.  Unsetting a 'local' variable
should be identical to unsetting a global, as that is what
actually happens (should happen).  That there is a saved copy that
will be restored later is irrelevant.


Same question:

Do you mean that unset can mark local variables at the same scope as unset,
that unset can remove local variables at a previous scope, or that unset in
a shell function can unset global variables?

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

Bash reference manual: shell parameter expansion examples

[James Weigle]

Hi!

In the Bash reference manual, there are a series of examples of testing if
a variable is null or unset:

   - Under *${parameter:-word}*, the usage is *${var-unset}*.
   - Under *${parameter:=word}*, the usage is *${var:=DEFAULT}*.
   - Under *${parameter:?word}*, the usage is *${var:?var is unset or null}*
   .
   - Under *${parameter:+word}*, the usage is *${var:+var is set and not
   null}*.

I got a little confused at the first example, because it’s the *only*
example where the colon is omitted. It still works—but why is that one
using a different form?

I suppose otherwise there would be no example showing the usage without a
colon. But could the example incorporate both ways, like so?

$ v=123
> echo ${v-does not exist}
> 123
> $ v=
> $ echo ${v-does not exist}
>
> $ echo ${v:-null or does not exist}
> null or does not exist
> $ unset v
> $ echo ${v-does not exist}
> does not exist
>

But that’s a bit verbose. One other way to do it would be to just add the
colon to the original example (it has the same behavior in this case)...

$ v=123
> $ echo ${v:-unset}
> 123
>

and just let the explanation of the '-' vs. ':-' usage suffice, in the
paragraph before these examples. What do you think?

Best,
James

Re: It is possible to remove the readonly attribute from {BASH, SHELL}OPTS

[Robert Elz]

Date:Fri, 23 Feb 2024 10:45:01 -0500
From:Chet Ramey 
Message-ID:  <3544bf1c-e8f6-4358-a281-24bd1be7d...@case.edu>

  | Do you mean that unset can mark local variables at the same
  | scope as unset,

no, but see below.

  | that unset can remove local variables at a previous scope,

yes, that one, but again, see below.   But note, had you said
'unset' there instead of 'remove', it would all be fine.

  | or that unset in a shell function can unset global variables?

no, of course not.

But I think the real issue here is that we would use the term 'scope'
with respect to shell variables at all, it really isn't appropriate.

The non-standard command 'local' which we all have, in slightly
different forms, but with similar enough effect that most simple
usages work fine in all shells, would better be called 'save'
(or something similar) as its real purpose is to preserve the
current state of a variable so that can be restored when the
functuon exits.  Once that's done, everything else (unset included)
just operates in the global var as if the 'local' (or 'save')
never happened.

This turns out to be quite similar to dynamic scoping, with
some subtle differences, but fits the shell's model of variable
use (everything is global, there is exactly one var named PATH
or IFS (etc)) better than actual dynamic scoping.

kre

ps: it is immaterial here what other effects the local command
has on the variable, once the previous val/state has been saved,
also having it unset the var, or assign it '', or alter
attributes (remove export for example) is unimportant.  Clearly
what, if anything, happens makes (or might) a difference to
the script, but none of that kind of variation affects the model.

Re: Bash reference manual: shell parameter expansion examples

[Chet Ramey]

On 2/23/24 10:44 AM, James Weigle wrote:

Hi!

In the Bash reference manual, there are a series of examples of testing if
a variable is null or unset:

- Under *${parameter:-word}*, the usage is *${var-unset}*.
- Under *${parameter:=word}*, the usage is *${var:=DEFAULT}*.
- Under *${parameter:?word}*, the usage is *${var:?var is unset or null}*
.
- Under *${parameter:+word}*, the usage is *${var:+var is set and not
null}*.

I got a little confused at the first example, because it’s the *only*
example where the colon is omitted. It still works—but why is that one
using a different form?


After this paragraph:

  "When not performing substring expansion, using the forms described
below (e.g., ‘:-’), Bash tests for a parameter that is unset or null.
Omitting the colon results in a test only for a parameter that is unset.
Put another way, if the colon is included, the operator tests for both
PARAMETER's existence and that its value is not null; if the colon is
omitted, the operator tests only for existence."

that you noted, the current manual has these:


  $ v=123
  $ echo ${v-unset}
  123
  $ echo ${v:-unset-or-null}
  123
  $ unset v
  $ echo ${v-unset}
  unset
  $ v=
  $ echo ${v:-unset-or-null}
  unset-or-null


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



OpenPGP_signature.asc
Description: OpenPGP digital signature

exec redirection is undone if fd (>9) has O_CLOEXEC

[Koichi Murase]

I have a question.  Maybe it's not technically a bug as fd > 9 is
involved, but the resulting behavior appears to be strange.  I
received some reports from users in my project and tried to find out
what is happening.  This is a reduced case:

  #!/usr/bin/env bash
  enable -f /path/to/lib/bash/fdflags fdflags
  exec 50>1.txt
  fdflags -s +cloexec 50
  exec 50>2.txt
  echo hello >&50
  ls -l 1.txt 2.txt

The result is

  $ ./reduced.sh
  -rw-r--r-- 1 murase murase 6 2024-02-24 13:53:01 1.txt
  -rw-r--r-- 1 murase murase 0 2024-02-24 13:53:01 2.txt

I expect that the string `hello' is saved in 2.txt, but it actually
goes into 1.txt.  It turned out that the redirection "50>2.txt" is
actually performed, but it's immediately undone at the end of `exec'
command.  The undo redirection is set up in the if statement starting
on redir.c:1288 (devel 43ecbeb3).  The code comment says:

  /* experimental: if we're saving a redirection to undo for a file
  descriptor above SHELL_FD_BASE, add a redirection to be undone if
  the exec builtin causes redirections to be discarded.  There needs
  to be a difference between fds that are used to save other fds and
  then are the target of user redirections and fds that are just the
  target of user redirections.  We use the close-on-exec flag to tell
  the difference; fds > SHELL_FD_BASE that have the close-on-exec flag
  set are assumed to be fds used internally to save others. */

This piece of the code seems to be introduced in commit cac4cdbf
(ChangeLog says the change to redir.c was made on 2011-10-09):

https://git.savannah.gnu.org/cgit/bash.git/commit/?h=devel&id=cac4cdbf5


I'm just curious about the background of this different treatment for
"fds that are used to save other fds and then are the target of user
redirections" (quoted from the code comment).  Also, it says
"experimental", but is there any alternative implementation, where the
user-supplied O_CLOEXEC fds are not undone?

Even if there is a technical background, the current behavior observed
by the users is strange, and this behavior doesn't seem to be
specified in the Bash documentation.  The documentation just says

  Redirections using file descriptors greater than 9 should be used
  with care, as they may conflict with file descriptors the shell uses
  internally.

but this behavior is non-trivial even if the user treats the fds with
"care".  Note that, in the actual code, the number 50 in the above
reduced case is chosen to be a number that is not used (which can be
tested by e.g. « ! : >&50 ». There can be false-negatives with this
test, but that's fine).

--
Koichi

Re: exec redirection is undone if fd (>9) has O_CLOEXEC

[Emanuele Torre]

On Sat, Feb 24, 2024 at 02:11:25PM +0900, Koichi Murase wrote:
> I have a question.  Maybe it's not technically a bug as fd > 9 is
> involved, but the resulting behavior appears to be strange.  I
> received some reports from users in my project and tried to find out
> what is happening.  This is a reduced case:

Thank you for bringing this up; I have also noticed weird behaviour with
redirections of cloexec file descriptors a couple months ago.

Bash seems to check for the CLOEXEC flag in the source file descriptor,
and preserves it in the destination file descriptor when performing a
n<&m redirection.

  $ bash -c 'enable fdflags; exec 6= 3.

I don't know why bash wants/needs to copy the CLOEXEC flag when the user
duplicates a file descriptor.

o/
 emanuele6