Re: comments inside command subst are handled inconsistently

[Denys Vlasenko]

On 7/28/23 19:51, Martin D Kealey wrote:

On the other hand, since everyone has now had 36+ years to update their scripts 
to get rid of backticks,


I don't know about others, but I missed the memo that `` is deprecated.

Please do not break compatibility.

The importance of compatibility is something people usually understand
after they get more experience being burned by failures to preserve it.

In the spirit of increased compatibility across Unix world, it'd be
quite useful if shells stop inventing incompatible "extensions".

The "function" keyword, for example. Why does it even exists?
It adds no functionality. When you use it, the only effect you get is
that now your script requires bash to work properly.

(searching the net... looks like it's historic, added for compatibility
with ksh).

Sorry for rambling.

Re: comments inside command subst are handled inconsistently

[G. Branden Robinson]

Hi Denys,

At 2023-07-31T13:38:00+0200, Denys Vlasenko wrote:
> On 7/28/23 19:51, Martin D Kealey wrote:
> > On the other hand, since everyone has now had 36+ years to update
> > their scripts to get rid of backticks,
> 
> I don't know about others, but I missed the memo that `` is
> deprecated.

The definitive "takedown" of `` is in Tom Duff's memo describing his
"rc" shell, which dates back to 1988-1991 or thereabouts.  (I wish I
could put a more precise date on it.)[1]

I'm attaching a copy for your edification.  Any use of a Bourne-derived
shell should read it and consider its points.

> The "function" keyword, for example. Why does it even exists?  It adds
> no functionality. When you use it, the only effect you get is that now
> your script requires bash to work properly.
> 
> (searching the net... looks like it's historic, added for
> compatibility with ksh).

Yes.  It'd be a better world if both `` quoting _and_ the "function"
keyword were retired.

Regards,
Branden

[1] It's possible Steve Bourne knew better but was pressured into
implementing the lexically simpler `` approach to command
substitution instead of recursively calling the parser (as Chet
pointed out) because it would be more performant.  John Mashey, the
author of the predecessor Unix shell (and successor to Ken
Thompson's original Unix shell) has gone on record as "goading"
Bourne into doing questionable things in the quest for speed.

https://www.in-ulm.de/~mascheck/bourne/segv.html


rc_A Shell for Plan 9 and Unix Systems_Duff.pdf
Description: rc_shell_tom_duff.pdf


signature.asc
Description: PGP signature

Re: comments inside command subst are handled inconsistently

[Chet Ramey]

On 7/31/23 7:38 AM, Denys Vlasenko wrote:


In the spirit of increased compatibility across Unix world, it'd be
quite useful if shells stop inventing incompatible "extensions".


That's an excellent way to stifle innovation.


The "function" keyword, for example. Why does it even exists?


You're 40 years too late to that game; `function' was in ksh83.

Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: comments inside command subst are handled inconsistently

[Chet Ramey]

On 7/31/23 8:22 AM, G. Branden Robinson wrote:


[John Mashey, the
 author of the predecessor Unix shell (and successor to Ken
 Thompson's original Unix shell) has gone on record as "goading"
 Bourne into doing questionable things in the quest for speed.


This is true. Mashey's v6 shell and Bourne's shell were the two competitors
for the research Unix shell. A series of head-to-head bakeoffs resulted in
both shells doing questionable things for performance.


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: comments inside command subst are handled inconsistently

[Chet Ramey]

On 7/30/23 2:21 PM, Martin D Kealey wrote:

Seriously though, the internet is awash with "helpful" Shell guides and 
examples loaded with poor practices. The only way this is going to improve 
is if there's a "linting mode" is included in the Shell and on by default.


Does shellcheck warn about features it considers obsolete?

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [PATCH] fix bind -X quoting

[Chet Ramey]

On 7/26/23 5:57 PM, Grisha Levit wrote:


It's just not transitive.



Another issue I didn't think of with printing the unquoted translated
command is that it can include newlines, which is a problem since you
have to read the `bind -X' output one line at a time to reuse it with
`bind -x'.

If there isn't a backwards compatible way to produce output that is
reusable given the current input format, I wonder if we can leverage a
format that's not currently valid as input.


Rather than embed more and more shell-specific output formats that readline
doesn't parse into readline itself, I'm more inclined to add a hook to
allow an application to print the value of a key binding itself. I don't
know if it makes sense to let the application display anything but macro
values, but something like

typedef void rl_macro_display_hook (const char *kseq, const char *value, 
int ptype);


where kseq is the key sequence, value is the `raw' macro value, and ptype
is a flag saying whether or not to print in a reusable way would be a
start at printing macro values. It's backwards compatible with other
versions of readline, since we're adding to the API and no application will
have this variable set currently.

This would allow us, for instance, to really go wild and make `bind -X'
display a series of `bind -x' commands to recreate the macro bindings.

That would also allow your proposal to have the display hook to use
whitespace separators with only `bind -x' having to understand how to read
them.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Tilde (~) in bash(1) is typeset incorrectly as Unicode character

[Chet Ramey]

On 7/29/23 10:05 PM, Bjarni Ingi Gislason wrote:

   Simply add

.if t .tr ~\(ti

   to "tmac/an.tmac",
instead of changing (hard coding) it in the sources (man pages).


This is probably excellent advice for the distros, but not something bash
is going to do.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [PATCH] fix bind -X quoting

[Chet Ramey]

On 7/31/23 9:30 AM, Chet Ramey wrote:

On 7/26/23 5:57 PM, Grisha Levit wrote:


It's just not transitive.



Another issue I didn't think of with printing the unquoted translated
command is that it can include newlines, which is a problem since you
have to read the `bind -X' output one line at a time to reuse it with
`bind -x'.

If there isn't a backwards compatible way to produce output that is
reusable given the current input format, I wonder if we can leverage a
format that's not currently valid as input.


Rather than embed more and more shell-specific output formats that readline
doesn't parse into readline itself, I'm more inclined to add a hook to
allow an application to print the value of a key binding itself. I don't
know if it makes sense to let the application display anything but macro
values, but something like

typedef void rl_macro_display_hook (const char *kseq, const char *value, 
int ptype);


So you end up with

typedef void rl_macro_print_func_t (const char *, const char *, int);

rl_macro_print_func_t *rl_macro_display_hook = (rl_macro_print_func_t *)NULL;

and have rl_macro_dumper call

	(*rl_macro_display_hook) (keyname, (const char *)map[key].function, 
print_readably);


if it's non-NULL, and let the hook function determine how to print the
binding.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: comments inside command subst are handled inconsistently

[Denys Vlasenko]

On 7/31/23 15:07, Chet Ramey wrote:

On 7/31/23 7:38 AM, Denys Vlasenko wrote:


In the spirit of increased compatibility across Unix world, it'd be
quite useful if shells stop inventing incompatible "extensions".


That's an excellent way to stifle innovation.


There is no reason to innovate in tools such as sed, awk, or sh.
They have fossilized. Use them when adequate, use something newer
when they can't do what you need.

For example, you can add floating-point math to sh. This would be innovation.
This would also break many thousands of existing scripts,
while being nowhere near enough for most serious uses
of floating point calculations, both in speed and features.
Not a good idea.

This happens time and again. A tool gets developed, then (if it's good)
gains widespread acceptance, but then it becomes tied by the compatibility
of the large installed base.

Another example: IRC.
Compare it to a browser-based chat website, (slack, googlechat, etc).
Imagine what would it take to improve IRC to be on par with the latter:
* audio/videoconferencing
* user profiles with pictures, time zones, snoozing, etc
* file store
* on-server search
* calendar
* etc

It would be a slow-motion nightmare to do so without breaking compatibility
with old IRC clients. The technology is just outdated. What it needs to do now
is not so much innovating, but continue to serve well the use cases
of the existing user base.

Re: [PATCH] normalization tweaks for macOS

[Chet Ramey]

On 7/25/23 4:06 PM, Grisha Levit wrote:

On Mon, Jul 24, 2023 at 10:12 AM Chet Ramey  wrote:


On 7/20/23 7:52 PM, Grisha Levit wrote:


I wonder if you'd be interested in changes
to the function that would skip escaping ASCII characters that aren't
glob-special as well.  I _think_ it would just be a slight variation
of the code that already does this for regexes but I haven't thought
about the edge cases carefully yet.


I agree that it would only be a slight variation on that code, and so I
added that case. It will be in the next devel branch push.


Re latest changes in [1], we need to preserve quoting also for any of
the following characters, at least if they are in a bracket
expression:
! - . : = ^


Why not just quote them all the time and not try to determine whether or
not they're part of a bracket expression?

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [PATCH] fix bind -X quoting

[G. Branden Robinson]

At 2023-07-31T09:30:30-0400, Chet Ramey wrote:
> Rather than embed more and more shell-specific output formats that
> readline doesn't parse into readline itself, I'm more inclined to add
> a hook to allow an application to print the value of a key binding
> itself.
[...]
> This would allow us, for instance, to really go wild and make `bind
> -X' display a series of `bind -x' commands to recreate the macro
> bindings.

I'm unlikely to be doing any of the work, but big +1 on this.  This
would be a valuable inspection and troubleshooting tool.  ("What does
bash _think_ I've got bound to this?")

Regards,
Branden


signature.asc
Description: PGP signature

Re: comments inside command subst are handled inconsistently

[G. Branden Robinson]

At 2023-07-31T16:08:31+0200, Denys Vlasenko wrote:
> There is no reason to innovate in tools such as sed, awk, or sh.

These are terrible examples to use; look at the history of all three.

> They have fossilized.

No, they've been _standardized_.  POSIX acknowledges that the common set
of expected functionality evolves; often to add features (see the
much-needed additions to standard make(1) in POSIX Issue 8, currently in
draft), but also to shrink in some necessary ways.  I think the Austin
Group (the POSIX team) deprecated gets() before WG14 (the C standard
committee) did.

> For example, you can add floating-point math to sh. This would be
> innovation.  This would also break many thousands of existing scripts,
> while being nowhere near enough for most serious uses of floating
> point calculations, both in speed and features.  Not a good idea.

It was done in ksh93.

Regards,
Branden


signature.asc
Description: PGP signature

Re: [PATCH] fix bind -X quoting

[Chet Ramey]

On 7/31/23 11:51 AM, G. Branden Robinson wrote:

At 2023-07-31T09:30:30-0400, Chet Ramey wrote:

Rather than embed more and more shell-specific output formats that
readline doesn't parse into readline itself, I'm more inclined to add
a hook to allow an application to print the value of a key binding
itself.

[...]

This would allow us, for instance, to really go wild and make `bind
-X' display a series of `bind -x' commands to recreate the macro
bindings.


I'm unlikely to be doing any of the work, but big +1 on this.  This
would be a valuable inspection and troubleshooting tool.  ("What does
bash _think_ I've got bound to this?")


`bind -X' already displays the key sequence and value, just without the
leading `bind -x' to make it really reusable. The default macro printer
displays macro bindings in a way that allows you to put them into an
inputrc file, but you're not doing that with `bind -x' anyway.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

A few suggestions for the GNU Manual

[Wiley Young]

Hi,
 Regarding the "Invoking Bash" page,
https://www.gnu.org/software/bash/manual/html_node/Invoking-Bash.html

It might be more consistent and therefore perhaps more clear for all of the
newbies out there if the description of "--restricted" also began with
"Equivalent to -r," as the very next description does.
"--restricted

Make the shell a restricted shell (see The Restricted Shell
).

--verbose

Equivalent to -v. Print shell input lines as they’re read."

Also, a few lines later, there's the phrase, "There are several
single-character options that may be supplied at invocation which are not
available with the set builtin."

Within that list there's the item,
"-r

Make the shell a restricted shell (see The Restricted Shell

)."
However, restricted mode is listed on the "Set Builtin" page:
https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html

"-r

Enable restricted shell mode. This option cannot be unset once it has been
set."
Cheers,
Wiley

Re: [PATCH] normalization tweaks for macOS

[Grisha Levit]

On Mon, Jul 31, 2023, 10:32 Chet Ramey  wrote:

> On 7/25/23 4:06 PM, Grisha Levit wrote:
> > Re latest changes in [1], we need to preserve quoting also for any of
> > the following characters, at least if they are in a bracket
> > expression:
> > ! - . : = ^
>
> Why not just quote them all the time and not try to determine whether or
> not they're part of a bracket expression?
>

I agree, that's a better approach. There's some performance benefit to only
preserving quoting that's required but getting this right is tricky and I
suspect not worth the increase in code complexity.

>

Re: comments inside command subst are handled inconsistently

[Ángel]

On 2023-07-31 at 09:15 -0400, Chet Ramey wrote:
> On 7/30/23 2:21 PM, Martin D Kealey wrote:
> 
> > Seriously though, the internet is awash with "helpful" Shell guides
> > and 
> > examples loaded with poor practices. The only way this is going to
> > improve 
> > is if there's a "linting mode" is included in the Shell and on by
> > default.
> 
> Does shellcheck warn about features it considers obsolete?

Shellcheck does warn on backticks.

[PATCH] fix getconf builtin ifdef's

[Grisha Levit]

The changes to getconf.c in 829aad36 (submitted in
https://savannah.gnu.org/patch/?10367) seem to have a typo. Should likely
be:

diff --git a/examples/loadables/getconf.c b/examples/loadables/getconf.c
index af5544f0..f3f1ffcd 100644
--- a/examples/loadables/getconf.c
+++ b/examples/loadables/getconf.c
@@ -274,11 +274,13 @@ static const struct conf vars[] =
 #ifdef _SC_AVPHYS_PAGES
 { "_AVPHYS_PAGES", _SC_AVPHYS_PAGES, SYSCONF },
 #endif
-#ifdef _NPROCESSORS_CONF
+#ifdef _SC_NPROCESSORS_CONF
 { "_NPROCESSORS_CONF", _SC_NPROCESSORS_CONF, SYSCONF },
+#endif
+#ifdef _SC_NPROCESSORS_ONLN
 { "_NPROCESSORS_ONLN", _SC_NPROCESSORS_ONLN, SYSCONF },
 #endif
-#ifdef _PHYS_PAGES
+#ifdef _SC_PHYS_PAGES
 { "_PHYS_PAGES", _SC_PHYS_PAGES, SYSCONF },
 #endif
 #ifdef _SC_ARG_MAX

[PATCH] fix getconf pathconf arg validation

[Grisha Levit]

$ getconf LINK_MAX
bash: getconf: pathconf LINK_MAX: (null): Bad address
undefined

diff --git a/examples/loadables/getconf.c b/examples/loadables/getconf.c
index f3f1ffcd..285f1b63 100644
--- a/examples/loadables/getconf.c
+++ b/examples/loadables/getconf.c
@@ -1059,12 +1059,12 @@ getconf_one (WORD_LIST *list)
   return (EXECUTION_FAILURE);
 }

- if (c->call_name == PATHCONF && list->next == 0)
+ if (c->call == PATHCONF && list->next == 0)
 {
   builtin_usage ();
   return (EX_USAGE);
 }
- else if (c->call_name != PATHCONF && list->next)
+ else if (c->call != PATHCONF && list->next)
 {
   builtin_usage ();
   return (EX_USAGE);

[PATCH] getconf CHAR_BIT -> CHAR_MAX

[Grisha Levit]

$ getconf -a | grep CHAR_BIT
CHAR_BIT 8
CHAR_BIT 255
---
diff --git a/examples/loadables/getconf.c b/examples/loadables/getconf.c
index 285f1b63..b0a854c8 100644
--- a/examples/loadables/getconf.c
+++ b/examples/loadables/getconf.c
@@ -125,7 +125,7 @@ static const struct conf vars[] =
 #ifdef _SC_CHAR_MAX
 { "CHAR_MAX", _SC_CHAR_MAX, SYSCONF },
 #else
- { "CHAR_BIT", CHAR_MAX, CONSTANT },
+ { "CHAR_MAX", CHAR_MAX, CONSTANT },
 #endif
 #ifdef _SC_CHAR_MIN
 { "CHAR_MIN", _SC_CHAR_MIN, SYSCONF },