bash completion after a multiline string
Hi All, I posted a question to STKO and someone suggest I should open a 'feature request' here. https://stackoverflow.com/questions/68065039/bash-completion-after-a-multiline-string Thanx in advance, Phi
Re: bash completion after a multiline string
May be posting a link is not appropriate so I cut/paste it here I bumped into this problem regarding bash completion, can't find reference to it. When doing $ echo foo "bar" /tm I got /tm expanded to /tmp/ that is indeed correct. But if I do $ echo foo "bar more bar" /tm No completion is done. My question is doest this behavior a feature or a bug? My bash version is GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu) Thanx in advance.
Re: Feature request: index of index
Meanwhile you may use functions to setup your variables something along
those lines.
PW$ function first-index { echo $1; }
PW$ function last-index { shift $(($#-1)) ; echo $1; }
PW$ declare -a array=([5]="hello" [11]="world" [42]="here")
PW$ declare -i first_index=$(first-index ${!array[@]})
PW$ declare -i last_index=$(last-index ${!array[@]})
PW$ echo $first_index $last_index
5 42
On Thu, May 6, 2021 at 1:24 PM Léa Gris wrote:
> Currently, to know the index of an array's indexes (example: first or
> last index), it needs an intermediary index array:
>
> > #!/usr/bin/env bash
> >
> > declare -a array=([5]=hello [11]=world [42]=here)
> > declare -ai indexes=("${!array[@]}")
> > declare -i first_index=${indexes[*]:0:1}
> > declare -i last_index=${indexes[*]: -1:1}
> > declare -p array indexes first_index last_index
>
> Which prints:
> > declare -a array=([5]="hello" [11]="world" [42]="here")
> > declare -ai indexes=([0]="5" [1]="11" [2]="42")
> > declare -i first_index="5"
> > declare -i last_index="42"
>
> It would be convenient to be able to index directly with this syntax:
>
> declare -i first_index=${!array[@]:0:1}
> declare -i last_index=${!array{@}: -1:1}
>
>
> --
> Léa Gris
>
>
>
function def crash
Hi All,
Dunno if this is a bug or pilot error, I bumped into this.
$ type echo
echo is a shell builtin
$ function echo { \echo the function version; \echo "$@"; }
After this I got different behavior on 'same' OS version, same BASH
version, meaning my environment must interact but can't tell to what extend
at the moment.
On 2 systems I got bash 'crash', probaly in recursion runaway, during the
function definition.
On 1 system the definition is OK, then the run for the function crash,
again on what's look a recursion run away.
I tried this tiny test case on debian 4.19.98-1 (2020-01-26) (little old)
and GNU bash, version 5.0.3(1)-release (i686-pc-linux-gnu) (little old too)
and I got the definition crash behavior.
I bumped into this as I tried to implement something that would tell me all
the 'command' definition of a word0 of a shell command, so having the same
identifier for a program basename, an alias and a function, I wanted to
make something (a function?) that would tell me
$ word0 echo
echo is an alias
echo is a function
echo is a program (a.out...)
I choosed randomly 'echo' as a test case, I could have used ls, etc
only echo seems to choke.
Cheers,
Phi
Re: function def crash
I discovered that
$ type -ta word
Would tell me what I wanted i.e all the defs of word :)
On Thu, Jun 24, 2021 at 9:48 AM Phi Debian wrote:
> Hi All,
>
> Dunno if this is a bug or pilot error, I bumped into this.
>
>
> $ type echo
> echo is a shell builtin
>
> $ function echo { \echo the function version; \echo "$@"; }
>
> After this I got different behavior on 'same' OS version, same BASH
> version, meaning my environment must interact but can't tell to what extend
> at the moment.
>
> On 2 systems I got bash 'crash', probaly in recursion runaway, during the
> function definition.
> On 1 system the definition is OK, then the run for the function crash,
> again on what's look a recursion run away.
>
> I tried this tiny test case on debian 4.19.98-1 (2020-01-26) (little old)
> and GNU bash, version 5.0.3(1)-release (i686-pc-linux-gnu) (little old too)
> and I got the definition crash behavior.
>
> I bumped into this as I tried to implement something that would tell me
> all the 'command' definition of a word0 of a shell command, so having the
> same identifier for a program basename, an alias and a function, I wanted
> to make something (a function?) that would tell me
>
> $ word0 echo
> echo is an alias
> echo is a function
> echo is a program (a.out...)
>
>
> I choosed randomly 'echo' as a test case, I could have used ls, etc
> only echo seems to choke.
>
> Cheers,
> Phi
>
Re: function def crash
Arghh! I knew it was a pilot error :)
unset PROMPT_COMMAND fix it :)
my PROMPT_COMMAND reference a function that do 'echo' and I should use
"command echo" in the echo function not \echo.
On Thu, Jun 24, 2021 at 10:51 AM Phi Debian wrote:
> I discovered that
>
> $ type -ta word
>
> Would tell me what I wanted i.e all the defs of word :)
>
>
>
> On Thu, Jun 24, 2021 at 9:48 AM Phi Debian wrote:
>
>> Hi All,
>>
>> Dunno if this is a bug or pilot error, I bumped into this.
>>
>>
>> $ type echo
>> echo is a shell builtin
>>
>> $ function echo { \echo the function version; \echo "$@"; }
>>
>> After this I got different behavior on 'same' OS version, same BASH
>> version, meaning my environment must interact but can't tell to what extend
>> at the moment.
>>
>> On 2 systems I got bash 'crash', probaly in recursion runaway, during the
>> function definition.
>> On 1 system the definition is OK, then the run for the function crash,
>> again on what's look a recursion run away.
>>
>> I tried this tiny test case on debian 4.19.98-1 (2020-01-26) (little old)
>> and GNU bash, version 5.0.3(1)-release (i686-pc-linux-gnu) (little old too)
>> and I got the definition crash behavior.
>>
>> I bumped into this as I tried to implement something that would tell me
>> all the 'command' definition of a word0 of a shell command, so having the
>> same identifier for a program basename, an alias and a function, I wanted
>> to make something (a function?) that would tell me
>>
>> $ word0 echo
>> echo is an alias
>> echo is a function
>> echo is a program (a.out...)
>>
>>
>> I choosed randomly 'echo' as a test case, I could have used ls, etc
>> only echo seems to choke.
>>
>> Cheers,
>> Phi
>>
>
Re: function def crash
Yes figured the hard way :)
Thanx
On Thu, Jun 24, 2021 at 1:51 PM Greg Wooledge wrote:
> On Thu, Jun 24, 2021 at 09:48:59AM +0200, Phi Debian wrote:
> > $ function echo { \echo the function version; \echo "$@"; }
>
> For the record, this isn't how you write a function that wraps a builtin.
> Use the "builtin" command instead.
>
> echo() {
> printf 'echo() wrapper invoked.\n'
> builtin echo "$@"
> }
>
> Or, if you aren't sure whether the command you're wrapping is a builtin
> or a program, use "command":
>
> echo() {
> printf 'echo() wrapper invoked.\n'
> command echo "$@"
> }
>
> The backslash hack only stops aliases from being used. It doesn't stop
> your function from calling itself recursively.
>
>
Re: bash completion after a multiline string
Thanx Chet for taking time to explain. May be readline API should have a way to know that a quote ['"`] is opened ine many previous lines and the first occurence of such quote ine the current one , is a closing one. Well I guess this is too much of a trouble for something living this way for such a long time :) Cheers,
Re: simple prob?
On Tue, Jun 29, 2021 at 10:23 PM L A Walsh wrote:
> I hope a basic question isn't too offtopic.
> Say I have some number of jobs running:
>
> > jobs|wc -l
> 3
> ---
>
> Would like to pass a varname to njobs to store the answer in, like:
>
> So I can run:
>
> > njobs n
> echo "$n"
> 3
>
>
This a a double question 'how to', and I see no bash bugs here.
The 2 questions are
- how do I pass a variable name as an output argument to a function ('n' in
your 'jobs n' example)
- how to set a variable in a sub command? that is no doable, a sub command
can't return variable to its parent, so you obviously have to do things
diffrently.
A simple 2 liners, solve all this, with no code injection blah
PW$ jobs
[1] Running sleep 111 &
[2] Running sleep 111 &
[3]- Running sleep 111 &
[4]+ Running sleep 111 &
PW$
PW$ function njobs
> { [ "$1" != "n" ] && typeset -n n="$1"
>typeset -a t ; readarray t <<<"$(jobs)" ;n=${#t[@]}
> }
PW$ njobs n ; echo $n
4
# explanations (you may skip here)
#===
[ "$1" != "n" ] && typeset -n n="$1"
This make sure the given output variable name is a valid SHELL identifier,
providing anything not valid in "$1" will break there.
This also enforce that the given $1 output variable name doesn't match our
own local nameref name, if it match we don't do our nameref, and re-use the
upper scope output variable name, that by definition is a valid variable
name if we got that far.
typeset -a t
define a local array that we will fill, being local mean the booking is
done at function return
readarray t <<<"$(jobs)" ;
Fill the array with your command you want to count lines for.
n=${#t[@]}
Fill the output variable
All is safe, all is clean, no 'apparent' temp file, no sub command :)
Shell programing is fun :)
Re: simple prob?
On Fri, Jul 2, 2021 at 9:24 AM Alex fxmbsw7 Ratchev
wrote:
> jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
>
This give
PW$ jobsn=( $( jobs -p ) ) jobsn=${jobsn[@]}
PW$ echo $jobsn
3644 3645 3646 3647
I guess you meant jobsn=${#jobsn[@]}
^ You missed the '#'
Yet there are some left over
PW$ jobsn=( $( jobs -p ) ) jobsn=${#jobsn[@]}
PW$ echo ${jobsn[@]}
4 3645 3646 3647
So it is not clean, and the OP want a function with a named output arg.
Yet to elaborate on your technic may be this one is a little cleaner
PW$ jobsn=$(printf '%c' $(jobs -p)) jobsn=${#jobsn} ; echo $jobsn
4
Re: simple prob?
On Fri, Jul 2, 2021 at 11:15 AM Alex fxmbsw7 Ratchev wrote: > good debugging, yea i missed the n # count char, i type stupidly from a > cell foun, .. > and yea leftover arr ekements exceots when u just use the first one as n > i just wanted to show shorter code > Yes I like those little one liners when other goes long way with multi fork/exec etc :), just fun challenges :)
Re: simple prob?
Ha yes I lost sight of <(jobs -t), I think it is a good improvement to the
challenge :)
Regarding the local t, you right, I missed to precise that any function
that do output parameters via nameref should namespace all their locals and
check the namespace that is indeed a bit combersome, that's why I tend to
stay away from that :) yet it was asked..
Here I typed to fast (challenged by providing the shortest answer :) ) and
throwed away my basics principles :)
Regarding the code injection I am not sure I got it.
If you are sitting at a prompt, why would you trick
unicorn:~$ njobs_ref 'x[0$(date>&2)]'
when you could simply type
unicorn:~$ date
I assume protected script/source (the ones you can't write into), are wise
enough not to run command based on user input, in short I guess no
protected script are doing thing like read in; eval $in :) that is the
simplest code injection :) and then would never let you have a chance to
enter 'x[0$(date>&2)]' at any time.
In all case since doing output parameter require some kind of name spacing
check it would reject input of the form 'x[0$(date>&2)]'
I guess a typical output parameter function should ressemble something like
this
function foo
{ [[ ! "$1" =~ ^[a-zA-Z_][a-zA-Z0-9_]*$ ]] &&
echo "Invalid parameter name '$1'" >&2 &&
return 1
[ "${1:0:4}" = "foo_" ] && echo "Namespace collision '$1' " >&2 &&
return 1
typeset -n foo_out="$1"
foo_out="value"
}
This one reject the bad 'x[0$(date>&2)]' only accept scalar variable names
as output parameter and reject scalar in the foo() namespace.
On Fri, Jul 2, 2021 at 2:01 PM Greg Wooledge wrote:
> On Fri, Jul 02, 2021 at 09:09:34AM +0200, Phi Debian wrote:
> > PW$ function njobs
> > > { [ "$1" != "n" ] && typeset -n n="$1"
> > >typeset -a t ; readarray t <<<"$(jobs)" ;n=${#t[@]}
> > > }
>
> <<<$() is a poor imitation of < <() which is what you really want.
>
> readarray -t t < <(jobs); n=${#t[@]}
>
> Combining <<< and $() only gives an approximation of the same result
> (the command substitution strips off all trailing newline characters,
> and then the here-string syntax appends one newline), and it's less
> efficient, because it involves slurping the entire input into memory,
> then writing it out to a temporary file, then opening the temporary
> file for input and unlinking it, then reading it back in a second time.
>
> Using < <() avoids the newline alterations and the temporary file.
>
> (Note: in some sufficiently new versions of bash, there may not be a
> temporary file in some cases. But it's still not the best solution,
> as it still involves storing and reading the whole output multiple times.)
>
>
> Stepping back a moment, you're using the name reference version of the
> "pass an output variable by reference" strategy. This requires bash 4.3,
> which is reasonable, and it requires some additional sanity checks which
> you did not show.
>
> What's really interesting to me is that you did a *partial* sanity check,
> refusing to create a circular name reference if the user passed "n"
> as their output variable name. But you forgot to check for "t", which is
> another local variable you're using. Also, in Linda's original example,
> the output variable was literally named "n", so choosing that as your
> name reference and explicitly disallowing it is a really spiteful choice.
>
> Finally, you didn't do any sanity checking of the output variable name
> (beyond comparing it to one of your two local variable names), so your
> function is susceptible to the same code injection attacks we discussed
> earlier in the thread.
>
> unicorn:~$ njobs_ref() { typeset -n n="$1"; n=42; }
> unicorn:~$ njobs_ref 'x[0$(date>&2)]'
> Fri Jul 2 07:54:49 EDT 2021
>
> As I mentioned a few days ago, all variants of the "pass a variable name
> by reference" method are basically equivalent to each other, and all
> of them need input sanity checking in order to avoid code injections.
> (Some of the variants avoid *some* flavors of code injection, but none of
> them avoid this one.)
>
>
Re: simple prob?
Ha ok, I got it, I was focused on our interactive session. Surely a script running on a UID behalf would be ill-advised to interpret (in the sense of shell evaluation) an arbitrary shell expression. On Fri, Jul 2, 2021 at 6:06 PM Greg Wooledge wrote: > On Fri, Jul 02, 2021 at 05:45:23PM +0200, Phi Debian wrote: > > Regarding the code injection I am not sure I got it. > > > > If you are sitting at a prompt, why would you trick > > > > unicorn:~$ njobs_ref 'x[0$(date>&2)]' > > > > when you could simply type > > unicorn:~$ date > > > > I assume protected script/source (the ones you can't write into), are > wise > > enough not to run command based on user input, in short I guess no > > protected script are doing thing like read in; eval $in :) that is the > > simplest code injection :) and then would never let you have a chance to > > enter 'x[0$(date>&2)]' at any time. > > For functions that you've written exclusively for personal use, it's > not an immediate concern. It's more of a thing that you want to be > aware of for the future. > > Where it becomes important is when you're writing scripts for other > people to use, or which run as different user accounts, or with > different privileges. > > The classic example of this is a script that's run by a web server in a > CGI environment, which accepts query parameters from the end user. If > one of those query parameters is used in an unsafe way, it can execute > undesired commands on the web server. > > Of course, there are *many* other places that shell scripts are used, > such as booting an operating system, starting various services, and > so on. In some of these cases, there is no external input being read, > or the external inputs are "trusted" files owned and edited only by > the system admin (root). But in other cases, untrusted input may be > read. > > So, there's merit in adopting a proactive strategy to shell script > security. Maintaining a slightly paranoid mindset can help you spot > potential security holes and possibly avoid disasters. > >
bash core dumps doing glob pattern on long string
I was looking at a bug on ksh93 that is "core dumps doing glob pattern on long string" and it happen that bash suffer the same. $ [[ $(printf '%010d' 0) == +(0) ]] I see 3 way of fixing this 1) [[ string == pattern ]] is for glob pattern, so string should be limited to PATH_MAX, so an upfront string length on string could prevent to call the glob pattern recursive functions, and then avoid the core dump. 2) Since some may have abused the glob pattern with long string bigger then PATH_MAX but smaller than core dump, imposing a PATH_MAX limit may break some wrong scripts, so instead we could have a fix recursion deep level, as we do have for shell functions calling, this hopefully should allow wrong doing script with abused string length to continue to run, yet avoiding core dump when reaching the limit, i.e break the call path. 3) Implement a stack deep check in the recursion, when getting close to the end of stack break the function trail (like function too deep for recursive functions). A last possibility is 'do nothing', since most of the people/scripts that works, don't care. Yet, the core dump limit is not the same on between bash and ksh93 then making porting hazardous. If the bash team plan to fix it, I'd like to know which way, so we could make ksh93 behave the same, for 1) and 2) it would be the exact same limit, for 3) it would be depending on stack usage, would not be the same exact string length that would break, but it would breaks instead of core dumping. Cheers,
Re: bash core dumps doing glob pattern on long string
@Oğuz A simple look at the core dump will suffice to convince you that the stack has overflowed. As Koichi stated, both ksh and bash do implement de 'simple' recursive approach. My question was only about the strategy that bash would follow, to converge with bash, i.e we can still both core dump, and if we decide not to then implement a common behavior.
Re: bash core dumps doing glob pattern on long string
Ok, I agry that PATH_MAX should not be considered, because not defined every where (OS's) If you decide a limit let me know I would use the same for ksh93 :) for the time being we core dump :) Cheers.
Re: bash core dumps doing glob pattern on long string
On Mon, Oct 10, 2022 at 9:08 PM Chet Ramey wrote: > That's not the point. The point is that shell pattern matching is used in > contexts other than filename globbing, and the shell shouldn't broadly > impose a filename-only restriction on those uses. > > Chet > Ok got it, I was confused because the manual occurences of the term GLOB in bash doc seems to appears only in the "Pathname Expansion" paragraph, which point to a sub-paragraph called "Pattern Matching" so it looked to me like if "Pattern Matching" applied to "Pathname Expansion" Inside "Pattern Matching" we see ref to things like glob-complete-word (glob-* :-) ) all apply to "pattern for pathname expansion" extglob If set, the extended pattern matching features described above under Pathname Expansion are enabled. nocaseglob If set, bash matches filename... So at the point I thought that globbing was kind of related to the max file path of an OS. On the contrary I see nor reference saying the 'pattern matching for pathname expansion can also be used by extension to any strings, but now I understand it is. Yet let me know the strategy or limit you choose, and I will do the same for ksh93 Cheers,
Re: bash core dumps doing glob pattern on long string
On Tue, Oct 11, 2022 at 7:23 AM Martin D Kealey
wrote:
>
> 4) compile globs into state machines, the same way that regexes get
> compiled, so that they can be matched without needing any recursion.
>
> -Martin
>
Do you mean the NFA colud trade recursion for malloc()'d backtracking point
record? this would trade stacksize vs bss size. Or may be you mean
converting the NFA into DFA, that I could not handle myself :)
That's why at first I thought limiting the input string length was may be
an option, but Chet demonstrated it was not.
Then returning an error instead of core dumps on recursion too deep on the
NFA walk seems to be the thing to do, and then I thought a stack
availabitly check for a given recursive level could be done, this is pretty
easy to do and almost costless regarding perf, specially because bash (and
other shells) have access to alloca(), well we could admit that os/arch
sans alloca() could still core dump.
A stack check with alloca() ressemble something along this lines
int stack_too_small(size_t s)
{ return alloca(s)!=NULL;
}
This non inlinable function do allocate the stack provision, return true if
not possible, false if possible and return the provision to the stack, the
caller is then assured there is enough space in the stack for its recursion
frame.
and the shell code would look like
... recursive_func(...)
{...
if(stack_too_small(recursion_frame_size)
{ handle nicely; // longjmp, return
}
What Chet call the right size to define is possibly something like the
recursion frame size, that is the stack needed by the current recursion
level instance (its locals) + all the frame size sub function would need
until the next recurence, i.e if the call path look like
a()-->b()-->c()-->a()...
The recursion frame size is sum(framesize(a)+framesize(b)+framesize(c))
Some empiric constant can also be used, for instance on linux a max
stacksize of 0x80 is pretty common deciding we big constant like
32K could be simple and good enough.
Re: bash core dumps doing glob pattern on long string
Hum, may be I was over optimistic regarding alloca(), it seems it core dumps as well :-), i.e never return NULL. I was mentioning alloca() because I use to do my own stacksize check using getrlimit() and getting the max stack size, and measuring the distance between the stack_base and the recursion stack level to iner how much stack remains, I didn't wanted to talk about that but seems it is the last resort... Well not a simple case to handle.
Re: bash core dumps doing glob pattern on long string
iner ==> infer sorry about typo's
Re: bash core dumps doing glob pattern on long string
On Tue, Oct 11, 2022 at 9:02 AM Martin D Kealey wrote: > Broadly I meant translating into a "preferably" Deterministic > (stackless/non-backtracking) Finite State Automaton. > > However I note that it's not always possible to make a Deterministic FSA > when you have repeatable groups which themselves don't have fixed lengths > (like a+(a|abbba|aabb*aba)b); either the extglob compiler would need to > start over and compile to a Non Deterministic (stacking) FSA, or just give > up and go back to the recursive approach. > > Personally I would favour the addition of «shopt -s dfa_extglob» that > would block the fall-back, causing extglobs that would need a stack to be > treated as magic match-never tokens. > > I say "extglob", but this would also speed up silly ordinary globs like > [a-z]*[a-z]*[a-z]*[a-z]*[a-z]*[a-z]*[a-z]*[a-z]*[a-z]*[a-z]*[a-z]*[a-z]*[a-z] > > -Martin > > Ok I see, that's true that when you see a glob pattern like +(a) one would think an optimiser could make it backtrack less. «shopt -s dfa_extglob» could be an option, Chet may say it is hard to explain when to use it... but we can go this path too, that is less of a hack regarding stack provision, but more of a hack regarding the extglob compiler :) Any implementation will please me :)
Re: `wait -n` returns 127 when it shouldn't
On Wed, May 17, 2023 at 12:21 PM Oğuz İsmail Uysal < oguzismailuy...@gmail.com> wrote: > > This boils down to the following > > true & > false & > wait -n > > There is no guarantee that `wait -n' will report the status of `true', > the shell may acquire the status of `false' first. It's not a bug > Ok for the randomness of result yet the $? should be 0 or 1 never 127 as the OP asked ? did I miss something?
Re: [PATCH] sleep builtin signal handling
Strange, on BASH_VERSION='5.1.16(1)-release' sleep don't get interupted by SIGWINCH ┌─none:/home/phi └─PW$ trap 'echo $LINES.$COLUMNS' SIGWINCH ┌─none:/home/phi # Resizing window here └─PW$ 79.80 └─PW$ 78.80 └─PW$ 77.80 └─PW$ 76.80 └─PW$ 75.80 ^C ┌─none:/home/phi └─PW$ sleep # Resizing window here, sleep is not interupted ^C ┌─none:/home/phi └─PW$
Re: [PATCH] sleep builtin signal handling
Well ┌─none:/home/phi └─PW$ type sleep sleep is hashed (/usr/bin/sleep) So sleep is not a builtin here.
Re: problem anomalies , possibly aliases related
On Thu, Jul 20, 2023 at 3:28 PM Greg Wooledge wrote: > > The idea that this would "work" is quite surprising to me. The basic > idea of a function is that it does stuff and then returns you to the > point where you were when the function was called. > > Really ? > In other languages, would you expect that you might call a function, > and have that function reach upward through the call stack and manipulate > your control flow? To name a few :-) In C longjmp() lands you anywhere on the the call path. Any try/catch language will do it too In C family there are even function that never return (see noreturn function attribute) FORTRAN alternate return is fun too. Not counting assembly where you can land anywhere with a ret *reg Basically this is hacker day to day job to return anywhere unexpected :-)
Re: formatting man pages in email (was: Assignment to RO variable)
I find it usefull and keep it :-) Thanx tons.
readonly [-aAf] [-p] [name[=word] ...]
> The given names are marked readonly; the values of these
> names
> may not be changed by subsequent assignment. If the -f
> option
> is supplied, the functions corresponding to the names
> are so
> marked. The -a option restricts the variables to indexed
> ar‐
> rays; the -A option restricts the variables to associative
> ar‐
> rays.
>
On Wed, Aug 16, 2023 at 8:06 AM G. Branden Robinson <
g.branden.robin...@gmail.com> wrote:
> At 2023-08-15T23:24:31-0500, Dennis Williamson wrote:
> > From man bash:
> >
> > readonly [-aAf] [-p] [name[=word] ...]
> > The given names are marked readonly; the values of these
> > names may not be changed by subsequent assignment. If the -f option is
> > supplied, the functions
> > corresponding to the names are so marked. The -a option
>
> That man page quotation came out somewhat awkwardly.
>
> I often find myself quoting man pages in email, so I have a shell
> function for this scenario. I call it "mailman", fully aware that
> that's also the name of mailing list manager. (Even if I ran it, I
> wouldn't ever do so at an interactive shell prompt, because it's a
> daemon.)
>
> mailman () {
> local cmd= opts=
> case "$1" in
> (-*)
> opts="$opts $1"
> shift
> ;;
> esac
>
> set -- $(man -w "$@")
> cmd=$(zcat --force "$@" | \
> grog -Tutf8 -b -ww -P -cbou -rU0 -rLL=72n -rHY=0 -dAD=l)
> zcat --force "$@" | $cmd | less
> }
>
> This relies upon man-db man(1)'s `-w` option to locate the requested
> pages (and it does the right thing if you specify file names, not just
> page topics).
>
> It also uses grog(1), much improved in groff 1.23.0 (released 5 July),
> to figure out which preprocessor(s) and macro package the document
> needs.
>
> I'll walk through those groff options.
>
> * `-Tutf8` formats for a UTF-8 terminal.
> * `-P -cbou` passes options to grotty(1) to turn off all ISO
> 6429/ECMA-48 escape sequences, _and_ all overstriking sequences; their
> formatting effects won't come through in plain text email anyway.
> * `-rHY=0` turns off hyphenation.
> * `-rLL=72n` sets the line length to 72 ens (character cells), which
> helps prevent ugly line wrapping.
>
> Two options are new groff 1.23 features.
>
> * `-rU0` turns off hyperlink support, so that any URIs in the man page
> will be formatted as text. This is a new groff 1.23.0 feature.
> * `-dAD=l` turns off adjustment (the spreading of output lines).
>
> Two more options are ones I use, but maybe only maintainers of man pages
> want them.
>
> * `-b` produces backtraces for any formatter warnings or errors.
> * `-ww` turns on all formatter warnings.
>
> I hope people find this useful.
>
Re: RFC: changing printf(1) behavior on %b
On Thu, Aug 31, 2023 at 9:11 PM Chet Ramey wrote: > > I doubt I'd ever remove %b, even in posix mode -- it's already been there > for 25 years. > > > Neither one is a very good choice, but `#' is the better one. It at least > has a passing resemblence to the desired functionality. > > Why not standardize another character, like %B? I suppose I'll have to look > at the etherpad for the discussion. I think that came up on the mailing > list, but I can't remember the details. > > Glad I red this thread before replying to the other one dealing with the same issue. I once worked on an issue on ksh93 regarding printf discrepency vs libc printf, and got replied that "ksh is not C". I Think we got to admit that shell's printf have departed from libc since long and now if a feature in libc appears and collide with printf(1) then we got to get yet another % exception char. In bash docco I see %b %q and %(datefmt...), so for a new feature we should get something that we think libc as little chance to target. My vote is for posix_printf %B mapping to libc_printf %b, with the idea that libc has little chance to have %B meaning UPPERCASE BINARY :-), as %x %X do. And yet one more line in the docco explaining this divergence.
Re: RFC: changing printf(1) behavior on %b
Well after reading yet another thread regarding libc_printf() I got to admit that even %B is crossed out, (Yet already choosen by ksh93) The other thread also speak about libc_printf() documentting %# as undefined for things other than a, A, e, E, f, F, g, and G, yet the same thread also talk about a A comming late (citing C99) in the dance, meaning what is undefined today become defined tomorow, so %#b is no safer. My guess is that printf(1) is now doomed to follow its route, keep its old format exception, and then may be implement something like c_printf like printf but the format string follow libc semantic, or may be a -C option to printf(1)... Well in all case %b can not change semantic in the bash script, since it is there for so long, even if it depart from python, perl, libc, it is unfortunate but that's the way it is, nobody want a semantic change, and on next routers update, see the all internet falling appart :-)
Re: bug#65659: RFC: changing printf(1) behavior on %b
On Fri, Sep 1, 2023 at 8:10 PM Stephane Chazelas wrote: > 2023-09-01 07:54:02 -0500, Eric Blake via austin-group-l at The Open Group: > > > FWIW, a "printf %b" github shell code search returns ~ 29k > entries > ( > https://github.com/search?q=printf+%25b+language%3AShell&type=code&l=Shell > ) > > Ha super, at least some numbers :-), I didn't knew we could make this kind of request... thanx for that.
Re: error message lacks useful debugging information
Since we are on the error path (not the perf path) may be the shell could go the extra miles and try some more diag, has it does for shebang, on ENOENT, the shell could try to open the a.out, if OK try some other euristics, at least the trivial one i.e the multilib case that seems the most disorientating one, the wrong arch (arm on intel) is already correctly handled, then 'may be' try an ldd (if ldd exist) to mention possible missing libs, all this leading to yet another 'error message' to be entered in the NLS... Or simply a reword of the current implementation bash: ./Candle: cannot execute: required file not found bash: ./Candle: cannot execute: Possible arch mismatch or missing shared libs Assuming no QA test already match the "required file not found", if so it would require yet another compat flag... With this more explicit error message the OP at least have some clue to of what to look for.
Re: error message lacks useful debugging information
On Thu, Oct 5, 2023 at 12:57 PM Greg Wooledge wrote: > On Thu, Oct 05, 2023 at 07:04:26AM +0200, Phi Debian wrote: > > Since we are on the error path (not the perf path) may be the shell could > > go the extra miles and try some more diag, has it does for shebang, on > > ENOENT, the shell could try to open the a.out, if OK try some other > > euristics, [...] > > Just for the record, trying to open the program for reading is only > possible for scripts. It's not too uncommon for a compiled program > to have +x permission for the user, but *not* +r permission. Such a > program can be executed, but not opened for reading. > > Correct, this is why I proposed the later case, when the shell can not find out more to say, just do nothing and simply suggest some known cases of trouble, yet if the shell can provide more insight, run file(1) and/or an ldd(1) could be usefull. This is the counter part for an interpreter of what compiler do, long time ago, diag from various compiler where very limited, and nowaday they try to be as accurate as possible.
Re: BUG: Colorize background of whitespace
On Wed, Oct 25, 2023 at 11:09 AM Holger Klene wrote: > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: linux-gnu > Compiler: gcc > Compilation CFLAGS: -g -O2 -flto=auto -ffat-lto-objects -flto=auto > -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security > -Wall > uname output: Linux BX-NB-015 5.15.90.1-microsoft-standard-WSL2 #1 SMP Fri > Jan 27 02:56:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux > Machine Type: x86_64-pc-linux-gnu > Bash Version: 5.1 > Patch Level: 16 > Release Status: release > > Description: > The initial bash background is hardcoded to some default (e.g. black) and > cannot be colorized by printing "transparent" tabs/newlines with > ANSI-ESC-codes. > Only after a vertical scrollbar appears, the whitespace beyond the window > hight will get the proper background color. > > Repeat-By: > run the following command line: > clear; seq 50; printf '\e[36;44m\nsome colored\ttext with\ttabs\e[m\n' > Play with the parameter to seq, to keep the line within the first screen > or move it offscreen. > > Reproduced in: > - in Konsole on Kubuntu 23.04 > - in the git bash for windows mintty 3.6.1 > - in WSL cmd window on Windows 11 > > I guess this is the way terminal emulator works, and I guess as well they are doing what real terminals use to do, though I have no such term at hand to prove :-) Worth to consider $ clear; seq 50; printf '\e[36;44m\nsome colored\ttext with\ttabs\e[m\n' | expand
Re: BUG: Colorize background of whitespace
On Wed, Oct 25, 2023 at 5:01 PM Greg Wooledge wrote: > > Ahh. That wasn't clear to me. Thanks. > > Ouch got caught the same way. This can be reduced to $ clear $ echo "\e[36;44;4m\nsome colored\ttext with\ttabs\e[m\n" $ # Recall and run prev command repeat the later until top lines scroll out. I added ;4 in the first \e i.e Underline to show that doing output is just a cursor movement (not a char painting) as would do any other cursor positioning esc seq. Doing so we see that on both the color and the underline are not decorated, on top lines, this is normal When the scroll out occurs, the underline is never painted, as expected, yet the BG color of the 'tab' is decorated. This is because there is a bogus \n right after the SGR sequence \e[36;44;4m\n this later \n say we open and SGR attribute, and \n will preserve it on scroll out, BUT colors at not completly SGR as stated here https://invisible-island.net/xterm/ctlseqs/ctlseqs.html search for *"ANSI X3.64-1979"* *When we remove the bogus \n all is normal* $ clear $ echo "\e[36;44;4m\some colored\ttext with\ttabs\e[m\n" many time. Generally speaking it is not good to leave an open SGR sequence before a \n, when emitting SGR before the \n it is wise to have a closing sequence \n[m\n
Re: unreliable value of LINENO variable
Confirmed on Ubuntu 22.04.3 LTS -- $ cat ~/yo.sh echo $BASH_VERSION echo 0 $LINENO if ((1)); then ( : ) | : ; echo 1 $LINENO fi echo 2 $LINENO $ for s in bash ./bash ksh zsh > do printf "\n$s\n" ; $s ~/yo.sh > done bash 5.1.16(1)-release 0 2 1 4 2 5 ./bash 5.2.21(5)-release 0 2 1 4 2 5 ksh 0 2 1 4 2 6 zsh 0 2 1 4 2 6
Docco
$ man bash ... CONDITIONAL EXPRESSIONS ... -a file True if file exists. -e file True if file exists. ... 'May be' would be nice for newbies to precise which options are [ specific vs [[ specific for instance -a file True if file exists ([[ only, for [ see test builtin) This to avoid things like $ [ -a /tmp ] && echo ok || echo nok ok $ [ ! -a /tmp ] && echo ok || echo nok ok I know it is obvious, unless this is intended to force a complete multi-pass man read...
Re: Docco
On Wed, Mar 27, 2024 at 10:28 AM Andreas Kähäri wrote: > On Wed, Mar 27, 2024 at 10:00:06AM +0100, Phi Debian wrote: > > $ man bash > > Would it not be untrue to say that "-a" is specific to "[[", as it is > clearly not the case? The fact that it is easy to confuse the two is > a different matter, but the documentation is correct for the current > implementation (which mimics the ksh shell with regards to the unary > "-a" operator). > > -- > Andreas (Kusalananda) Kähäri > Uppsala, Sweden > > . Ok may be my wording is not correct, but yet it require a good reading compile to get it right, first read all about [[ that is at the top of the man (at 5%), then get the 'CONDITIONAL EXPRESSIONS' distingo between [[ vs [ (at 40%) and finally get to 'test expr' (at 92%) to discover the whole thing about -a vs -e (same for other options), a little heads up when quick reading directly at 'CONDITIONAL EXPRESSIONS' would not hurt and not jeopardise the docco semantic I guess?
Re: Docco
Interestingly, the ksh docco say that 'Conditional Expressions' applies to [[ only :-) and then say the -a is obsolete. test expression later says the -a and -o binary oper- ators can be used, but they are fraught with pitfalls due to grammatical ambiguities and therefore deprecated in favor of in- voking separate test commands Ok let's forget about this...
How to run bash test suite
Hi All, I tried to subscribe to bug-bash but never got the confirmation mail. I grabed the latest source code available with my ubunto distro and made a bash build. All is fine. I'd like to run the test suite, and I found no docco about it. Did a brute "make test" from the build src dir, but I don't understand how to decipher all the output there, should I trust it and place it in a reference output before doing shell hacks experiments and rerun and compare outputs ? Or does the test self sufficient, i.e produce errors in its outputs ? I can see warning: may be errors will shows up as error: and then a simple make test >out 2>&1 is enough then grep error: out Any help appreciated. Cheers, Phi
Re: How to run bash test suite
Hi Chet, Thanx for the primer on QA :) I did manage to run them. I reworked run-all to run all this in parallel, I fixed some to ad more determinism in the output (background jobs), I got now a fast way to run the QA and that trig attention flag only when needed. Let's hack the bash now :) Cheers, Phi
Re: Bracketed paste mode breaks cooked mode's tab + backspace
Hi All, I don't reproduce it. This is not directly related to bash, it depend on the line discipline on the tty, I use ^H as erase and got no problems with . gnome-term, xterm are ok with , generate 0x8, terminator is confused is it doesn't use X resouces and then can't map 0x8 on Using generate 0x7f is a flaw since ages :) Cheers, Phi
Re: Bracketed paste mode breaks cooked mode's tab + backspace
Hi Egmont On Tue, Jan 30, 2018 at 9:47 AM, Egmont Koblinger wrote: > Not sure why you don't see this bug. (You have at least v4.4, and > started a new shell after enabling bracketed paste, is that right?) > One theoretical explanation could be that your kernel's tty driver is > slightly different and expects the leading ESC byte to also move the > cursor, resulting in 8 characters in total, making no difference in > the modulo 8 computation for the tabs. (I don't think this is the > explanation, though.) I guess I got an std kernel driver :) TC$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 16.04.3 LTS Release:16.04 Codename: xenial > > Even without bracketed paste mode, the issue should be reproducible > with commands like: > echo -ne '\e[1m'; cat > Obviously it's not bash's fault and there's nothing bash could or > should do with this one. Partially reproduce, is ok, i.e jump fwd 8 and backward 8 XX is ok got X jump fwd 7 X backward 7 Xwrong got X jump fwd 7 backward 4 (yet ok in the buffer) > > gnome-terminal doesn't use X resources either. gnome-terminal and > terminator both use the same VTE widget for terminal emulation, and > both have a setting under profile prefs -> compatibility to specify > what Backspace and Delete do. Gnome-terminal has its own backspace key mapping (so no xresource, yet mapped ok), terminator unfortunatly don't have it :) xterm use xresources, that what I use i.e xmodmap backspace give 0x8 xresources VT100 Backsapce 0x08 stty erase ^H With that I am almost an happy camper :) Cheers, Phi
command_not_found_handle() flaw
Hi All,
While trying to implement an function autoload feature, with a lazier
function loading and invocation than the example provided in bash doc, I
bumped into this bash bug (or feature, you let me know).
I asked stackoverflow for some ideas, one pointed me to
command_not_found_handle() that I was not aware off and looked promising
for what I wanted to achieve.
This SO post is quite big, so I will report here a shorter version.
In a nutshell to implement a function autoloading I want to plug into
command_not_found_handle(), but I need command_not_found_handle()be
evaluated in the shell context, not in a subshell.
At the moment BASH_VERSION='5.0.16(14)-release' (and all way back) the is
wrongly called as a sub-shell, that doesn't hurt the command-not-found
packages found in many distro as at this point they don't care about
setting things in the shell instance, so their implementation is good
enough.
To demonstrate the bug, suffice to do this
function command_not_found_handle
{ echo $$ ; sh -c 'echo $PPID'
}
When run in shell context we got the correct answer
$ function command_not_found_handle
> { echo $$ ; sh -c 'echo $PPID'
> }
$
$ command_not_found_handle
11370
11370
But when called from 'not found' context we got
$ ddd # dd not found on my system then call command_not_found_handle
11779
11783
$
As we see here, $$ lies pretend to be the shell PID 11779 while it real $$
is 11783, so a child.
This is a real problem because then there is nothing I can do in
command_not_found_handle) to setup things in the shell context, i.e not new
alias setup, no new functin define, etc...
I propose a fix, yet bear in mind hacking bash is not my day to day job :)
so it must be reviewed by gurus. I send the proposed [atch separatly.
With this fix I got this run
PW$ PS1='$ '
$ function command_not_found_handle
> { echo $$ ; sh -c 'echo $PPID'
> }
$ ddd
12165
12165
$ function command_not_found_handle
> { echo in cnf
> A="cnf-was-called"
> return 127
> }
$ A=''
$ ddd
in cnf
$ echo $A
cnf-was-called
$
Meaning now command_not_found_handle() is a real function called from the
shel context, shell context can then be touch (vars, alias, functions,
etc..) what I need to implement my autoload mechanism.
Not that it is indeed possible to chain the handlers, so assuming a
command-not-found package installed I got this on ubuntu.
$ declare -f command_not_found_handle
command_not_found_handle ()
{
if [ -x /usr/lib/command-not-found ]; then
/usr/lib/command-not-found -- "$1";
return $?;
else
if [ -x /usr/share/command-not-found/command-not-found ]; then
/usr/share/command-not-found/command-not-found -- "$1";
return $?;
else
printf "%s: command not found\n" "$1" 1>&2;
return 127;
fi;
fi
}
And a run gives
$ ddd
Command 'ddd' not found, but can be installed with:
sudo apt install ddd
If I now want to install my handler I rename the current handler
# Rename current command_not_found_handle
_cnf_body=$(declare -f command_not_found_handle | tail -n +2)
eval "function _cnf_prev $_cnf_body"
Then I create mine chaining to the prev one
function command_not_found_handle
{ echo in cnf
A="cnf-was-called"
_cnf_prev "$@"
}
$ unset A
$ ddd
in cnf
Command 'ddd' not found, but can be installed with:
sudo apt install ddd
$ echo $A
cnf-was-called
This shows that my handler was called, on failure to find a function, it
forward to the previous handler, then it shows that it ran in the shell
contect as A= is set correfctly.
I got no memory leak detected with this patch, but again review thouroughly.
Hope this helps.
Cheers,
Phi
Patch: command_not_found_handle() flaw
diff --git a/execute_cmd.c b/execute_cmd.c
index 3864986d..ef32631a 100644
--- a/execute_cmd.c
+++ b/execute_cmd.c
@@ -5370,6 +5370,20 @@ execute_disk_command (words, redirects,
command_line, pipe_in, pipe_out,
command = search_for_command (pathname, CMDSRCH_HASH|(stdpath ?
CMDSRCH_STDPATH : 0));
QUIT;
+
+#define PHI_CNF 1
+#if(PHI_CNF)
+ if (command == 0)
+ { hookf = find_function (NOTFOUND_HOOK);
+ if(hookf)
+{ wl = make_word_list (make_word (NOTFOUND_HOOK), words);
+ result=execute_shell_function (hookf, wl);
+ wl->next=0;
+ dispose_words(wl);
+ goto parent_return;
+}
+ }
+#endif // PHI_CNF
if (command)
{
@@ -5448,7 +5462,10 @@ execute_disk_command (words, redirects,
command_line, pipe_in, pipe_out,
if (command == 0)
{
+#if(!PHI_CNF)
+// with PHI_CNF==1 hookf==0
hookf = find_function (NOTFOUND_HOOK);
+#endif
if (hookf == 0)
{
/* Make sure filenames are displayed using printable
characters */
Re: command_not_found_handle() flaw
Hi All, I think it is a bug, it is not working as intended at least in the man page, it says. If that function exists, it is invoked with the original command and the original command's arguments as its arguments, and the function's exit status becomes the exit status of the shell. If that function is not defined, the shell prints an error message and returns an exit status of 127. A function invocation, don't imply a sub shell. Secondly, if yuio insist on going on the subshell path, then $$ and $PPID ought to be setup appropriately, here ther are plain bogus. Third, the argument saying to late we forked already is dumb, it is too late because the sequence hookf = find_function (NOTFOUND_HOOK); execute_shell_function (hookf, wl); is misplaced after the fork, while it should be done before the fork. I provided a fix that no one tested, that don't jeoparise the existing code or behavior, and to respect what a function invocation is. I can survive a "don't fix" well I can fix for myself, but has it is you must at least fix 2 points, fix the doc, sayin the command_not_found_handle() is called in a subshell and fix $$ and $PPID that are bogus in the subshell. BTW fixing $$ and $PPID will take more effort that just placing the hook before the fork, but that's your decision indeed :) I think that either command_not_found_handle() is a gross hack to satisfy command-not-found distro package, and in this case it should simply not be documented in the docco, or else it should be done as specifed i.e a function incovation is not a subshell running a function, that frankly is a bit ridiculous. Cheers, Phi
Re: command_not_found_handle() flaw
Hi All, Ok I got the picture now, and I owe you some apologize, but with what I got in front of me (ubuntu 20.04, bash-git latest) mistake from my side was easy to do. Regarding the docco, I admit I got the latest bash source (git) BUT I read the ubuntu 20.04 doc, that still DON'T mention the separate context (for the 2 years to come :) ), when reading the 5.x latest bash.1 I reckon the separate context is there, and up to that point (i.e with old doc in mind) I could not think that this function was a "separate context" Then the $$ vs $BASHPID confusion was a consequence of this. So now the questionning about the implementation of this 'feature' that seems to be, 'how to do something when a command is not found in an command interpreter' and the idea behind is the implementation of a mechanism for command interpreter (at large) to be able to suggest 'distro package suggestion', or if the goal was not that at least this is how it is used a the moment. To me, to fullfil the feature as stated here, 2 implementation where possible. - A pure 'handler' like a trap handler, then executed in the top shell context. This handler has then to be installed (sourced) in the shell startup (similar to now) - A pure script/prog invocation, then in a separate process, then separate context. Something along the line of BASH_CNF && $BASH_CNF "$@", this is pseudo code done in execute_disk_command(), i.e check for BASH var for user liking doing CNF and if so, simply prepand $BASH_CNF (likely to be command-not-found) (as currently it prepand command_not_found_handle) and retry search_for_command() once with the new run string, this way it would go and chase command-not-found in whatever place distro place it in the PATH, (Or user replacement), The distro would simply source BASH_CNF=command-not-found, and the user have control in her rc path for sdjusting the setting. A function invocation in a sub-shell (separate context) is the worst idea, simply because 1) It require doc fixup, what was done for 5.x and a clear refresh of the users mind, misleading path. 2) It need some ugly code in execute_disk_command() I think in the current logic (distro sugestion) a pure chase of command-not-found in PATH would have been simpler to document and implement. Yet I personnaly prefer the 'trap logic', because it serve the same services as current (distro sugestions) yet open the door for other things, like function autoloading, customised error handling on CNF. After all a CNF situation is an error, and a trap 'blah' ERR is correctly trigged, and the handler executed in the top shell context. But we don't want to implement CNF handling via trap ERR, because trap '' ERR is a bit heavy. So as an 'enhancemen't 'may be dreamin', having an ERRCNF pseudo signal would be ideal, could even keep the 'old' half hearted command_not_found_handle() (not a handler, not a script) and work together. I noted the subtility in that function name, it is not called command_not_found_handler() just because it is not an handler :) that's what should have warned me from the beginning :) Anyway, I will cope with that, and I have an implementation of lazy function autoload that works with the current implementation, command_not_found() simply sending a signal to main shell, saying we got a CNF :) ugly but it works. I love bash and all the evolution of it. Cheers, Phi
Re: [PATCH 0/4] Add import builtin
On Fri, May 3, 2024 at 5:26 AM Koichi Murase wrote: > > > By the name "import", I expect also something like an include guard, > i.e., it loads the same file only once at most. I have a > shell-function implementation,`ble-import', in my framework. It > doesn't support namespace, but it supports the include guard. It also > accepts a setting for the paths to search. It also has a feature like > `with-eval-after-load' of elisp (or the "onload" event). > > -- > Koichi > > I don't think we need yet another builtin named 'import' for all the reason mentioned in the mail thread. Setting BASH_IMPORT_PATH in an .rc file is not less confusing than simply setting PATH in the same .rc file. In the case one mix'n'match dir setup in both PATH and BASH_IMPORT_PATH, and in case of mix'n'match -x more setting of a .sh files, then all sort of documentation must be done to explicitly tells which case takes precedence etc... I prefere by far to handle $PATH myself in my .rc and place myself my 'shell packages' in my hierarchies. Note that a package (an import) generally is non executable, as some mentioned, yet beeing able to 'run' a package can be interesting in some case, for instance a pacakge run can display its embedded documentation, useful during development (yet debatable) My package management have the same feature as @Koichi one, i.e a C like #pragma once. I add a 'force' option to my 'import', this is for interactive 'import', to bring more functions (global, alias), yet during bring-up one may fix a package an force a reload. Some packages, on the other hand, are to be used 'imported' by other packages and as such I have versioning mechanism, so 'import' can tell about new version available from the git repos, or even an auto update from git (when suitable) etc... and indeed it is possible for the import to tell which minimal version is needed for the 'import'. So doing an 'import' feature, may be to be on par with python/perl/nameit, require more than just a new ENV var and the hack to use it. Cheers
Re: Re: Re: [PATCH 0/4] Add import builtin
On Sat, May 4, 2024 at 4:44 AM Matheus Afonso Martins Moreira <
math...@matheusmoreira.com> wrote:
>
> By "library system" I just mean the exact mechanism through which
> bash will load a "library". By "library", I mean ordinary scripts
> whose purpose is to collect related functions and variables
> for reuse by other scripts or interactive bash users.
>
May be that's why the term 'library' is not very well suitable as there are
other ways to load functions and variables in a shell context, i.e shared
libraries/objets dynamic builtins, so the 'import got to cover all of none.
In this case the import would find the shared libs in the restricted path
IMPORT_PATH along with the scripts, and then a 'library' could either be
text script or binary .so a.out.
Sounds more like a pandora box :-)
> By introducing the "module/library system" I want to do the following:
>
> 1. Add a builtin primitive that can be used to load libraries
> 2. Establish a convention for where bash will look for libraries
> 3. Separate the libraries from the commands/executables
>
If its all that simple may this 3 liners to stick in your.rc file or hide
it in yet another . findable script
shopt -s expand_aliases # Some may dislike this
xyz_import='for xyz in ${XYZ_IMPORT_PATH//:/\/$1 }/$1;do xyz=$(realpath -q
$xyz);[ -r "$xyz" ]&&. $xyz&&return;done'
alias xyz_import='source /dev/stdin <<< "$xyz_import"'
1. It create a new primitive (well a command in shell parser vocable)
xyz_import
2. It Establish a convention for where bash will look for libraries
XYZ_IMPORT_PATH
3. Separate the libraries from the commands/executables by looking in the
$XYZ_IMPORT_PATH dir set only
Murphy's law
Note the I namespaced all this with xyz/XYZ, change for your taste, but
beware of other 'import'
While testing this I discover (and forgot) that I installed the
'imagemagick' package (linux debian) which in turn setup an alternative
path on 'import'
$ ll /usr/bin/import
lrwxrwxrwx 1 root root 24 Feb 26 07:43 /usr/bin/import ->
/etc/alternatives/import
$ ll /etc/alternatives/import
lrwxrwxrwx 1 root root 23 Feb 26 07:43 /etc/alternatives/import ->
/usr/bin/import-im6.q16
So I guess it is wise to stay away from 'import' name.
Re: [PATCH 0/4] Add import builtin
On Sun, May 5, 2024 at 9:47 PM Greg Wooledge wrote: > On Sun, May 05, 2024 at 03:32:04PM -0400, Lawrence Velázquez wrote: > > > The idea to add a BASH_SOURCE_PATH variable that gets searched before, or > instead of, PATH when using the source builtin -- that sounds good. I > don't really understand the concepts behind the rest of the discussion. > > Well even this is unclear 'BASH_SOURCE_PATH get searched before PATH' or 'BASH_SOURCE_PATH get searched instead of' or even 'BASH_SOURCE_PATH get searched after PATH' Each have valid reasons to exist... Anyway if this is the only desire then a simple In module system init .sh (used by .rc or other .sh) IPATH=... ; alias xyz_import ='PATH="$IPATH" source' ; alias xyz_type ='PATH="$IPATH" type' Above the setup is for 'instead of PATH', 'PATH="$IPATH:$PATH" source' for before PATH and so on. Then you can use xyz_import foo.sh or xyz_type foo.sh BTW I didn't red all the patches, but I wonder what the semantic of the builtin 'type' becomes in your implementation, I guess that on top of currently defined concepts (file, function, alias, builtin) you now add 'library (better name needed)' with all the docco and jazz about it. The risk/profit (or even ROI) seems not favorable, i.e the implementation, the QA tests, the docco. For something that can be done in a 1 liner in an .rc file, I am really not convinced.
Re: Re: [PATCH 0/9] Add library mode to source builtin
On Mon, May 6, 2024 at 5:43 AM Matheus Afonso Martins Moreira < math...@matheusmoreira.com> wrote: > > I think every single use of the term "library" in this whole endeavor > > is misleading and misguided. > > I don't think so. A library is just a logical collection of code and data > that you can use to implement software. This is a mechanism meant > to load those libraries separately from executables. > Not in my parlance. Libraries are 'compiled collection of object module', can be static/dynamic, dynamic imply a runtime load. This is valid for interpreters with JIT compiler that can produce librairies. Bash do have libraries by means of loadable .so files, .so files can legitimatly called libraries. Those libraries do have their own PATH thing (i.e LD_LIBRARY_PATH) A 'script' is a bash source file with the +x bit set (executable), and as such don't have extension like 'foo'. A non executable bash file, better with a .sh extension, is a bash source. The source builtin just load 'file', albeit .sh file, not library.
Re: Re: [PATCH 0/4] Add import builtin
On Mon, May 6, 2024 at 7:28 AM Matheus Afonso Martins Moreira < math...@matheusmoreira.com> wrote: > Yet the feature has been described as "irritating"! > I really don't understand the cause for this > and it's making me feel really unwelcome. > I think it is not personnal, you proposed something, and other told you what you propose is overkill due to the simplicity to do the same with no change in bash, and a 1liner of code in your top level project file. Your proposition could not be 'builtin', you want a new 'model' with 'packages/module' (not saying library on purpose, way too much for me). Starting from there your model need at least an init line to say you want bash operating in your model 'packages/module'. Form there your new model can be initialized with a 1liner, either a dynloadable builtin (.so file you provide), or an init file (.sh no +x) you source. At this point whatever technic, you have your 'import'. Is that big of a deal to ask your developers to comply with your model and init your 'packages/module' paradigm bringing 'import' in a 1 line in their top script (here script mean +x executable bash source file). TBH, I have a packages system for both bash and ksh since decades, they all start by an init line at top level, and they never required any shell hack, and they do far more that intended here, repos, versioning, dependencis, multi-arch (run time context) etc... I think it is very good to be enthusiastic about hacking a free software, a good start point is to fix bugs, even tiny one, even simple docco. Another option is to create a fork of a project (here bash) implement your hack, provide it (github kind) then see the stats, if your new bash skyrocket, may be your hack will be back ported. You could adverstise your new bash in stko, each one asking for a package manager in bash you could reply with your new invention, boosting the stats. All that to say, I am not too sure that distro (and installer of all sort) will be willing to get a fatter bash for higher risk of bugs, hack, secu, name it, and would surely ask this feature to be optional, then always requiring an init line to enable it. So I guess it will take a long time before seeing that in the shell, but I may be wrong.
Re: [PATCH 0/4] Add import builtin
On Mon, May 6, 2024 at 7:51 PM Kerin Millar wrote:
>
>
> I'll put it in a little more detail, though no less plainly. I find the
> terminology of "libraries" and "modules" to be specious in the context of a
> language that has no support for namespaces, awkward scoping rules, a
> problematic implementation of name references, and so on. These
> foundational defects are hardly going to be addressed by a marginally more
> flexible source builtin. Indeed, it is unclear that they can be - or ever
> will be - addressed. Presently, bash is what it is: a messy, slow, useful
> implementation of the Shell Command Language with an increasing number of
> accoutrements, some of which are fine and others of which are less so (and
> virtually impossible to get rid of). As curmudgeonly as it may be to gripe
> over variable and option names, this is why the import of library, as a
> word, does not rest at all well in these quarters. That aside, I do not
> find the premise of the patch series to be a convincing one but would have
> little else to say about its prospective inclusion, provided that the
> behaviour of the posix mode were to be left unchanged in all respects.
>
> --
> Kerin Millar
>
>
Thanx @Kerin, I got an intuitive reluctance with the patch series, but
could not formalize it that way, that is exactly it (specially the nameref
to me :-))
That brings up some questioning about the bash dev workflow. I personally
only monitor bash-bug (not the others bash-*), specially to be aware of new
incoming patches.
Generally, the few patch that shows up here are patches that fix a specific
standing bug, or on some occasion, the declaration of a bug along with a
patch to fix it, they are generally reply with 'thanx for the report and
patch!'
I rarely see patches about "Hey guys I got this great idea what do you
think", so I wonder for this kind of request does git{lab,hub,whatever} be
more appropriate? or may be I should say convenient, a public git project
(clone) generally have all the infrastructure for discussions, enhancement
and fix.
The bash code owner can then pull the great idea when the demand for it
start roaring?
Just questioning, I really have no idea what is the bash dev team way of
receiving enhancement request.
Re: [PATCH 0/4] Add import builtin
Ok thanx for clarification.
I agry about git{lab,hub} position that's why I added {,whatever} meaning
something really free.
Ok now all is clear and I will not be surprised anymore if some other
'feature' proposal shows up in the future.
Re: [PATCH v2 5/8] builtins/source: parse the -i option
On Sat, May 18, 2024 at 3:23 PM Matheus Afonso Martins Moreira < math...@matheusmoreira.com> wrote: > > > That would cause shell scripts to see it and exhibit a change in > behavior. > > Only if the -i option is not implemented. > If it is, there will be zero change in behavior > whether BASH_SOURCE_PATH is set or not. > Only if the script explicitly passes -i to source > will the behavior change. > I am xing fingers the consensus, if any of theses takes off, is to have the -i implemented (not omitted) along with a protection against alias source='source -i' I really depend on actual 'source' behavior, and can't update my packages (distributed since long). I don't plan to use BASH_SOURCE_PATH, so if one, distro, sysadmin, and ultimatly the user fiddle with BASH_SOURCE_PATH in the ENV (*rc files), changing the good'ol 'source' behavior, it would be a serious regression to me. So to me BASH_SOURCE_PATH enablement must be guarded by a never used yet option to 'source or dot' (here -i apparently) and an enforcement that no alias source='source -i' could ever be possible. All this is because one writing a 'main' script (#!/bin/bash), sourcing a package of mine through my package management system, expecting the current source behavior, and later add the loading of a 'libraries' from a friend (terminal color jazz) that in turn start to mess around with alias source='source -i' and BASH_SOURCE_PATH cold start to break my way of finding my packages. An alternative to this would be BASH_SOURCE_PATH="" don't do nothing with 'source', and allow the 'libraries' designer to do typeset -r BASH_SOURCE_PATH="" in the early step of the main script, so nobody will ever be able to do anything clever with BASH_SOURCE_PATH, or could add import from incompatible packages system. But even this idea is a pb to me because it would require a change an my side that is not really doable. Well in short don't break the old source semantic.
Re: [PATCH v2 5/8] builtins/source: parse the -i option
On Sun, May 19, 2024 at 1:17 AM Matheus Afonso Martins Moreira <
math...@matheusmoreira.com> wrote:
>
> Those should continue to work without any changes.
> They should even be compatible with each other:
> there should be no problems with sourcing -i a script
> which then sources another script normally or uses
> one of the module management solutions.
>
> -- Matheus
>
Not true, the hack to break the whole thing is
old source.sh relying on the current {source,.,$PATH} behavior
The source.sh is alive (maintained)
One come along a say look I got this neat library to colorize stuff, it
depend on a couple of other library (meaning they do source -i inside) the
library (the top one may be) decide that source -i is the general way to go
a decide a despotic alias source='source -i' this could be a general setup
of this package manager.
Back to good'ol source.sh now any 'source' it does are biased.
I agry that true to for other shell options/variable,
'libraries/packages/imports' can break the whole thing too though (IFS is a
good one) but so far we managed :-)
So at the end, the poor one that rely on actual {source,.,$PATH}would have
to warn in its coding convention, never ever source a .sh that rely on
BASH_SOURCE_PATH,
I know a grep would suffice to enforce it, but there is no reasons to force
that.
Another things that bugs me, why not implementing your invention in a
dynamically loaded builtin, advertise it on github, and let the one who
really needs this great thing be their choice download it, and use it.
I personally have my own bash and own ksh for our internal needs, we
maintain them, and we have our own set of internally crafted builtins, I
would not dare push them to shell maintainers, fearing breaking all who
depend on bash/ksh/dash whatever for their OS startup, init script,
configure script etc...
Re: [PATCH v2 5/8] builtins/source: parse the -i option
On Sun, May 19, 2024 at 10:58 AM Matheus Afonso Martins Moreira < math...@matheusmoreira.com> wrote: > > This is not a problem that's introduced by this patch. > People can already do that today. Anyone could write > `alias source='rm -rf --no-preserve-root /*'` right now, > nothing stops them. So I don't see why this would be > a reason against inclusion of this feature. > I was not talking about breakage of a script, anybody knows you can stick an rm -rf anywhere in your script (BTW protection/containers,... mitigate the damages). I was talking of two incompatible package manager, where one can break the other, yours can break mine, mine can't break yours, I was asking for a way to secure the old (current) way for mine. So far I saw typeset -r BASH_SOURCE_PATH='' in my init path, but not even sure it will be enough, beside I don't control my users who may miss the init update but still can mix'n'match the package managers. Here, we are not talking adding one more feature into bash, we are talking about one feature predate (intentionally or not) another one, it is normal that some head's up occurs. > I understand your concern though. Perhaps we can find > ways to work around it or fully address it. > Not too sure about that. > > > Another things that bugs me, why not implementing your invention in a > > dynamically loaded builtin, advertise it on github, and let the one who > > really needs this great thing be their choice download it, and use it. > > I don't really want to maintain such a thing. I'm struggling to find time > to work on my personal projects as it is. We are lucky to have someone > who maintains bash and who is considering this feature for addition. > Strange argument, you want to forward the hot potatoes to someone else regarding maintenance. First of all you claim the feature is tiny, almost bug free in its inception, secondly it is vastly needed. If those assertion are true, then your maintenance workload is almost nil for your feature you can always access on top of bash with dynload. If there is a vast need for it, and if unfortunately a bug shows up, then in this vast population, there will be a good soul providing a pull request for it, your load would be minimal. That's the magic of open software :-) So I think a dynloadable builtin of yours would be a perfect use case of dynload, and a good way to challenge your invention, i.e measure the vast population, bug rate, etc... I think it is a good 'preview' way, before eventually move from dyn load to static load one day. Dynloadable builtins exist, use it. > I don't want to fork bash. I want to be able to download the latest version > with my package manager and get access to this. It is perfectly doable, clone, configure, build, stick you patch and provide it to you (or your users) not a big deal of effort. You can have that today, all that hidden in... bingo a library :-) > If I had to fork bash to > get this, I would simply give up and continue working on my own > programming language instead. > I think this is a great valuable idea! PS : On the citations side, I remember one, a software is bug free when you trash it :-)
Re: [PATCH v2 5/8] builtins/source: parse the -i option
On Mon, May 20, 2024 at 7:54 PM Greg Wooledge wrote:
> On Mon, May 20, 2024 at 07:43:10PM +0200, Andreas Kähäri wrote:
> > On Mon, May 20, 2024 at 05:31:05PM +, Matheus Afonso Martins Moreira
> wrote:
> >
> > PATH=${BASH_SEARCH_PATH-$PATH} . file
> >
> > without the need to add any options to . or to source. But maybe that
> > too pedestrian?
>
> Are we going in circles yet? This would clobber the value of PATH for
> the duration of sourcing "file", which would potentially cause commands
> in "file" to break.
>
> hobbit:~$ cat bar
> echo hi | cat
> hobbit:~$ PATH=. source bar
> bash: cat: command not found
>
Yes we are probably circling because of the schizophrenic nature of the
feature.
At the beginning, I think not sure I remember well, the whole idea was to
get bash 'libraries' described as bringing more functions written by other
into a runnable (-x) script. This was perceived (I guess) as an 'include'
but since it was function may be as a 'link' or both combined, and as such
required a special treatment ala -I -L compiler like using INC_PATH,
LIB_PATH kind of thing.
Some thought 'source' was a good candidate for this, except that source is
a read/parse/eval feature while the intention was more a read/parse for the
benefit of the 'importer'
so construct like
hobbit:~$ PATH=. source bar
is perfectly ledgit if we are honest and admit bar is an import only and
execute no code, simply bring functions.
So PATH=$BASH_SOURCE_DIR . greatlib
is the way to go, still assuming libs are functions/vars only and no code
execution.
As soon as the 'library' concept assume that eval is also possible, along
with read/parse, then kaboom we need PATH back so the
PATH=$PATH:/path/to/lib .
Real life package actually require the eval, because it is the way to
configure a 'library' depending the user context.
So back to square one, what we need is source as it is and should not be
broken.
Apparently konsolebox wrote a package manager, and survived the source 'as
is', yet he still advocate for 'enhancement'. I do have a package manager
too, and survived source 'as is' and don't require any enhancement.
'May be' bash could investigate the ksh93/zsh $FPATH autoload, but don't
know if that would be good enough for the initial purpose.
Don't know if that FPATH thing was investigated before, if so, why it was
rejected. If good enough and could be adopted, it would be some convergence.
Re: [PATCH v2 5/8] builtins/source: parse the -i option
On Tue, May 21, 2024 at 1:15 PM Greg Wooledge wrote: > On Tue, May 21, 2024 at 10:12:55AM +, Matheus Afonso Martins Moreira > wrote: > > > the schizophrenic nature of the feature > > > > First the feature was "irritating" ... Now it's "schizophrenic" ? > > I must be mentally ill for trying to contribute this? > > > > Yeah, I'm done. > > I don't think "schizophrenic" was used as an insult. Rather, it looks > like an attempt to describe the fact that everyone in the thread has a > different concept and/or experience regarding how 'source' and '.' > should work. > > Well, what I really meant was that there are 2 directions one with read/parse the other read/parse/eval and not being able to make a mind, like the Buridan ass https://en.wikipedia.org/wiki/Buridan%27s_ass 'import' read/parse (along with a PATH var) make sense and don't break interfere with source A 'source' read/parse/eval is there with its semantic and should not go away. An 'import' with source look alike and tweaking its flags and semantic is border line, besides, it and can be implemented in shell only with no bash modification, there is no perf consideration here, we are not on the perf path, unless one come up with a source in the inner loop :-) No offense, just pointing that these two path sounds not mixable, an implementation of your $BASH_SOURCE_PATH can be all done in shell in one of your libs, (say your bootstrap lib) whence this one is loaded, it can load all the others.
Re: [PATCH v2 5/8] builtins/source: parse the -i option
On Tue, May 21, 2024 at 1:17 PM Koichi Murase wrote: > There are already shell-function implementations at > /examples/functions/autoload* in the Bash source. They reference FPATH > to load functions, though one needs to call `autoload' for each > function in advance (by e.g. `autoload "$fpath_element"/*' ). > > However, I personally do not think the FPATH mechanism is useful > because a file can only contain one function per file. Significantly > non-trivial functions are usually implemented by a set of helper > functions or sub-functions. Also, in libraries, we usually have a set > of functions that are closely related to one another and share the > implementations. I don't think it is practical to split those > functions into dozens or hundreds of files. It would also be slow to > read many different files, which requires access to random positions > on the disk. > Jeez, I forgot about this examples/functions/autoload. First of all thanx for pointing it, it demonstrate that things can be implemented as bash code (not C hack in bash) at least in the experimental phase. Second, there is no such limitation about 1 source 1 function mapping, since it is sourced any functions defines in the files become available. This open two path. 1 package setup function is autoloaded bringing all the other function that are now callable Say file foo defines foo() bar() then autoload foo() brings both foo() and bar(), if foo is the setup thing for file foo then the doc sez autoload foo to bring up bar() along. Another path is to have a symlink on each 'exported' function on the file foo, so a ln -s foo bar would make bar autoloadable. Note I used 'exported' here meaning the foo file may define a lot of function, only the exposed API one need a symlink. The real divergence from this autoload, and the ksh93 one is that ksh93 will try an autoload on foo using FPATH on the 'command-not-found' situation, something bash don't handle in the user context (it does it in a subshell for unknown reasons). So basically in ksh93 script, beside setting FPATH, you simply call functions directly, on command-not-found then try FPATH to find your function and if found proceed with the load, define func, run setup code, and run the func with parameters... This is neat, but the fact we don't have that on bash is no big deal, we can live without it, again no need to break things that works.
Re: proposed BASH_SOURCE_PATH
@Greg, @Martin +1, lost sight of the feature, and xing fingers that current semantic/behavior is not destroyed, @Oğuz -1 I'd like to avoid fixing script that run today just because bash was updated or I would advocate distros to keep a frozen bash as macos did.
Re: pwd and prompt don't update after deleting current working directory
Oops I forgot the reply to all in my reply to @David Hedlund I was on the same tune as you @g...@wooledge.org . == On Fri, Jul 12, 2024 at 12:14 AM David Hedlund wrote: > > > When a directory is deleted while the user is inside it, the terminal > should automatically return to the parent directory. > > Jeez, this looks like pretty dangerous isn't it, imagine a script (or a fast typer for interactive shell) the is doing some bookeeping inside a directory, (i.e removing some files in the $PWD), and $PWD is unlinked by another random process, then your script (or fast typer, who type and read afterwards) would then continue its bookeeping in the parent (of login shell dir?). It make no sense. So when a $PWD is unlinked while working in it, the only things a shell can do is to emit a message like $ rm -rf $PWD $ >xyz bash: xyz: No such file or directory Now you may argue this is not explicit enough, nothing in there tells you that the $PWD was unexpectedly unlinked. You may implement something more explicit like this for interactive shell, for script simply bail out on error. $ PS1='$(ls $PWD>/dev/null 2>&1||echo $PWD was unlinked)\n$ ' $ pwd /home/phi/tmp/yo $ ls asd qwe rty $ rm -rf $PWD /home/phi/tmp/yo was unlinked $ ls /home/phi/tmp/yo was unlinked $ cd .. $ mkdir -p yo/asd ; >yo/qwe >yo/rty ; cd yo $ pwd /home/phi/tmp/yo = Bottom line, this is not a bug, and there is no enhancement request here. Or 'may be' on bash error bash: xyz: No such file or directory the full path could be expanded bash: /home/phi/tmp/yo/xyz: No such file or directory But even that I am not sure.
Re: procsub doesn't release the terminal without reading one byte
On Wed, Oct 9, 2024 at 8:02 PM Oğuz wrote: > The first letter I type after running this command won't show up on > the terminal: > > : < <(cat) > > This is reproducible on 5.3 beta too. > > Oğuz > > May be your 1st char is eaten by the dangling cat(1) >From /dev/pts/0 === ┌─none /home/phi └─PW$ : < <(cat) ┌─none /home/phi └─PW$ >From /dev/pts/1 === ┌─none /home/phi └─PW$ psg cat phi21786 21181 0 12:17 pts/000:00:00 cat ┌─none /home/phi └─PW$ psg cat phi21786 21181 0 12:17 pts/000:00:00 cat ┌─none /home/phi └─PW$ kill -9 21786 >From /dev/pts/0 again === ┌─none /home/phi └─PW$ echo yo yo Note you can get a better effect with ┌─none /home/phi └─PW$ : < <(sed '') # :-)
Re: "printf %d ''" should diagnose the empty string
On Sat, Feb 8, 2025 at 8:51 AM Robert Elz wrote: > > There are a gazillion different ways of getting rid of the octal > conversion "problem", one I prefer in cases where I know the value > is 2 digits, always, but might be 0n for 0<=n<=9, is $(( 1$x - 100 )) > > kre > > gazillion++ You say "I know the value is 2 digits" you forgot to say in base 10 but it was kinda implicit in the thread, then a more general one not limited to 2 decimal digit accepting leading 0 you could simply do $((10#$i)) That really sez you expect a base 10, some more readable I think... (may be)
Re: "printf %d ''" should diagnose the empty string
On Mon, Feb 10, 2025 at 1:15 PM Greg Wooledge wrote:
Ha now we introduce signed (not obvious from original post)
> The following line noise is the best *general* answer we've found
> so far:
>
> $((${num%%[!+-]*}10#${num#[-+]}))
If 'best *general*' refer to the shortest line noise does
$((${i/?([-+])/&10#}))
Qualify for better than best ? :-) if so you may add in your web bible
don't forget to mention the inventor :-)
Nice and easy
Re: "printf %d ''" should diagnose the empty string
On Mon, Feb 10, 2025 at 6:44 PM Greg Wooledge wrote: > > > No, I've shown that it *is* working in older versions of bash, and I'm > asking *why*. > > As far as I can tell by reading the CHANGES file, it shouldn't work. > > But it does. > Jeez, sorry about that, my 'fast reading' parser skipped the 'why', and I admit I don't know why.
Re: "printf %d ''" should diagnose the empty string
On Mon, Feb 10, 2025 at 3:43 PM Greg Wooledge wrote: > > Can you please explain *how* this is working in older bash versions? > I forgot to mention 'best on modern bash' :-), as well as you forgot to mention 'The following line noise is the best *general* answer we've found so far: with dyno bash' :-)
Re: "printf %d ''" should diagnose the empty string
On Mon, Feb 10, 2025 at 5:48 PM Chet Ramey wrote:
>
> There isn't a reward for brevity or obfuscation; say what you mean:
>
> isnum2()
> {
> case "$1" in
> [-+] | '') return 1;; # empty or bare `-' or `+'
> [-+]*[!0-9]*) return 1;; # non-digit with leading sign
> [-+]*) return 0;; # OK
> *[!0-9]*) return 1;; # non-digit
> *) return 0;; # OK
> esac
> }
>
> It obviously doesn't handle 0x constants, but could be changed to.
>
I think the point was to convert what would be seen as an octal literal by
$((...)) to something that is 10# with leading zeroes, basically date
output but not limited to.
i.e make $((i)) understand that i is signed leading 0 base 10
$ echo $BASH_VERSION
5.2.21(1)-release
$ shopt -s extglob
$ i=0012 ; echo "$((i)) $((${i/?([-+])/&10#}))"
10 12
$ i=+0012 ; echo "$((i)) $((${i/?([-+])/&10#}))"
10 12
$ i=-0012 ; echo "$((i)) $((${i/?([-+])/&10#}))"
-10 -12
As you see it does exactly what I mean, and cherry on the pie, you can even
fix 'i' for ever with
((i=${i/?([-+])/&10#}))
Now if the problem is to define is_num as a predicate, (not what I was
adressing) you noticed that you omitted 0x, but not only that but all the
base# that brings a far more complex function :-)
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
> ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>
Re: "printf %d ''" should diagnose the empty string
On Mon, Feb 10, 2025 at 7:06 PM Andreas Schwab wrote:
> On Feb 10 2025, Greg Wooledge wrote:
>
> > No, I've shown that it *is* working in older versions of bash, and I'm
> > asking *why*.
>
> Does it? If it did, it should have printed -23, not -19.
>
> --
> Andreas Schwab, SUSE Labs, sch...@suse.de
> GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7
> "And now for something completely different."
>
> Yes modern bash git it right not the dyno one :-)
$ echo $BASH_VERSION
5.2.21(1)-release
$ i=-0023 ; echo "$((i)) $((${i/?([-+])/&10#}))"
-19 -23
Re: "printf %d ''" should diagnose the empty string
On Mon, Feb 10, 2025 at 5:34 PM Zachary Santer wrote:
>
> int_regex='([+-]?)0*([[:digit:]]+)'
> if [[ ${var} =~ ${int_regex} ]]; then
> var="${BASH_REMATCH[1]}${BASH_REMATCH[2]}"
> else
> printf '%s\n' "Argument '${var}' is not a valid integer" >&2
> fi
>
The error message should be 'not a valid signed decimal with leading zeros
constant'
Re: return [n] documentation.
I still don't know the impact (implication) of extdebug, does it impact perf ?
Re: return [n] documentation.
It was not obvious from the reading that it would work when trap is inside the function but that is exactly what I need. Again, may be it is just me, but the reading with the back and forth reading and long distance jump in the docco to figure out how it works looks complicated, but that's good enough, I can proceed :-) Thanx tons @greg @chet
Re: "printf %d ''" should diagnose the empty string
On Mon, Feb 10, 2025 at 5:48 PM Chet Ramey wrote
>
> There isn't a reward for brevity or obfuscation; say what you mean:
>
> isnum2()
> {
> case "$1" in
> [-+] | '') return 1;; # empty or bare `-' or `+'
> [-+]*[!0-9]*) return 1;; # non-digit with leading sign
> [-+]*) return 0;; # OK
> *[!0-9]*) return 1;; # non-digit
> *) return 0;; # OK
> esac
> }
>
> It obviously doesn't handle 0x constants, but could be changed to.
>
Would you accept this one then, should be general enough, faster too, and
explicit, really mean what is accepted as num literal
$ echo $BASH_VERSION
5.2.21(1)-release
$ function isanum
{ (($1)) || (($1==0)) && return 0
return 1
} >/dev/null 2>&1
$ isanum 0 && echo yes || echo no
yes
$ isanum 1 && echo yes || echo no
yes
$ isanum 010 && echo yes || echo no
yes
$ isanum -010 && echo yes || echo no
yes
$ isanum 0x10 && echo yes || echo no
yes
$ isanum +0x10 && echo yes || echo no
yes
$ isanum 64#I_like_this_one && echo yes || echo no
yes
$ isanum '' && echo yes || echo no
no
$ isanum '+' && echo yes || echo no
no
$ zero=0
$ one=1
$ isanum zero && echo yes || echo no
yes
$ isanum one && echo yes || echo no
yes
$ isanum '' && echo yes || echo no
no
$ isanum '+' && echo yes || echo no
$ bad=4#j
$ isanum bad && echo yes || echo no
no
Look good to me ...
Re: "printf %d ''" should diagnose the empty string
I admit the function was mis-named, should be isanumexpr
As @chet noted this attempt was not too bad just need a little $1 check
before pursuing the idea of letting the bash scanner do the job for
complicated valid number (any base)
I come up with this one
function isanum
{ [[ "$1" =~ ^[+-]?[0-9][0-9a-zA-Z#@_]*$ ]] || return 1
(($1)) || (($1==0)) && return 0
return 1
} >/dev/null 2>&1
$ isanum 1+1 && echo y || echo n
n
$ isanum 'i[$( date >&2 )]' && echo y || echo n
n
$ isanum 64#yo && echo y || echo n
y
$ isanum -1 && echo y || echo n
y
$ isanum yo && echo y || echo n
n
$ isanum '' && echo y || echo n
n
$ isanum 0 && echo y || echo n
y
This time the 1st =~ do accept only number characters at large, the
complexity of deciding which [:letter:]@_ is ledgit for a given base is
differed to $(())
Getting closer to the perfect isanum() :-)
Note the I have a full one based only on regex =~ it is real ugly so that's
why I prefere de let $(()) do the scan, specially if one day it accept 0b
as base (unlikely but who knows. This code would still work.
Again modern bash.
Note too that I use function redirection because I got no idea how to nuke
$(( )) syntax error message
Re: "printf %d ''" should diagnose the empty string
On Mon, Feb 10, 2025 at 11:10 PM Greg Wooledge wrote: > > hobbit:~$ phi 029 > bash: ((: 029: value too great for base (error token is "029") > bash: ((: 029: value too great for base (error token is "029") > > So we come full circle. > Nope phi() is not mine and doesn't remove the error message, see my prev post about nuking $(()) syntax error 029 is not a number my latest post gives $ isanum 029 && echo y || echo n n
Re: "printf %d ''" should diagnose the empty string
On Mon, Feb 10, 2025 at 11:46 PM microsuxx wrote:
> a bit unrelated .. cheap numbers filter ..
>
> gimme_num( ) { declare -n b=BASH_REMATCH\[1] ; local o m=[0-9,.] s=$n i t ;
> t="(-?$m+)" ; unset -v o ; for n ; do unset -v i ; while [[ $n =~ $t ]] ;
> do n=${n/"$b"} i= o+=$b$s ; done ; [[ -v i ]] && o+=$s ; done ; [[ -v o ]]
> && printf %s "$o" ; }
>
> gimme_num ' ab3.44 cd---.99 ' cd cd 77.7.77
>
> 3.44
> -.99
>
> 77.7.77
>
>
Rejected
$ gimme_num '0x10'
0x10xx
$ gimme_num '64#yo'
64xx
$ gimme_num '0x10'
0#yo10#yo#yo
gimme_num should be renamed random_string()
Re: "printf %d ''" should diagnose the empty string
Can follow that fast, the n-1 version gives
gimme_num( ) { local y o m='[0-9,.]' s=$'\n' i t ; t="(-?$m+)" ; unset -v o
; for n ; do unset -v i ; while [[ $n =~ $t ]] ; do y=${BASH_REMATCH[1]}
n=${n/"$y"} i= o+=$y$s ; done ; [[ -v i ]] && o+=$s ; done ; [[ -v o ]] &&
printf %s "$o" ; }
$ gimme_num 0x10 60#yo 0x10
0
10
60
0
10
That can be streamlined as
$ gimme_num2()
{ printf '%d\n' ${@//[^0-9]/ }
}
$ gimme_num2 0x10 60#yo 0x10
0
10
60
0
10
The last version gives
$ gimme_num( ) { local y o m='[0-9,.]' s=$'\n' i t ; t="(-?$m+)" ; unset -v
o ; for n ; do i= ; if [[ $n =~ $t ]] ; then while [[ -v BASH_REMATCH[++i]
]] ; do o+=${BASH_REMATCH[i]}$s ; done ; fi ; [[ $i != 1 ]] && o+=$s ; done
; [[ -v o ]] && printf %s "$o" ; }
$ gimme_num 0x10 60#yo 0x10
0
60
0
Dunno what use case I could do with that
Re: "printf %d ''" should diagnose the empty string
On Tue, Feb 11, 2025 at 9:37 AM microsuxx wrote:
> how does it output strings .. ?
> i mean i dont get ur outputs
> 1 it doesnt include # and @ etc
> 2 no hex etc yet , only numbers and dots / commas
>
> gimme_num 0x10 60#yo 0x10
> 0
> 10
>
> 60
>
> 0
> 10
>
> but i dont get ur outputs such as 0x10xx or 64xx or the other ..
>
I cut/pasted as is your code
$ gimme_num( ) { declare -n b=BASH_REMATCH\[1] ; local o m=[0-9,.] s=$n i t
; t="(-?$m+)" ; unset -v o ; for n ; do unset -v i ; while [[ $n =~ $t ]] ;
do n=${n/"$b"} i= o+=$b$s ; done ; [[ -v i ]] && o+=$s ; done ; [[ -v o ]]
&& printf %s "$o" ; }
$
Then I run it with
$ echo $BASH_VERSION
5.2.21(1)-release
$ shopt
autocd off
assoc_expand_once off
cdable_vars off
cdspell off
checkhash off
checkjobs off
checkwinsize on
cmdhist on
compat31 off
compat32 off
compat40 off
compat41 off
compat42 off
compat43 off
compat44 off
complete_fullquote on
direxpand off
dirspell off
dotglob off
execfail off
expand_aliases on
extdebug off
extglob off
extquote on
failglob off
force_fignore on
globasciiranges on
globskipdots on
globstar on
gnu_errfmt off
histappend off
histreedit off
histverify off
hostcomplete on
huponexit off
inherit_errexit off
interactive_comments on
lastpipe off
lithist on
localvar_inherit off
localvar_unset off
login_shell off
mailwarn off
no_empty_cmd_completion off
nocaseglob off
nocasematch off
noexpand_translation off
nullglob off
patsub_replacement on
progcomp on
progcomp_alias off
promptvars on
restricted_shell off
shift_verbose off
sourcepath on
varredir_close off
xpg_echo off
And got this
$ gimme_num 0x10 60#yo 0x10
0x10xx60xx0x10xx$
Re: PIPESTATUS differs from $? for compound command
On Tue, Dec 10, 2024 at 10:42 AM Mike Jonkmans wrote:
> On Tue, Dec 10, 2024 at 09:28:31AM +0100, Ulrich Müller wrote:
> > > On Mon, 09 Dec 2024, Mike Jonkmans wrote:
> >
> > >> Why is `if false; then :; fi' not a pipeline? It is a command, and
> the
> > >> components of a pipeline are commands.
> >
> > > It is a pipeline indeed, but not the last
> > > (doc says: most-recently-executed).
> > > The last is the 'false' simple command/pipeline.
> >
> Also compare:
> $ if false; then echo foo; fi
> $ echo "ret = $?, status = ${PIPESTATUS[*]}"
> ret = 0, status = 1
> To:
> $ if ! :; then echo foo; fi
> $ echo "ret = $?, status = ${PIPESTATUS[*]}"
> ret = 0, status = 0
>
>
You may also add this one in your serie :-)
$ [ ! : ] && echo foo
$ echo "ret = $?, status = ${PIPESTATUS[*]}"
ret = 1, status = 1
Re: printf width format specifier doesn't work when assigned
On Sat, Jan 25, 2025 at 4:06 PM Andreas Kähäri
wrote:
>
> Sorry, I pressed send too quickly. I was going to suggest an alternative
> to using a command substitution and tr:
>
> printf -v foo '%*s' 80 ' '
> echo "${foo// /*}"
>
> --
> Andreas (Kusalananda) Kähäri
> Uppsala, Sweden
>
>
While we are showing printf tricks, we can notice we are cheating a bit
here, the foo variable is set to a serie of space ' ' and then each access
to this variable require a space substitute with your desired char-pattern
here '*', unless you re-assign the variable with the subsitution i.e
printf -v foo '%*s' 80 ' '
foo="${foo// /*}"
echo "${foo}"
In order to get foo set up with a desired number of '*' (or any 1 char
pattern you like) you may consider this one :-)
$ printf -v foo "%(*)T" {1..4} ; echo "'$foo'"
''
I admit it is a kind of abuse :-)
Note the upper bound is hard coded though (as 80 in your example).
To get a parametrisable upper bound and without fork/exec (eliminating
seq(1)) we can consider this one that is still fast and more general
$ c=10
$ foo='' ; while((${#foo}
Re: "printf %d ''" should diagnose the empty string
On Thu, Feb 13, 2025 at 6:18 AM Oğuz wrote: > > > Changes made to built-in printf have to be adopted by GNU printf, otherwise > the user will have two flavors of printf on the same system by the same > name. > -- > Oğuz > You sure about the adopted by GNU printf assertion ? $ /usr/bin/printf '%(*)T\n' /usr/bin/printf: %(: invalid conversion specification $ printf '%(*)T\n' * It is clear the buitin printf != /bin/printf
Re: "printf %d ''" should diagnose the empty string
On Thu, Feb 13, 2025 at 10:27 AM Oğuz wrote: > And yes, I am sure; if built-in printf and GNU printf differ in common use > cases that's a clear bug from the user's perspective. > There must be a bug somewhere then :-) $ printf 1 2 3 4 1 $ /bin/printf 1 2 3 4 1/bin/printf: warning: ignoring excess arguments, starting with ‘2’ $ printf '1\n' 2 1 $ /bin/printf '1\n' 2 1 /bin/printf: warning: ignoring excess arguments, starting with ‘2’ As long as the 2 printf's are not from the same source, divergence is always possible...
Re: "printf %d ''" should diagnose the empty string
Hi, My personal vote is for no printf warning on bad format of %fmt args, but other may argue against that. This is no big deal because I can always do in my scripts alias printf='printf 2>/dev/null' That being said if warnings are printed I suppose they must be accurate ? I don't really get this one $ echo $BASH_VERSION 5.3.0(4)-beta $ echo $((4+2#1)) 5 $ printf '%d\n' 2#1 bash: printf: 2#1: invalid number 2 Why 2#1 is invalid ?
Re: "printf %d ''" should diagnose the empty string
On Sat, Feb 8, 2025 at 10:31 AM Robert Elz wrote: > > > It isn't a valid form for a C constant.bash allows that kind of thing > in its arithmetic contexts, but this is not one. > > Jeez moving target, thanks to bring C in the loop I got it now :-) I like this orthogonality :-) $ printf '%d\n' 2#1 bash: printf: 2#1: invalid number 2 $ printf '%d\n' $((2#1)) 1 $ printf '%d\n' 0b1 1 $ printf '%d\n' $((0b1)) bash: 0b1: value too great for base (error token is "0b1") $ lol
Re: Bug: please document extended and alternate for loop syntax
On Mon, Mar 10, 2025 at 5:39 PM Zachary Santer wrote:
>
> Another alternative would be for bash to print a warning whenever it
> encounters this syntax.
>
Don't do that, the one who don't care about shell portability. i.e the
script started with #!/bin/bash may well be using this construct "for ((
i=0; i<3; ++i )) { echo $i; }" since ages and don't want to see warnings
all of the sudden. If you go with new warnings path, then it got to be with
a new shopt kind of optin that old script don't use.
Re: Support ksh93 x=${cmd;}
Besides, HP-UX sh is ksh88 :-) no risk anyone port anything from there.
Re: [sr #111166] ngettext syntax
On Fri, Feb 21, 2025 at 3:08 PM Chet Ramey wrote: > On 2/20/25 11:44 PM, Phi Debian wrote: > > > > > > On Thu, Feb 20, 2025 at 11:41 PM Chet Ramey > <mailto:chet.ra...@case.edu>> wrote: > > > > A response to the question about printf supporting %n$ conversion > > specifications I posted to savannah. > > > > > > Thanx @chet, I didn't knew this thread. The thread say ksh93 is buggy, > > that's not what I observe with latest patches, but at time of the thread > it > > was probably buggy. > > Yesterday? > > > Given the following, which POSIX says is unspecified: > > printf '%s %3$s %s\n' A B C D > > ksh93-u+m prints "A D", which is just wrong. No matter how you mix numbered > and unnumbered specifications, or whether you implement numbered > specifications at all, you can't just drop it. > Well mine gives $ echo $KSH_VERSION Version AJM 93u+m/1.1.0-alpha+b5087279 2025-01-15 $ printf '%s %3$s %s\n' A B C D A C B D Which is just right I guess unless I am misled somewhere. Yet mixing numbered and unumbered specification is questionable as you said, but so is the re-use of format string when reaching excess args... we are in the invent area here :-) The more C like could be (fully numbered) latest ksh93 gives $ printf '%3$s %2$s %1$s\n' A B C D C B A D Still the D is the args fmt re-use, but output is similar to C (modulo D) and bash 5.3.0(1)-beta gives $ printf '%3$s %2$s %1$s\n' A B C D bash: printf: `$': invalid format character Which indeed is not an invalid format or should I say is a plausible format specification. > > Yet it doesn't prevent an attempt to make it better, low priori I rekon, > > but if someone want to cut a patch there is no reason to veto it. Old > > script will still work, and new script could benefit it. > > It will happen, but not in bash-5.3. > I like that :-) > > > > I like also to have %d (and all integer format) accept a valid integer > > expression as argument (printf '%d' 1+1 ; i=1 ; printf '%d' i+1 etc...) > > again a very low prio, but it is clean, I know that a simple $((...)) > can > > do this, but it is 5 char of line cluttering for each % integer format, > for > > not that much clarity. > > It decreases clarity. If you have an argument '1+1', your proposal > makes it depend on the conversion specifier. Right now, there's no > ambiguity: 1+1 is a string, and $(( 1+1 )) is 2, regardless of whether > or not the conversion specifier accepts an integer argument. > For me clarity means that %d expect an integer expression value (ala C even though, there will be voice saying BASH is not C :-) so %d arg fetch must fetch not a string but a string that is an integer expression really removing $((...)) At the moment %d fetch a string, applying an internal sh_eval(string) should be doable... but if that doesn't happen, no big deal... :-) So the ambiguity you are talking about is the problamatic of scanning the format string, and recognise the d, i, o, u, x, or X conversion char and accept the input args is a integer expresion 'string' :-) I would say it is an internal ambiguity, but at the user level it is not insane to think the string could be an integer expresion, at the moment is must be string that is a valid integer... why limiting the check to integer vs integer expression ? Cheers,
Re: [sr #111166] ngettext syntax
On Thu, Feb 20, 2025 at 11:41 PM Chet Ramey wrote: > A response to the question about printf supporting %n$ conversion > specifications I posted to savannah. > Thanx @chet, I didn't knew this thread. The thread say ksh93 is buggy, that's not what I observe with latest patches, but at time of the thread it was probably buggy. Yet it doesn't prevent an attempt to make it better, low priori I rekon, but if someone want to cut a patch there is no reason to veto it. Old script will still work, and new script could benefit it. I like also to have %d (and all integer format) accept a valid integer expression as argument (printf '%d' 1+1 ; i=1 ; printf '%d' i+1 etc...) again a very low prio, but it is clean, I know that a simple $((...)) can do this, but it is 5 char of line cluttering for each % integer format, for not that much clarity. I don't really push for that to happen, it is just a wish list... Cheers
Re: [sr #111166] ngettext syntax
On Sat, Feb 22, 2025 at 1:54 AM Robert Elz wrote: > > > That is, in the example > > printf '%s %3$s %s\n' A B C D > > The updated ksh93 seems to ignore the numbered conversions when counting > the args for the unnumbered ones (so "A C B" (for the first line anyway, > I won't go on about repeating the format string again). > > The previous version made an unnumbered arg conversion use the next arg > after the last one used (which would give "A C D" using a more reasonable > interpretation of %3$). > Yes, since numbered unumbered usage was not define (C libc doesn't allow the mix, doesn't define fmt reuse either) I decided to find a possible way of doing it. Previous attempt of placing unumbered after the last numbered one was leading with serious problem when going into number/unnumbered num.prec. So previous attempt was trying to do for '%3s %s' the %s is next to the third arg then would be equivalent to '%3s %4s', this lead to all sort of bug and crash and then was never used. The new semantic is simple - numbered are indexed args access (easy to understand) - unumbered are counting only the unumbered from the fmt string This way when you don't mix then the both work as expected. When you mix them, you need a little head scratch, it is useless, but predictable and unbreakable. Since nothing worked before i.e all shell either crashed or refused to run so complicated construct, then doing this didn't break anything, just makes numbered working, not breaking unumbered. Now the dare devil can still mix'n'match, it may not produce what they would like, but at least it don't crash, and produce result as announced for this semantic. Again other semantic can be done for mix'n'match... I would vote for refusing mix'n'match altogeher, yet this is not what I've done for ksh93 because though bugged it was still defined and one could ask for fix > > > printf '%s %s %s %3$s %s\n' A B C D > $ printf '%s %s %s %3$s %s\n' A B C D A B C C D Easy to predict 4 %s take in that order and one number that access the indexed one, no surprise > > > printf '%s %$4s %s %s %s' A B C D > $ printf '%s %$4s %s %s %s' A B C D A /d1/ksh-latest/arch/linux.i386-64/bin/ksh: printf: $: unknown format specifier I guess you meant $ printf '%s %4$s %s %s %s' A B C D A D B C D Can't blame the syntax error :-) I am dislexic too again no surprise here the %s are counted sequentially and the indexed is find directly. The more interesting is $ printf '%s %6$s %s %s %s' A B C D A B C D The 6th does't exist so get '' Again simple rule of thumb if you don't mix'n'match they all works as expected, if you insist on mix'n'match you are entering someting that has never worked before, and then you got to test do decide how to mix'n'match > > | It's worse -- even if a format string using only numbered conversions > | doesn't consume all the arguments, as long as it consumes the last one, > | the format string doesn't get reused. It all depends on whatever > peculiar > | definition of "satisfy" you use. > > Yes, which is where the trick of including a conversion "%999$.0s" in the > format string is supposed to stop format string reuse -- as long as there > are > no more than 999 args given of course. This issue has nothing to do with > the > mixed numbered/unnumbered problem, which rightly should just be > unspecified, > and implementations do whatever they like, reasonable or not (as in the eye > of the coder), but that format string reuse is simply wrong in the numbered > conversion case, no-one really needs that. > > Again the rule I used for ksh93 is that the next arg for fmt reuse is max( number_of_unumbered_occurences, highest_numberd_index ) A simple rule easy to understand, and making non mixed mode to work as expected, again leaving mixed mode in the complexity of counting things. On top of the fact those rules are simple, they also ease the fmt scan and arg fetching. Another way of saing all this is that %s (unnumbered) are in fact internally numbered with 'seq' so %s %s in internally %(seq++)s %(seq++)s and then numbered are in the midle of this %(seq++)s %4s %(seq++s) Next fmt skip max(seq,4) I forgot to mention your trick to nuke the fmt reuse still works $ printf '%s %s %s %999$s' A B C D E F G A B C :-) Cheers
Re: [sr #111166] ngettext syntax
On Sun, Feb 23, 2025 at 12:34 AM Robert Elz wrote: > Date:Sat, 22 Feb 2025 09:48:29 +0100 > From:Andreas Schwab > Message-ID: <87r03q5pr6@linux-m68k.org> > > | On Feb 22 2025, Phi Debian wrote: > | > I forgot to mention your trick to nuke the fmt reuse still works > | > $ printf '%s %s %s %999$s' A B C D E F G > | > A B C > > Using %999$.0s rather than just %999$s was deliberate, it ensures that > if there happen to be 999 args, nothing is printed (the aim isn't to > do anything with the arg, just consider it used). Also, sticking a space > before it is not a good idea, your output would be "A B C " - and while > that might be what you wanted, it usually would not be. > Agry for the trailing useless space, I got bitten by it in the QA suite test I added for ksh93. In my implementation %999$s is enough to kill the fmt reuse, as I noted you should stick in there a majorant that you know will not happen 4096 is doable too. > > I should also mention that this isn't guaranteed to work according to > the POSIX spec - it says: > > ... > > Note, it says "consumed" not "referenced" there, an argument that does > not exist (as we're presuming, or perhaps hoping, is the case with the > 999th arg being selected) cannot really be said to be consumed, as it does > not exist. > > Further, this spec also violates Chet's rule: > > chet.ra...@case.edu said: > | There is no user who would think that using a numbered conversion > specifier > | is not an absolute position in the original argument list. > > ... I didn't paid attention to POSIX for this thing, because mainly, the idea was 'do not mix number/unnumbered, and whatever POSIX said, the few shell I checked before going ahead where simply buggy or refused such mix. In the ksh case, it was the most advanced one yet still bugged, so I decided to make something workable, mainly because I really needed numbered to work. Since there is no implementation approaching POSIX fuzzy description and fully operational, then my implementation isn't breaking anything in the real life. POSIX may reword the unumbered/numbered mix in my direction one day, if they go toward another one we will adapt, for now there is nothing that work regarding shells, and ksh93 was core dumping on bad combo, so now it doesn't core dump and now if propose something that has a logic that unless proving wrong, is still correct. As for @chet thing regarding fmt-reuse and numbered access, I choose to stop counting args at max(unumbered_occurences, highest_index) and re-use from there+1, and it sounds satisfactory. As you said, and I said many time, yes this thread is useless, but it has the virtue of brain dumping what's known at this time, for a future implementation that post bash 5.3 may consider. As far as I am concerned I really care is non mixed, and I am pretty sure the bash designer will implement a valid numbered access one day. > > And finally, to Andreas' comment which was what inspired me to continue > this relatively meaningless thread: > > | As long as NL_ARGMAX >= 999. > > Yes as you said there is no limit on the %num$, theorically, yet imposing a limit like this one would allow to know for sure how to nuke the fmt re-use, removing both the strtod() overflow, the malloc() failure on provisioning args, etc... a relativly modest limit of 4096 (NL_ARGMAX) could be a good idea. I think we covered most of the problematic for mix'n'match, so bash >5.2 can start their implementation with ideas. I would conclude with - When doing numbered don't focus only on %ndx$c only (ndx=index, c=conversion char) but remember that numbered width.prec do exist too. - if allowing mix and match numbered/unumbered then stays away from unumbered following a numbered mean the unumbered is last the numbered+1 this is tempting but wrong, i.e '%3$s %s' one would be tempted to say %s is equiv to %4$s, but the width.prec predate that, because then one must define what happen when index back, i.e '%4$s %s %2$s %s' what is the last %s? some will say 3, other will say 6, in all case it will be verbose doc to say something complicated, so my recommendation is to say non unnumbered start at 1 and increment at each unumbered, easy to remember and matching the case of fully unumbered and gaps are skipped over and ignored. With all this, bash designer, gentlemen start your engine!!! :-)
Re: [sr #111166] ngettext syntax
On Fri, Feb 21, 2025 at 8:52 PM Chet Ramey wrote: > On 2/21/25 1:37 PM, Phi Debian wrote: > > > > I got that behavior with Version AJM 93u+m/1.1.0-alpha 2022-07-31. > > I just updated my version using macports and I get the same behavior as > you with Version AJM 93u+m/1.0.10 2024-08-01. So it looks like Martijn > was working on it. > Actually this is my patch to ksh93 :-) commit 87480ec207a56d7b17de722789437c8cc7827495 Author: Phi Date: Wed May 17 03:51:20 2023 +0200 > > > > No, reusing the format string is required by POSIX. The only question is > the conditions under which you do that. > > > The more C like could be (fully numbered) latest ksh93 gives > > $ printf '%3$s %2$s %1$s\n' A B C D > > C B A > >D > > Still the D is the args fmt re-use, but output is similar to C (modulo D) > > You're not going to be able to give up format re-use. Everyone seems to > agree on the behavior when all of the format conversions are numbered. > I miss expressed myself I suppose, I don't want to give up the fmt reuse I am the first to abuse it with a great interest, actually it is a powerfull iterator. I wanted to express that while trying to implement the %n$ you got to depart C model because it C doesn't allow mix'n'match numbered and unumbered specification, but at the same time the fmt reuse is a departure from C, and bash is not C I hear many times, and I agry to, so meaning, if we can invent fmt re-use we can invent %$n implementation, and in the same vein we can invent mix'n'match numbered and unumbered departing C, that's what I did for ksh93 and was accepted. The idea is to invent something coherent. In the ksh93 I decided that numbered are simply indexed access into the args, while unumbered just walk on args possibly triping in an already accessed to a numbered one so $ printf '%3$s %s %2$s %s %1$s\n' A B C D E F G H C A B B A | | | | | | | | | +-> %1$s indexed arg | | | +---> %s 2nd non indexed | | +-> %2$s indexed arg | +---> %s 1st arg (non indexed) +-> %3$s indexed arg F D E E D G H H G This way pure unumbered (no mixed indexed) works as before (as we used to) Pure numbered (no unumbered) works as it does in C. Mixed depart from C (as do fmt re-use) and the rule of thumb is easy to remember, count unnumbered as they appears, and numbered as indexed, there is one last catch, for fmt re-use, where to start the next iteration, well I decided the next one arg in line for next fmt re-use is the max of unumbered one and numbered one max index, i.e if there are 4 unumbered and max index is 3 then the next fmt arg start is 5, and the other way around if only 2 unumbered was used and max indexed is 4 then again next fmt start is at 5. This is one approach, other approach could be made, like refusing mix'n'match, accepting only pure access (numbered or unumbered exclusivly) Note that I used the same rule for numbered unumbered mix'n'match for the implementation of %*.*s and %n$*w$.*p$s that is the use of num.prec | | prec | width pos Note the pos is not too intuitive but that's libc implementation, it comes before num.prec. $ printf '%3$*2$.*1$s\n' 2 4 6 6 $ printf '%3$*2$.*1$s\n' 2 4 67 67 $ printf '%3$*2$.*1$s\n' 2 4 678 67 $ printf '%3$*2$.*1$s\n' 2 4 678 3 67 $ printf '%3$*2$.*1$s\n' 2 4 678 3 5 67 $ printf '%3$*2$.*1$s\n' 2 4 678 3 5 7 67 7 $ printf '%3$*2$.*1$s\n' 2 4 678 3 5 78 67 78 $ printf '%3$*2$.*1$s\n' 2 4 678 3 5 789 67 789 $ printf '%3$*2$.*1$s\n' 2 4 678 3 5 7890 67 789 $ And the cherry on the pie mix'n'match with width.prec $ printf '%3$*.*2$s\n' 4 2 6 6 $ printf '%3$*.*2$s\n' 4 2 67 67 $ printf '%3$*.*2$s\n' 4 2 678 67 $ This is completly useless nobody will use that, yet it is coherent, the rules are predictable, it doesn't crash :-) With ksh93 we start with a plain buggy implementation, so I suppose nobody ever used it on purpose to get the buggy result, then allowing an implementation that is correct for a given rule of thumb, don't produce false positive error message. Yet it allows simple usage like '%2$*1$.*1$s\n' where with==prec avoiding the double arg width and prec so $ printf '%2$*1$.*1$s\n' 2 45 45 $ printf '%2$*1$.*1$s\n' 2 456 45 $ printf '%2$*1$.*1$s\n' 2 456 3 45 $ printf '%2$*1$.*1$s\n' 2 456 3 4 45 4 $ printf '%2$*1$.*1$s\n' 2 456 3 45 45 45 $ printf '%2$*1$.*1$s\n' 2 456 3 456 45 456 $ printf '%2$*1$.*1$s\n' 2 456 3 4567 45 456 This later usage could be more common. >
Re: [sr #111166] ngettext syntax
On Sat, Feb 22, 2025 at 9:49 AM Andreas Schwab wrote: > On Feb 22 2025, Phi Debian wrote: > > > I forgot to mention your trick to nuke the fmt reuse still works > > > > $ printf '%s %s %s %999$s' A B C D E F G > > A B C > > As long as NL_ARGMAX >= 999. > Sounds like I was over optimistic $ echo $KSH_VERSION Version AJM 93u+m/1.1.0-alpha+b5087279 2025-01-15 $ printf '%s %s %999$s\n' 1 2 3 4 5 6 7 8 1 2 $ unset A B C D E F G H $ printf '%s %s %999$s\n' A B C D E F G H ./ksh: printf: C: parameter not set ./ksh: printf: D: parameter not set ./ksh: printf: E: parameter not set ./ksh: printf: F: parameter not set ./ksh: printf: G: parameter not set ./ksh: printf: H: parameter not set A B $ I admit I didn't entered a way big arg index (exceeding the actual number of args) in the QA suite for ksh93, will do soon, along with a fix to avoid such wrong error output. $ getconf NL_ARGMAX 4096 The idea to nuke fmt re-use is to use a index arg number bigger then the expected number of args, and if unknown got for the max :-) Thanx for raising this... But here we are talking bash 'possible' implementation of printf fmt indexed args access, and I wanted to share what I did for ksh93, kind of heads up of pitfall to watch out. So the implementation make '%s %s %s' as '%(seq++)s %(seq++)s %(seq++)' with (seq) a theorical syntax don't exist in reality. The problem that led me to go with seq++ access of unumbered args, i.e internally treat them as numbered in seq++ is that mix'n'match of numbered/unumbered not only occurs for each % but inside them i.e '%s %2% %s' is mix'n'match for % but indexed access also occurs in width.prec as Unnumbered $ printf '%*s\n' 2 4 4 numbered (works as documented in sprintf(3) ) $ printf '%2$*1$s\n' 2 4 4 Mix'n'match UGLY, don't use, but if you insist printf '%2$*s\n' 2 4 6 8 4 8 Here the arg value is indexed at 2$ but the width is not (unumbered), so here with 'my' implementation treat this like '%2$*(seq++)$s\n' really fetching the correct width, yet an explicit *1$ is much preferable. With an implementation (as was ksh93 when it was plain bugged) the unumbered would be next to last index and would be similar to %2$*3$s and would not work. By applying the rule (seq++) for any unumbered not only it works (in the sense produce something predictable), and it goes along with ftm reuse nicely. So this was my best guess for something that was ugly from the start, i.e mixin numbered/unumbered. The other approach consisting of refusing mix'n'match is more complicated internally, because it force a double pass on fmt string, while the seq++ just walk produce thing (not necessarilly to yourt liking) and is predictable. This this the reflexion that led me to this implementation...
Re: [sr #111166] ngettext syntax
May be some orthogonality regarding %d expecting a string vs num-expr
$ a=1234 i=1
$ echo ${a:i+1}
34
$ echo ${a:$((i+1))}
34
$
Why do we have this 'offset' evaluated, you could use the same argument you
used for printf that is $((...)) is good enough, add clarity, remove
ambiguity and the docco for 'offset' could have been "a valid number"
Why printf is different?
How many people bother to write ${a:$((i+1))} when ${a:i+1} is good enough.
I am pretty sure that if printf was orthogonal with ${var:offset:length},
nobody would ever use $((...)) for %d conversion specifier.
I think the 'clarity/ambiguity' argument is not completely real, may be it
is more "to be aligned with /bin/printf" that is not able to evaluate
expressions, but if this is the reason, then bash printf has no reason to
support say %n that /bin/printf could not honor.
$ /bin/printf '123%n\n' i ; echo "i=$i"
123/bin/printf: %n: invalid conversion specification
i=3
$ printf '123%n\n' i ; echo "i=$i"
123
i=3
$
So if we admit bash printf is not /bin/printf then %d expecting a 'a string
that must be valid num-expr' instead of 'a string that must be a valid
number' is a possible deviation. Again may be with a flag... if
compatibility we xyz is a concern
Re: [sr #111166] ngettext syntax
On Mon, Feb 24, 2025 at 8:22 AM Martin D Kealey wrote: > > > On Mon, 24 Feb 2025 at 13:48, Phi Debian wrote: > >> I also forgot to mention that C99 introduced this >> >>There may be no gaps >>in the numbers of arguments specified using '$'; for example, if >> argu‐ >>ments 1 and 3 are specified, argument 2 must also be specified >> some‐ >>where in the format string. > > > That rule is necessary in C because of the many ways that varargs can: > different kinds of values can potentially occupy differing numbers of > consecutive bytes in the space used to store as parameters. (I would say > "in the call stack", except that's not how ISO-9899 defines things, even > though it is how most implementations work in practice.) > > In the shell, we already have an argv with consistently sized elements, so > randomly indexing into it does not pose any particular challenge. So I > wouldn't expect that rule to be necessary, in the sense that it doesn't > offer any flexibility to the implementation. > That's true, but yet it define a kind of idiom regarding printf that I agry a shell don't have to follow (and it doesn't in my ksh93 patch) but still could be considered. > -Martin > > PS: Hmmm, well I guess repetition could be implemented by simply shifting > off as many args as there are conversions, and that would be a simple way > to get consistent repetition whether conversions are numbered or not, but > it feels perverse and I don't like it - though maybe I'll change my mind > when I've thought it over. > Not sure I get it, but if you mean $ printf '%1$s %1$s %1$s\n" q w e r t y Should produce q q q r r r I don't agry, or I would say this is not the current ksh93 behavior $ printf '%1$s %1$s %1$s\n' q w e r t y q q q w w w e e e r r r t t t y y y Still ksh93 can produce your output (perverted indeed) $ printf '%1$s %1$s %1$s%3$Z\n' q w e r t y q q q r r r So while ksh can produce both output (flexible) your algo would prevent the former output, so you loose flexibility > PPS: I wouldn't actually shift the list of course; I'd just track a > striding index offset instead. >
Re: [sr #111166] ngettext syntax
On Mon, Feb 24, 2025 at 4:14 PM Chet Ramey wrote:
>
> >
> > The beast of burden is already done in bash, it does scan the fmt string
> > recognise integer conversion specifier fetch the arg 'string' and apply
> an
> > integer validity test on it, this is on this last operation that the
> > validity check could be replaced by a airth_eval(string)
>
> Sure, I get how it could be done. I just don't buy the argument for doing
> it.
>
>
>
> Oh, it certainly could. I just don't think it adds anything. I know ksh93
> does it, but ksh93 threw arithmetic evaluation in a bunch of different
> places before $((...)) came along, and I think arithmetic expansion renders
> them superfluous.
>
Well I can provide a reason that pushed me toward arith exp for numeric
conversion char in printf, but again I have no problem with bash not doing
it,
this is mainly to reduce my interactive work keyboard typing experience,
i.e avoid to constantly type $((...) on each printf I type hundred of time
a day.
So bash can be used as a scripting language, and 'may be' verbose keyboard
typing is acceptable with a lot a $((...)) everywhere in a script, but when
doing interactive work saving this keyboard typing of $((...)) is a big
deal, specially that my typing skills are below the average, and dislexie
add to the complexity I keep doing $(...) in lieux of $((...)) with indeed
and great usage of the history to fix the typo, thanx for that :-)
In some aspect, the fmt reuse make a kind of iterator to produce thing
while printf was not an iterator in spirit, we have of other iterator like
{x..y} etc, the ftm reuse allow printf to become an iterator to
prepend/append things to a word in wordlist, I abuse it a lot, on the same
vein, I think the eval of num conversion is a kind of shortcut, and I love
shortcut when I am typing :-)
And indeed, printf could have a flag allowing num expression for num
conversion, either printf option or shopt option, after all we do have
option for extglob, meaning we deal with various form of patterns
No need to say my .rc file have extglob on all the time and would have
printf_numexpr on all the time too :-)
Re: [sr #111166] ngettext syntax
On Sun, Feb 23, 2025 at 8:03 PM Chet Ramey wrote: > On 2/22/25 12:38 AM, Phi Debian wrote: > > > The new semantic is simple > > - numbered are indexed args access (easy to understand) > > - unumbered are counting only the unumbered from the fmt string > > > > This way when you don't mix then the both work as expected. > > This isn't unreasonable. coreutils git chose the same interpretation. Too > bad FreeBSD went the other way. > Hum, not using FreeBSD so not spotted that one, may be they will support multi semantic with printf flags (again the docco will be epic) > > > Now the dare devil can still > > mix'n'match, it may not produce what they would like, but at least it > don't > > crash, and produce result as announced for this semantic. > > No one should mix numbered and unnumbered conversions unless they control > everything -- the shell, the shell version, the format string, and all the > arguments. > I know, so the proposal to refuse working on mix'n'match, but that one is problematic and require a full fmt scan before start working really, yet doable. For ksh93 the mix'n'match, though bugged, existed since long time, so the try to make something running without crashing, and still be usefull for full numbered, even that was broken, so at least full numbered works, and my guess is there is not that much mix'n'match out there that I may have broken if they were relaying on the erratic output. > > > Again the rule I used for ksh93 is that the next arg for fmt reuse is > > max( number_of_unumbered_occurences, highest_numberd_index ) > > So you make numbered conversion specifiers relative to some max base from > the previous scan through the format string, initializing that base to 0. > I had the same idea. > Yes, numbered are 1 based (0 is the fmt string :-) and don't move) and the next fmt roll start past the highest numbered index so for '%2*4$.*1$s' next roll arg1 is the 5th, for unumbered we count '%s %s %s' next roll start at 4. And the ugly mix '%s %3$s %s' next roll start at 4 (sigma(%s)=2 max(indexes)=3 ==> next is 4) '%s %s %3$s %s %s' next roll start at 5 (sigma(%s)=4 max(indexes)=3 ==>next is 5) And we have gaps in numbered. This mix is useless but easy to explain what result you will get, and full numbered and unumbered works. With my simplistic rule, documentation is reduced, no complicated case to explain, count your unumbered, and use your index. Simple predictable Now the docco may say mix'n'match is forbiden (with error trowing) or Mix'n'Match is the counting method not necessarilly producing something usefull but avoid lengthy explanation and avoiding crash. I also forgot to mention that C99 introduced this There may be no gaps in the numbers of arguments specified using '$'; for example, if argu‐ ments 1 and 3 are specified, argument 2 must also be specified some‐ where in the format string. Another rule that could be imported, for ksh93 I didn't implemented it, so I accept what so called gap, so the dance with consumed vs referenced, all thoses complications, again my choice for unumbered counting and numbered with gaps, and the rule to define how we reuse the fmt, make the docco simple, with the provision written in BOLD font, don't mix'n'match it may not produce what you expect (yet it will not crash) that being said, a full non mixed, numbered (with or without gap TBD) or unumbered works a specified.
Re: getcwd() returns with stale errno value upon error
In the given reproducer, I am not sure btrfs is relevant here on my good'ol ext4 I got. $ mount | grep '/d2 ' | sed 's/(.*//' /dev/sda1 on /d2 type ext4 $ cd /d2 ; mkdir -p a/b/c ; cd a/b/c ; rmdir /d2/a/b/c $ bash -c 'pwd' shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory pwd: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory $ ksh -c 'pwd' ksh: pwd: cannot determine present working directory [No such file or directory] $ zsh -c 'pwd' . That's true that the bash shell-init: could be omitted and defered until one ask for pwd explicitly. I don't see any *ino* jazz here, but the classic, my current dir has been removed under my feet. I may have mis-read the *ino* part though.
