Re: bashbug chooses wrong $EDITOR executable

[Daniel]

On Jun 16, 9:38 pm, Chet Ramey  wrote:
> Bob Proulx wrote:
> > It seems okay to leave PATH alone to me.  Why set it at all?

I agree.  If as a user I care to set my environment in some specific
way, I expect other programs to honour my settings.
If all my scripts were resetting PATH and hardcoding #!/bin/sh, it
would be much harder to install and use newer versions of programs
(bash included).

> > I don't see the security issue that you are concerned about.  Could
> > you educate me?

> I suppose it's not a large security hole if $EDITOR is used, only
> when bashbug chooses $DEFEDITOR.

Would it be better to set DEFEDITOR to the real path which was tested
just before?
As in:

elif [ -x /usr/contrib/bin/jove ]; then
DEFEDITOR=/usr/contrib/bin/jove # instead of just jove

It would seem more robust to me, since /usr/contrib/bin is not
necessarily in PATH.

> Frankly, though, it's a good idea to set PATH to have the standard
> binary directories before any others when writing a shell script,
> especially one that can be run by root.  That's just good practice.

Would a fix prepending system paths if user = root be a better
approach?
Or even reset PATH completely?

if [ $(/usr/bin/id -u) -eq 0 -o $(/usr/bin/id -r) -eq 0 ]; then
  PATH=$(/usr/bin/getconf PATH)
fi

--
Daniel

Re: Behaviour of test -v with assoc array and quote character in key

[Daniel Gröber]

Hi Greg and Chet,

On Mon, Feb 15, 2021 at 08:48:15AM -0500, Greg Wooledge wrote:
> Do it this way instead:
> 
> unicorn:~$ mytest() { array[$1]=123; test -v 'array[$1]'; echo "$?"; }

Ah! I didn't know test would exand that '$1' in there, cool.

> Yours expands $1 first, then passes array["] as an argument to test,
> which then performs a *second* round of expansion/parsing.

Right that's sort of what I figured I was seeing and I wasn't sure if this
is intentional.

> You might also want to look at the assoc_expand_once option, although
> single-quoting will work in a wider range of bash versions.

Thanks for the tip!

On Mon, Feb 15, 2021 at 09:11:48AM -0500, Chet Ramey wrote:
> `test' is always going to be problematic here because, as a shell builtin,
> its arguments undergo a round of word expansions before it's invoked. It's
> difficult to reliably detect the closing bracket in an array subscript as a
> result, even if the array subscript expansion didn't perform any expansions
> on its own. (Consider what would happen if $1 were `]').

You're absolutely right, I didn't consider the ']' case at all! That would
obviously break. So indeed this just user-error then and not really a bug.

> There are workarounds you can use to avoid the problem, involving adding a
> set of quotes to suppress the expansion the array subscript evaluation
> performs.

The `test -v 'array[$var]'` thing Greg suggested above seems pretty good to
me. Since the array subscript expands variables internally (which I didn't
know) that seems to be the way to go.

The workaround I came up with left to my own devices was to use eval
`printf %q`, I assume that's the sort of quoting you were referring to:

# Usage: array_key_exists ARRAY KEY
#
# The ARRAY argument must be a valid bash identifier, but this is not
# checked. KEY may be any string.
#
array_key_exists () {
eval "[ -n \"\${$1[$(printf '%q' "$2")]:-}\" ]"
}

I was also wondering how array subscript interacts with `[[`, but it seems that

[[ -v array[']'] ]]

has the same problem as `test`. Looking at the manual it says that while
word splitting and filename expansion isn't performed quote removal still
is so that's no help in this regard.

I wonder if this particular point, how to check an arbitrary array key
exists or not, is documented anywhere yet? I'd be happy to send a
patch. Can someone point me to the sections in the man{ual,page} that could
use commentary for this?

Thanks,
Daniel

Re: bug report

[Daniel Mills]

On Fri, Apr 23, 2021 at 10:43 AM john  wrote:

> From: john
> To: bug-bash@gnu.org
> Subject: ls dumps bash
>
> Configuration Information [Automatically generated, do not change]:
> Machine: x86_64
> OS: linux-gnu
> Compiler: gcc
> Compilation CFLAGS: -march=x86-64 -mtune=generic -O2 -pipe -fno-plt
> -DDEFAULT_PATH_VALUE='/usr/local/sbin:/usr/local/bin:/usr/bin'
> -DSTANDARD_UTILS_PATH='/usr/bin' -DSYS_BASHRC='/etc/b
> ash.bashrc' -DSYS_BASH_LOGOUT='/etc/bash.bash_logout'
> -DNON_INTERACTIVE_LOGIN_SHELLS
> uname output: Linux john-arch 5.11.16-arch1-1 #1 SMP PREEMPT Wed, 21 Apr
> 2021 17:22:13 + x86_64 GNU/Linux
> Machine Type: x86_64-pc-linux-gnu
>
> Bash Version: 5.1
> Patch Level: 4
> Release Status: release
>
> Description:
>  After the two commands as specified, the bash session ends
> unexpectedly
>
> Repeat-By:
>  set -e extglob
>  ls ?(0)9,v
>
>
>
You want shopt -s extglob.

set -e extglob turns on errexit, and sets "$1" to "extglob". ls then exits
>0 and the shell terminates.

Long variable value get corrupted sometimes

[Daniel Qian]

Hi all,

I encountered a problem that long variable value get corrupted sometimes.

OS: Alpine linux 3.15.0 (docker container)
Bash version: GNU bash, version 5.1.8(1)-release (x86_64-alpine-linux-musl)

Reproduction steps:

A UTF-8 encoded file containing a lot of Chinese characters, file size ~35K.

A test script read content from `foo.txt` and assign the content to a variable,
and then check md5sum for that variable.

```bash
#!/bin/bash

FOO=$(cat /tmp/foo.txt)
want_q_md5=$(cat /tmp/foo.txt.md5 | cut -d ' ' -f 1)
got_md5=$(echo "$FOO" | md5sum -b | cut -d ' ' -f 1)
if [[ "$got_md5" != "$want_q_md5" ]]; then
  echo "failed"
  echo "$FOO" > /tmp/foo-corrupt.txt
  fail=true
else
  echo "succeed"
fi
```

**Sometimes**, the md5sum check failed.

Output variable value to `foo-corrupt.txt` when check fail, compare it
with `foo.txt`,
found that a random byte are inserted into multiple positions.

I created a github repo for this issue, all the scripts are there:
https://github.com/chanjarster/bash-5_1_8_long_var_corrupt

-- 
Daniel Qian
Apache Committer(chanjarster)
blog：https://chanjarster.github.io
github：https://github.com/chanjarster
segmentfault: https://segmentfault.com/u/chanjarster

Re: Long variable value get corrupted sometimes

[Daniel Qian]

I'm not familiar with Bash version/release policy, I only found 5.1.8,
5.1.12, 5.1.16 at
download page https://ftp.gnu.org/gnu/bash/

Is this fix included in 5.1.16 version?

Chet Ramey  于2022年2月16日周三 21:59写道：
>
> On 2/16/22 3:10 AM, Daniel Qian wrote:
> > Hi all,
> >
> > I encountered a problem that long variable value get corrupted sometimes.
> >
> > OS: Alpine linux 3.15.0 (docker container)
> > Bash version: GNU bash, version 5.1.8(1)-release (x86_64-alpine-linux-musl)
> >
> > Reproduction steps:
> >
> > A UTF-8 encoded file containing a lot of Chinese characters, file size ~35K.
> >
> > A test script read content from `foo.txt` and assign the content to a 
> > variable,
> > and then check md5sum for that variable.
>
> https://lists.gnu.org/archive/html/bug-bash/2022-01/msg9.html
>
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/



-- 
Daniel Qian
Apache Committer(chanjarster)
blog：https://chanjarster.github.io
github：https://github.com/chanjarster
segmentfault: https://segmentfault.com/u/chanjarster

Re: Long variable value get corrupted sometimes

[Daniel Qian]

Thanks for your tips, a lot learned.

Greg Wooledge  于2022年2月16日周三 20:47写道：

>
> On Wed, Feb 16, 2022 at 04:10:40PM +0800, Daniel Qian wrote:
> > FOO=$(cat /tmp/foo.txt)
> > got_md5=$(echo "$FOO" | md5sum -b | cut -d ' ' -f 1)
>
> In addition to what other people have said... echo is not reliable.  It
> may alter your text, if you feed it backslashes, or arguments like "-e"
> or "-n".
>
> text=$(cat /tmp/foo.txt)# this strips all trailing 
> newlines
> md5=$(printf '%s\n' "$text" | md5sum -b)# this adds one newline
> md5=${md5%% *}  # this removes " *-"
>
> If you need to preserve the correct number of trailing newlines, then
> you'll also have to change the command substitution.  The common
> workaround is:
>
> text=$(cat /tmp/foo.txt; printf x)
> text=${text%x}
>
> If you do this, remember that the final newline(s) are still inside the
> variable, so you don't need to add one:
>
> md5=$(printf %s "$text" | md5sum -b)
>
> Finally, you should get in the habit of NOT using all-caps variable
> names for regular shell variables.  The all-caps namespace is "reserved"
> for environment variables (like HOME and PATH) and special shell variables
> (like BASH_VERSION and SECONDS).
>
> Ordinary variables that you use in a script should contain lowercase
> letters.  Mixed caps/lowercase is fine, if you swing that way.
>


--
Daniel Qian
Apache Committer(chanjarster)
blog：https://chanjarster.github.io
github：https://github.com/chanjarster
segmentfault: https://segmentfault.com/u/chanjarster

declare -F incorrect line number

[Daniel Castro]

   Configuration Information [Automatically generated, do not change]:

   Machine: x86_64

   OS: linux-gnu

   Compiler: gcc

   Compilation CFLAGS: -g -O2
   -fdebug-prefix-map=/build/bash-Smvct5/bash-5.0=.
   -fstack-protector-strong -Wformat -Werror=format-security -Wall
   -Wno-parentheses -Wno-format-security

   uname output: Linux danicc097 5.4.0-126-generic #142-Ubuntu SMP Fri Aug
   26 12:12:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

   Machine Type: x86_64-pc-linux-gnu


   Bash Version: 5.0

   Patch Level: 17

   Release Status: release


   Description:

 declare -F yields the wrong line number for a function that has
   nested functions declared within. Instead it gives the line number of
   the last nested function.


   Repeat-By:

 ```

 #!/bin/bash


   set -e


   fn-a() {

 echo "..."

 echo "..."

 echo "..."


 nested-fn() {

   :

 }


 nested-fn2() {

   :

 }


 nested-fn

   }


   shopt -s extdebug

   ln="$(declare -F fn-a)"

   shopt -u extdebug


   echo "$ln" # fn-a 14, should be 5


 ```

   Fix:

   [Description of how to fix the problem.  If you don't know a

   fix for the problem, don't include this section.]

RE: declare -F incorrect line number

[Daniel Castro]

   In this case I’m not doing anything too fancy: my use case is I have a
   main project script where I declare functions that can get executed
   independently with some prefix: e.g. ``x.build()``, ``x.test()`` so I
   can call ``myscript test`` or ``myscript build`` by parsing all x.*
   functions, and sometimes I have functions within them just for clarity
   since they’re just some abstraction for that particular function logic.

   I parse these x function's comments as well to show in ``--help`` and
   therefore happened to notice the bug. The workaround for me is to
   simply  move them outside the top level function, but I don't know if
   it would get in the way for some people doing more convoluted stuff
   with functions.


   From: [1]Chet Ramey
   Sent: Wednesday, 5 October 2022 20:22
   To: [2]Daniel Castro; [3]bug-bash@gnu.org
   Cc: [4]chet.ra...@case.edu
   Subject: Re: declare -F incorrect line number


   On 10/2/22 4:51 AM, Daniel Castro wrote:


   > Bash Version: 5.0

   >

   > Patch Level: 17

   >

   > Release Status: release

   >

   >

   > Description:

   >

   >   declare -F yields the wrong line number for a function that has

   > nested functions declared within. Instead it gives the line
   number of

   > the last nested function.


   Thanks for the report. I'll take a look, but I have a question. Why are
   you

   declaring functions inside functions? Are you trying to do some kind of

   conditional definition?


   Chet


   --

   ``The lyf so short, the craft so long to lerne.'' - Chaucer

 ``Ars longa, vita brevis'' - Hippocrates

   Chet Ramey, UTech, CWRUc...@case.edu
   http://tiswww.cwru.edu/~chet/

References

   1. mailto:chet.ra...@case.edu
   2. mailto:danicc...@gmail.com
   3. mailto:bug-bash@gnu.org
   4. mailto:chet.ra...@case.edu

Re: Next word of alias to alias that end with is not checked for alias

[Daniel Douglas]

> Of course, all mention of aliases should really be removed from
> POSIX - aliases are even worse than "set -e".

It should be removed because the spec is full of inaccuracies and isn't
useful. People that know enough to define the standard properly have
other priorities clearly. I think many of us are not inclined to follow
the standard anyway because details of alias expansion easily force
unacceptable constraints on the implementation that are more important
than emulating historical shells.

This is one among many examples where it makes little sense to point
out one small difference without considering the overall design of the
parser. There are more where this came from. I could write about alias
bugs all day long if finding small inconsistencies is the only goal.

Re: Vulnerability Report(No SPF Record)

[Daniel Douglas]

On Thu, Feb 16, 2023 at 2:42 PM Greg Wooledge  wrote:
>
> On Thu, Feb 16, 2023 at 07:21:14PM -, Syed Maaz wrote:
> > Hey Team,
> >
> > I am a security researcher,I have found this vulnerability related to 
> > your website bash-hackers.org.
>
> Just for the record, "bash-hackers.org" is a third-party web site, not
> affiliated with the Free Software Foundation or with Chet Ramey.  I'm
> not sure who operates it, but I bet there's a decent chance they'll see
> your messages here.  So, you're mostly in the right place -- just keep
> in mind that it's not an officially supported site.

It is run by the user that used to go by TheBonsai on OFN / freenode.
( that one -> https://wooledge.org/~greybot/meta/ops ). I haven't seen
them around lately.

Re: Vulnerability Report(No SPF Record)

[Daniel Douglas]

On Thu, Feb 16, 2023 at 6:31 PM Daniel Douglas  wrote:
>
> On Thu, Feb 16, 2023 at 2:42 PM Greg Wooledge  wrote:
> >
> > On Thu, Feb 16, 2023 at 07:21:14PM -, Syed Maaz wrote:
> > > Hey Team,
> > >
> > > I am a security researcher,I have found this vulnerability related to 
> > > your website bash-hackers.org.
> >
> > Just for the record, "bash-hackers.org" is a third-party web site, not
> > affiliated with the Free Software Foundation or with Chet Ramey.  I'm
> > not sure who operates it, but I bet there's a decent chance they'll see
> > your messages here.  So, you're mostly in the right place -- just keep
> > in mind that it's not an officially supported site.
>
> It is run by the user that used to go by TheBonsai on OFN / freenode.

Whoops I mean OPN. Typo or Freudian slip.

Re: nofork command substitution

[Daniel Douglas]

These are both very useful!

Re: comparison inside [[]] is not numeric comparison?

[Daniel Fleischman]

>From "man bash"

When  used with [[, The < and > operators sort lexicographically
  using the current locale.

You want (( and )):

$ if (( 1000 > 200 )); then echo pass; else echo wierd; fi
pass

Daniel.

On Fri, Nov 19, 2010 at 19:45, john.ruckstuhl wrote:

> In bash, a comparison inside "[["/"]]" is lexicographic not numeric?
> This isn't what I expected.
> Which part of the documentation would set me straight?  If someone
> could quote the fine manual, that would be great.
>
> $ if [[ 2000 > 200 ]]; then echo pass; else echo wierd; fi
> pass
>
> $ if [[ 1000 > 200 ]]; then echo pass; else echo wierd; fi
> wierd
>
> $ set | grep BASH_VERSION
> BASH_VERSION='3.2.51(24)-release'
>
> Thanks,
> John R.
>

set -e, braces and compound commands

[Daniel Villeneuve]

I'm using GNU bash, version 4.1.2(1)-release (i386-redhat-linux-gnu).

Following the change of semantics of "set -e" in bash 4,"man bash" seems 
to imply that the following script should exit after the for command 
(from the text "or one of the commands executed as part of a command 
list enclosed by  braces" where "one of the commands" should match a 
"for" command) .  However it does not.



set -e
{
  for i in a b c; do
[ -z "$i" ] && echo "null"
  done
  echo "'for' return code = $?"
}
echo "brace return code = $?"


For reference, I've consulted
http://thread.gmane.org/gmane.comp.standards.posix.austin.general/282, 
http://thread.gmane.org/gmane.comp.shells.bash.bugs/13465 and

the current public POSIX spec I have access to which still says:

*-e*
   When this option is on, if a simple command fails for any of the
   reasons listed in Consequences of Shell Errors
   
   or returns an exit status value >0, and is not part of the compound
   list following a *while*, *until*, or *if* keyword, and is not a
   part of an AND or OR list, and is not a pipeline preceded by the *!*
   reserved word, then the shell shall immediately exit.


Question 1: Is it the case that the bash 4 behavior is as intended here 
and that it's the documention of "set -e" in bash 4 that is imprecise 
("commands" being too broad)?


Also, note that the man page section on "trap ERR" says that the trap is 
triggered if a "simple command" returns >0 (sharing exceptions with "set 
-e").  It seems however that the trigger is the same as for "set -e", as in


trap 'echo "error caught ($?), exiting"; exit 1' ERR
(exit 2)
echo done

Question 2: Are "set -e" and "trap ... ERR" triggered by the same events?

Clues would be appreciated.
--
Daniel Villeneuve
AD OPT, a Kronos Division

Re: [patch #7818] Loop breaks when trying to write to a readonly variable

[Daniel Amthor]

Chet,

2012/7/13 Chet Ramey 

> On 7/13/12 8:53 AM, Daniel Amthor wrote:
>
> Yeah, that's an assignment error.  Posix says a non-interactive shell
> should exit in that case and that an interactive shell should write an
> error message without exiting.  Bash doesn't exit the shell, but it does
> cause the current command to fail.
>
So...works as designed ?

Best
Dan

Static analysis of bash scripts

[Daniel Reichelt]

Hi everyone,

I'm maintaining a collection of shell scripts in some 50 different
debian packages. Some of them have no dependencies (with respect to
external commands), some of them depend on other debian packages and
some depend on each other.

I have grown sick of dependency management, yet I want it to be clean
and - newly - automated as far as possible. There are a hand full of
perl scripts out there to parse bash scripts in a "print every 1st word
on a line or whatever stands after a semicolon or a pipe" kind of way
which didn't satisfy me at all.

So I thought why re-invent the wheel...bash already has it's own parser,
why not use it to print every would-be-issued-command?

I did some hacking and came up with the attached patches, which
- add a new command-line option -N (implying -n)
- acts like a "verbose noexec" flag: walk the syntax tree and print each
command with its associated command type.


Currently it's in POC state and I'd like to hear your thoughts and I
have a few questions of my own:
- Would this feature be considered to be included upstream?
- What needs adjustment to be considered (style, technical reasons)?


There are some constructs which still aren't covered and I'm gonna need
help with:
1   a=$(( $(foo) + $(bar) ))
2   $(foo)
3   `foo`
4   case $(foo) in
5   < <(foo)

- 1 through 3 show up as simple_commands with the assignment flag set
but are not further evaluated by the parser (I guess that happens at
runtime...)
- 3 shows up as case_cmd, however unevaluated otherwise as above
- 5 doesn't show up at all in the output of ./bash-patched -nN
./testscript.sh


I don't have any knowledge of the inner workings of parser.y let alone
how to use it or how bash interacts with it - I just happened to have
found the right place to hook into...do you have an idea how to cover these?

Thanks
Daniel
>From 698c8ae89e1bbea8d047d48c66067b09e0c38dc9 Mon Sep 17 00:00:00 2001
From: Daniel Reichelt 
Date: Sun, 31 Mar 2013 15:08:49 +0200
Subject: [PATCH 1/2] add flag 'N'

---
 bash/flags.c |   16 +++-
 bash/flags.h |2 +-
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/bash/flags.c b/bash/flags.c
index d3b38ad..fd78008 100644
--- a/bash/flags.c
+++ b/bash/flags.c
@@ -75,6 +75,11 @@ int place_keywords_in_env = 0;
destructive. */
 int read_but_dont_execute = 0;
 
+/* Non-zero means print commands if read_and_print_but_dont_execute is non-zero.
+   This is useful for static analysis of commands to be executed in a
+   script */
+int read_and_print_but_dont_execute = 0;
+
 /* Non-zero means end of file is after one command. */
 int just_one_command = 0;
 
@@ -184,6 +189,7 @@ const struct flags_alist shell_flags[] = {
   { 'm', &jobs_m_flag },
 #endif /* JOB_CONTROL */
   { 'n', &read_but_dont_execute },
+  { 'N', &read_and_print_but_dont_execute },
   { 'p', &privileged_mode },
 #if defined (RESTRICTED_SHELL)
   { 'r', &restricted },
@@ -274,6 +280,14 @@ change_flag (flag, on_or_off)
 	read_but_dont_execute = 0;
   break;
 
+	case 'N': /* set 'n' as well */
+		if (interactive_shell) {
+			read_but_dont_execute = 0;
+		} else {
+			read_but_dont_execute = 1;
+		}
+		break;
+
 case 'p':
   if (on_or_off == FLAG_OFF)
 	disable_priv_mode ();
@@ -317,7 +331,7 @@ void
 reset_shell_flags ()
 {
   mark_modified_vars = exit_immediately_on_error = disallow_filename_globbing = 0;
-  place_keywords_in_env = read_but_dont_execute = just_one_command = 0;
+  place_keywords_in_env = read_but_dont_execute = read_and_print_but_dont_execute = just_one_command = 0;
   noclobber = unbound_vars_is_error = echo_input_at_read = 0;
   echo_command_at_execute = jobs_m_flag = forced_interactive = 0;
   no_symbolic_links = no_invisible_vars = privileged_mode = pipefail_opt = 0;
diff --git a/bash/flags.h b/bash/flags.h
index d8fa757..aab4291 100644
--- a/bash/flags.h
+++ b/bash/flags.h
@@ -42,7 +42,7 @@ extern char optflags[];
 
 extern int
   mark_modified_vars, exit_immediately_on_error, disallow_filename_globbing,
-  place_keywords_in_env, read_but_dont_execute,
+  place_keywords_in_env, read_but_dont_execute, read_and_print_but_dont_execute,
   just_one_command, unbound_vars_is_error, echo_input_at_read,
   echo_command_at_execute, no_invisible_vars, noclobber,
   hashing_enabled, forced_interactive, privileged_mode,
-- 
1.7.2.5

>From a0ca52e731716d5339036df0a19f1ed23b598b73 Mon Sep 17 00:00:00 2001
From: Daniel Reichelt 
Date: Tue, 2 Apr 2013 10:39:11 +0200
Subject: [PATCH 2/2] print command type in eval.c and adjust Makefile

---
 bash/Makefile.in   |4 ++
 bash/eval.c|2 +
 bash/print_cmd_on_noexec.c |  126 
 bash/print_cmd_on_noexec.h |   30 ++
 bash/testsh|   16 ++
 5 files changed, 178 i

Static analysis of bash scripts

[Daniel Reichelt]

(2nd try, after 5hrs the first mail still didn't go through but didn't
produce any non-DSN either)


Hi everyone,

I'm maintaining a collection of shell scripts in some 50 different
debian packages. Some of them have no dependencies (with respect to
external commands), some of them depend on other debian packages and
some depend on each other.

I have grown sick of dependency management, yet I want it to be clean
and - newly - automated as far as possible. There are a hand full of
perl scripts out there to parse bash scripts in a "print every 1st word
on a line or whatever stands after a semicolon or a pipe" kind of way
which didn't satisfy me at all.

So I thought why re-invent the wheel...bash already has it's own parser,
why not use it to print every would-be-issued-command?

I did some hacking and came up with the attached patches, which
- add a new command-line option -N (implying -n)
- acts like a "verbose noexec" flag: walk the syntax tree and print each
command with its associated command type.


Currently it's in POC state and I'd like to hear your thoughts and I
have a few questions of my own:
- Would this feature be considered to be included upstream?
- What needs adjustment to be considered (style, technical reasons)?


There are some constructs which still aren't covered and I'm gonna need
help with:
1   a=$(( $(foo) + $(bar) ))
2   $(foo)
3   `foo`
4   case $(foo) in
5   < <(foo)

- 1 through 3 show up as simple_commands with the assignment flag set
but are not further evaluated by the parser (I guess that happens at
runtime...)
- 3 shows up as case_cmd, however unevaluated otherwise as above
- 5 doesn't show up at all in the output of ./bash-patched -nN
./testscript.sh


I don't have any knowledge of the inner workings of parser.y let alone
how to use it or how bash interacts with it - I just happened to have
found the right place to hook into...do you have an idea how to cover these?

Thanks
Daniel

>From 698c8ae89e1bbea8d047d48c66067b09e0c38dc9 Mon Sep 17 00:00:00 2001
From: Daniel Reichelt 
Date: Sun, 31 Mar 2013 15:08:49 +0200
Subject: [PATCH 1/2] add flag 'N'

---
 bash/flags.c |   16 +++-
 bash/flags.h |2 +-
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/bash/flags.c b/bash/flags.c
index d3b38ad..fd78008 100644
--- a/bash/flags.c
+++ b/bash/flags.c
@@ -75,6 +75,11 @@ int place_keywords_in_env = 0;
destructive. */
 int read_but_dont_execute = 0;
 
+/* Non-zero means print commands if read_and_print_but_dont_execute is non-zero.
+   This is useful for static analysis of commands to be executed in a
+   script */
+int read_and_print_but_dont_execute = 0;
+
 /* Non-zero means end of file is after one command. */
 int just_one_command = 0;
 
@@ -184,6 +189,7 @@ const struct flags_alist shell_flags[] = {
   { 'm', &jobs_m_flag },
 #endif /* JOB_CONTROL */
   { 'n', &read_but_dont_execute },
+  { 'N', &read_and_print_but_dont_execute },
   { 'p', &privileged_mode },
 #if defined (RESTRICTED_SHELL)
   { 'r', &restricted },
@@ -274,6 +280,14 @@ change_flag (flag, on_or_off)
 	read_but_dont_execute = 0;
   break;
 
+	case 'N': /* set 'n' as well */
+		if (interactive_shell) {
+			read_but_dont_execute = 0;
+		} else {
+			read_but_dont_execute = 1;
+		}
+		break;
+
 case 'p':
   if (on_or_off == FLAG_OFF)
 	disable_priv_mode ();
@@ -317,7 +331,7 @@ void
 reset_shell_flags ()
 {
   mark_modified_vars = exit_immediately_on_error = disallow_filename_globbing = 0;
-  place_keywords_in_env = read_but_dont_execute = just_one_command = 0;
+  place_keywords_in_env = read_but_dont_execute = read_and_print_but_dont_execute = just_one_command = 0;
   noclobber = unbound_vars_is_error = echo_input_at_read = 0;
   echo_command_at_execute = jobs_m_flag = forced_interactive = 0;
   no_symbolic_links = no_invisible_vars = privileged_mode = pipefail_opt = 0;
diff --git a/bash/flags.h b/bash/flags.h
index d8fa757..aab4291 100644
--- a/bash/flags.h
+++ b/bash/flags.h
@@ -42,7 +42,7 @@ extern char optflags[];
 
 extern int
   mark_modified_vars, exit_immediately_on_error, disallow_filename_globbing,
-  place_keywords_in_env, read_but_dont_execute,
+  place_keywords_in_env, read_but_dont_execute, read_and_print_but_dont_execute,
   just_one_command, unbound_vars_is_error, echo_input_at_read,
   echo_command_at_execute, no_invisible_vars, noclobber,
   hashing_enabled, forced_interactive, privileged_mode,
-- 
1.7.2.5


>From a0ca52e731716d5339036df0a19f1ed23b598b73 Mon Sep 17 00:00:00 2001
From: Daniel Reichelt 
Date: Tue, 2 Apr 2013 10:39:11 +0200
Subject: [PATCH 2/2] print command type in eval.c and adjust Makefile

---
 bash/Makefile.in   |4 ++
 bash/eval.c|2 +
 bash/print_cmd_on_noexec.c |  126 ++

Numbers in the begining of variable names generates problem

[Daniel Hagberg]

Message-Id: 
Date: Fri, 11 Oct 2013 17:43:42 +0200

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' 
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' 
-DSHELL -DHAVE_CONFIG_H   -I.  -I../bash -I../bash/include 
-I../bash/lib  -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector 
--param=ssp-buffer-size=4 -Wformat -Wformat-security 
-Werror=format-security -Wall
uname output: Linux StefanO-desktop 3.2.0-54-generic #82-Ubuntu SMP Tue 
Sep 10 20:08:42 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Machine Type: x86_64-pc-linux-gnu

Bash Version: 4.2
Patch Level: 25
Release Status: release

Description:
the command:
export 2_ave=$(cat 
"$type"_"$simulation"_"$from_year"_"$to_year"_"$variable"_areaaverage.txt|grep 
-v Col|head -n 1|sed 's/ //g')

generates:
not a valid identifier
but:
export _ave=$(cat 
"$type"_"$simulation"_"$from_year"_"$to_year"_"$variable"_areaaverage.txt|grep 
-v Col|head -n 1|sed 's/ //g')

works fine ...

Repeat-By:
A number in the beginning of the name generates this problem

Fix:

--

Daniel Hagberg

Physical Geography and Ecosystem Analysis
Land-Atmosphere Group
Lund University
Sölvegatan 12, 22362 Lund, Sweden.

Email daniel.hagb...@nateko.lu.se

Phone +46 46 2223659
Cellphone +46 735925452

Inconsistent string comparison operators n and z

[Thibault, Daniel]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' 
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I../bash -I../bash/include -I../bash/lib  
-D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat 
-Wformat-security -Werror=format-security -Wall
uname output: Linux sds-dut-vb 3.9.3 #1 SMP Mon Mar 24 18:48:39 EDT 2014 x86_64 
x86_64 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 4.2
Patch Level: 25
Release Status: release

Description:
 The string comparison operators -n and -z are designed to be mutually
complementary. ! -z should be interchangeable with -n and ! -n should be
interchangeable with -z. But such is not the case. Consider these lines:

$ if [   -z `pgrep pname` ]; then echo "not r" ; else echo "r" ; fi
$ if [ ! -z `pgrep pname` ]; then echo "r" ; else echo "not r" ; fi
$ if [   -n `pgrep pname` ]; then echo "r" ; else echo "not r" ; fi
$ if [ ! -n `pgrep pname` ]; then echo "not r" ; else echo "r" ; fi

They should be equivalent but are not: -z correctly detects the process's
presence or absence, while -n returns true even when the process is not
running.

Turns out this is how the script needs to be written to work correctly:

$ if [   -z "`pgrep pname`" ]; then echo "not r" ; else echo "r" ; fi
$ if [ ! -z "`pgrep pname`" ]; then echo "r" ; else echo "not r" ; fi
$ if [   -n "`pgrep pname`" ]; then echo "r" ; else echo "not r" ; fi
$ if [ ! -n "`pgrep pname`" ]; then echo "not r" ; else echo "r" ; fi

Repeat-By:
 See the examples above.

`read -a': spurious ctrl char when unescaping

[Daniel Dawson]

Configuration Information [Automatically generated, do not change]:
Machine: i486
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='i486' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i486-pc-linux-gnu' 
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I../bash -I../bash/include -I../bash/lib   -g -O2
uname output: Linux ddawson.foo 2.6.17-ddawson #5 Sat Sep 16 12:04:46 PDT 2006 
i686 GNU/Linux
Machine Type: i486-pc-linux-gnu

Bash Version: 3.1
Patch Level: 17
Release Status: release

Description:
`read -a' is inserting a spurious ASCII 1 whenever it encounters a
backslash-escaped character. To be precise, it appears the backslash is
simply replaced by this character. For instance, the string "foo\ bar"
comes out as the sequence 66 6f 6f 01 20 62 61 72, instead of
66 6f 6f 20 62 61 72.

Repeat-By:
$ read -a ary
foo\ bar\\baz\"quux
$ echo -n ${ary[0]} | hexdump -C


___
Bug-bash mailing list
Bug-bash@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-bash

read builtin oddity/bug in bash 3.2?

[Daniel Musgrave]

I have found a seeming inconsistency with the behavior of the read
builtin of bash (3.2.0(1)-release, also tested in 3.00.15(1)-release).
I'm working on a Centos 4.4 system (RedHat derivative).  Let me describe
the conditions that cause the bug in as much detail as I have discovered
thus far.

If the current working directory is /usr/bin (and only /usr/bin, I
think), and you use the 'while read var; do ... ; done < file' construct
inside a shell script, read appears to incorrectly set the value of var
when file contains a line that:

 * starts and ends with '[' and ']' characters, respectively; and
 * contains no whitespace (this condition tested with all alphanumeric
characters, as well as a few punctuation chars); and
 * contains one or more 'w' characters

If the line matches the above conditions and is echoed without being
quoted ($var, not "$var"), the result is the single character 'w'.  If 
the line is quoted while being echoed, it prints normally.  Similarly 
(and this is the kicker, in my mind), if the working directory isn't 
/usr/bin, this behavior does not occur, regardless of the quoting on $var.


Below is the sample script and file that I used to test this behavior. 
You'll have to change the location of the file it reads.


#!/bin/bash
#
# Strange read bash builtin behavior

echo -e "Bash version: $BASH_VERSION\n"

bug_dir=/usr/bin
other_dir=/usr/sbin

run() {
  cd $1
  echo "in $PWD"
  while read line; do
[ "$line" ] && {
  echo -e "$line  =>  \c"
  echo $line
} || echo
  done < ~/list
  echo
}

run $bug_dir
run $other_dir
exit 0

-

[w]
[w ]
[ w]
[ w ]
[w][w]
[w][q]
[w [w]
[w [w]]
[www]
[w13]
[word]
[throw]
[away]
[award award]

-


I have tried several values for other_dir, including /usr/sbin, /bin,
/sbin, /etc, and /home, all of which do not exhibit this behavior.

So, down to the real question.  Is this somehow intended behavior?  It
seems this only happens with 'w'; I tested all the remaining
alphanumeric characters, all of which worked fine.  Similarly, 'echo
[w]' produces the expected output of '[w]', and you can actually modify
the above code to test the value of $var to see that it indeed does
equal 'w' when the above error conditions are met.  Obviously this is a
surmountable problem, for it can be avoided by either changing the
working directory or quoting var when using it.  On the other hand, it 
seems that consistent behavior is probably desired, even if this is

corner of a corner case ;) .

Daniel Musgrave
Abodio Software
(206) 454-1024
[EMAIL PROTECTED]



___
Bug-bash mailing list
Bug-bash@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-bash

Re: read builtin oddity/bug in bash 3.2?

[Daniel Musgrave]

Chet Ramey wrote:
> Daniel Musgrave wrote:
>> I have found a seeming inconsistency with the behavior of the read
>> builtin of bash (3.2.0(1)-release, also tested in 3.00.15(1)-release).
>> I'm working on a Centos 4.4 system (RedHat derivative).  Let me describe
>> the conditions that cause the bug in as much detail as I have discovered
>> thus far.
>
> There are a couple of things that might shed light on your situation.
> First, [w] is a globbing pattern that matches `w'.  Second, /usr/bin/w
> exists as an executable file.
>
> So, in /usr/bin, [w] will be globbed to `w', since there is a matching
> filename.
>
> Chet
>

Of course, I had completely forgotten about globbing.  I suppose the first 
thing I should have done would be to look in /usr/bin for a file called `w', 
but up until today I was unaware there were any binaries with single-character 
names (it would have been more obvious if I were getting `cp' back, or 
something).  After reading the filename expansion section of the manual, it 
seems that setting the nullglob shell option would make the other patterns that 
I was inadvertently specifying expand into a null string if they didn't match 
any files in the directory, rather than just to the pattern themselves.

Thanks for clearing that up.

Daniel




___
Bug-bash mailing list
Bug-bash@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-bash

Feature request for the 'local' builtin command - passthrough return status

[Daniel Webb]

First off, please CC me on any replies because there are no instructions on
the FSF bash page (http://directory.fsf.org/bash.html) to tell me how to
subscribe to this list.

My request is to add a new switch to the 'local' builtin which tells it to not
modify the return status (I use the -p switch for this below).  I have never
once used the return status from local, although I understand it is necessary.
However, I often use local like this:

local var=$(echo -n message | something_pipey) 

However, if I want to check the return status of my command substitution, I
have to do something like this:

local var
var=$(echo -n message | something_pipey) || die

Much nicer would be something like this:

local -p var=$(echo -n message | something_pipey) || die

Perhaps it is silly to ask for this, but at the same time, it makes sense
to me because there is variable assignment allowed within the local command.  
If you allow assignment you have to assume that someone's going to use command
substitution to make the assignment.



___
Bug-bash mailing list
Bug-bash@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-bash

Offset error (when using PS1 with colors and newline)

[dAniel hAhler]

Configuration Information [Automatically generated, do not change]:
Machine: i486
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='i486' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i486-pc-linux-gnu' 
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I../bash -I../bash/include -I../bas
h/lib   -g -O2 -Wall
uname output: Linux base 2.6.22-7-generic #1 SMP Mon Jun 25 17:33:14 GMT 2007 
i686 GNU/Linux
Machine Type: i486-pc-linux-gnu

Bash Version: 3.2
Patch Level: 17
Release Status: release

Description:
If using colors and a newline (\n) in PS1, there is some offset error,
with autocompletion or ctrl-a. It appears to be related to readline.

Please see https://bugs.launchpad.net/ubuntu/+source/bash/+bug/119938
for the bug reported in Ubuntu and another recipe to reproduce it.

Repeat-By:
1. Set PS1="\033[01;37m[ \[\033[01;34m\]\w\[\033[00m\] 
\033[01;37m]\n\$\033[00m "
2. Type "foo bar baz"
3. Press ctrl+a
4. Cursor should jump back to the beginning of the line, but instead 
jumps back to "foo b."

This is how it looks like with "_" being the cursor:
foo b_ar baz


signature.asc
Description: This is a digitally signed message part.
___
Bug-bash mailing list
Bug-bash@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-bash

Re: Offset error (when using PS1 with colors and newline)

[dAniel hAhler]

On 2007-07-12 Andreas Schwab wrote:

> > 1. Set PS1="\033[01;37m[ \[\033[01;34m\]\w\[\033[00m\]
> > \033[01;37m]\n\$\033[00m "
>
> You need to bracket _every_ nonprinting sequence of characters with \[\].

I've asked you before, but you've not answered, or it got lost.

What nonprinting sequence is not escaped above?

And is there anything unescaped in the testcase below?
1. PS1="\n\[\033[0m\]"
2. Type "ls ./_ /foo" ("_" => cursor)
3. Press TAB twice, where the "_" is above

Result:

ls ./ /foo
[...]

ls ./ /foo_

(where "_" is the cursor again)

I've just tested this with bash 3.2.25(1).

For reference, the bug has been reported at launchpad:
https://bugs.launchpad.net/ubuntu/+source/bash/+bug/119938


-- 
http://daniel.hahler.de/


signature.asc
Description: This is a digitally signed message part.

Re: Offset error (when using PS1 with colors and newline)

[dAniel hAhler]

Matthew Woehlke wrote:

 1. Set PS1="\033[01;37m[ \[\033[01;34m\]\w\[\033[00m\]
 \033[01;37m]\n\$\033[00m "
>>> You need to bracket _every_ nonprinting sequence of characters with
>>> \[\].
>>
>> I've asked you before, but you've not answered, or it got lost.
>>
>> What nonprinting sequence is not escaped above?
> 
> Two occurrences of "\033[01;37m]".

I see now. Thanks for pointing at it.

Fixing the PS1 above still triggers the bug, if you use "ls ./_ /fo"
though.
Please note that the PS1 above does not come from me, but from a bug
comment. There seems to be an issue

Can you reproduce the bug with PS1="\n\[\033[0m\]" (which seems to be
valid as far as I can see)?
I can trigger the misplacement of the cursor there with just "ls ./",
followed by tab-tab, while PS1="\[\033[0m\]\n" works.

It seems to get triggered by a newline and following non-printable
characters.

Typo leads to wrong formatting in manpage

[Daniel Hornung]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: x86_64-pc-linux-gnu-gcc
Compilation 
CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' 
-DCONF_MACHTYPE='x86_64-pc-linux-gnu' -DCONF_VENDOR='pc' 
-DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H   -I.  
-I. -I./include -I./lib   -O2 -march=athlon64 -fomit-frame-pointer -pipe
uname output: Linux whatnow 2.6.22-gentoo-r9 #4 SMP Tue Nov 13 23:43:43 CET 
2007 x86_64 AMD Athlon(tm) X2 Dual Core Processor BE-2300 AuthenticAMD 
GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 3.2
Patch Level: 17
Release Status: release

Description:
There's a formatting error in the man page for bash, leading to 
continuous 
underlining of every word for some sentences.

Repeat-By:
"man bash"
"/longest match of"
The bug appears in the next sentence.  (EXPANSION -> Parameter 
Expansion -> 
${parameter/pattern/string})

Fix:
This might fix it, I don't know *roff, but I guess it might work.

--- bash.1~ 2007-11-29 20:42:55.0 +0100
+++ bash.1  2007-11-29 20:44:35.0 +0100
@@ -2539,7 +2539,7 @@
 pathname expansion.
 \fIParameter\fP is expanded and the longest match of \fIpattern\fP
 against its value is replaced with \fIstring\fP.
-If \Ipattern\fP begins with \fB/\fP, all matches of \fIpattern\fP are
+If \fIpattern\fP begins with \fB/\fP, all matches of \fIpattern\fP are
 replaced with \fIstring\fP.  Normally only the first match is replaced.
 If \fIpattern\fP begins with \fB#\fP, it must match at the beginning
 of the expanded value of \fIparameter\fP.


I hope this helped you a bit


signature.asc
Description: This is a digitally signed message part.

kill job vs. pid

[Daniel Norton]

A simple question and perhaps its simplicity explains why I can't find
an answer elsewhere:

  How do I tell bash to kill job 1, rather than pid 1 ?

What does "kill <>" mean?  What if there is no jobspec <> and I
type "kill <>" ?

e..g.

$ jobs
[1]+  Stopped blahblahblah
$ ps 1
  PID TTY  STAT   TIME COMMAND
1 ?    Ss 0:02 /sbin/init

Thanks.

--
Daniel

Multiple Words on Bash Programmable Auto Completion

[Daniel Fleischman]

Description:
I don't know if this is a bug or not, but it has to do with programmable
auto completion. Whenever the options have multiple words I don't get what I
want.
Here is an example, just for show:

f() {
COMPREPLY=""
c=0
while read line; do
COMPREPLY[$c]=$line
c=$[$c + 1]
done <<< "$(compgen -W "a\ b a\ b\ c b" --
${COMP_WORDS[COMP_CWORD]})"
}
complete -F f lala

now watch this:

[EMAIL PROTECTED] ~ $ lala 
a b   a b c  b
[EMAIL PROTECTED] ~ $ lala

This is what I expected, but now, if I do:

[EMAIL PROTECTED] ~ $ lala a

I will get:

[EMAIL PROTECTED] ~ $ lala a b

while I expected

[EMAIL PROTECTED] ~ $ lala a\ b

as auto-completion normaly does with file names.

I "fixed" this by changing one line of the function f. The new line is:

COMPREPLY[$c]=${line// /\\ }

But now, I get this:

[EMAIL PROTECTED] ~ $ lala 
a\ b   a\ b\ c   b
[EMAIL PROTECTED] ~ $ lala

So, what I want to know is: is there any way to have the spaces escaped
when completing a word, but not when showing the completion options
(COMPREPLY)?

If so, please, tell me how. If not, well, I've found a bug :P

Thank you in advance,
Daniel Fleischman.

Repeat-By:
1) Type the "f" function
2) write "lala a" on bash

Re: Multiple Words on Bash Programmable Auto Completion

[Daniel Fleischman]

Thank you, that's exactly what I wanted.

On Thu, Oct 2, 2008 at 09:16, Chet Ramey <[EMAIL PROTECTED]> wrote:

> > Description:
> > I don't know if this is a bug or not, but it has to do with
> programmable
> > auto completion. Whenever the options have multiple words I don't get
> what I
> > want.
>
> You can get filename-like quoting by specifying that readline should treat
> the matches returned by the completion function as filenames, using the
> `-o filenames' option to complete: complete -o filenames -F f lala.
>
> Chet
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>
> Chet Ramey, ITS, CWRU[EMAIL PROTECTED]
> http://tiswww.tis.case.edu/~chet/ 
>

Re: Variable getopts lost

[Daniel Bunzendahl]

Am Dienstag, 23. Februar 2010 12:50:41 schrieb Pierre Gaston:
> On Tue, Feb 23, 2010 at 1:14 AM, DanielBu  wrote:
> > Hello all,
> >
> > I get crazy with getopts:
> > Some Times my script (500 Lines) don't take input parameters like this:
> 
> It's really impossible to make a guess with a report as vague as that.
> 

#!/bin/bash


while getopts ':b:d:f:l:o:vh' OPTION ; do
 case $OPTION in
 v) VERBOSE=y
 ;;
 b) pdffront="$OPTARG"
 ;;
 d) BOOKLET="$OPTARG"
 echo "Booklet: "$BOOKLET
 ;;
 f) FSEITE="$OPTARG"
 ;;
 l) LSEITE="$OPTARG"
 ;;
 o) FORMAT="$OPTARG"
 ;;
 h) usage $EXIT_SUCCESS
 ;;
 \?) echo "Unbekannte Option \"-$OPTARG\"." >&2
 usage $EXIT_ERROR
 ;;
 :) echo "Option \"-$OPTARG\" benötigt ein Argument." >&2
 usage $EXIT_ERROR
 ;;
 *) echo "Dies kann eigentlich gar nicht passiert sein..."
>&2
 usage $EXIT_BUG
 ;;
 esac
done

# Verbrauchte Argumente überspringen
shift $(( OPTIND - 1 ))
# Eventuelle Tests auf min./max. Anzahl Argumente hier
if (( $# < 1 )) ; then
 echo "Du hast was vergessen (evtl. PDF-Datei oder Argument für einen 
Schalter)" >&2
 usage $EXIT_ERROR
fi

case $BOOKLET in
4) papersizex=500
   papersizey=700
   scalerate=90
   scalerateif=90
   ;;
5) papersizex=420
   papersizey=595
   scalerate=90
   scalerateif=50
   ;;
6) papersizex=298
   papersizey=420
   scalerate=90
   scalerateif=25
   ;;
*) papersizex=500
   papersizey=700
   scalerate=90
   scalerateif=90
   ;;
esac


echo "Breite x Höhe: "$papersizex"x"$papersizey
resthight=$papersizey
restbreite=$papersizex
echo "Restbreite x Resthöhe: "$restbreite"x"$resthight
booklet_page="1"
booklet_page_counter="1"


# Schleife über alle Argumente
for ARG ; do
 if [[ $VERBOSE = y ]] ; then
 echo -n "Argument: Ich bin eine V-erquasseltes Bash-Script"
 fi
 echo "Argument: " $ARG
done

pdf=$1


if [ !$LSEITE ]; then 
 LSEITE=$(pdfinfo $pdf | grep Pages: | sed -e 's/Pages:[[:space:]]//g')
echo "-l automatisch auf $LSEITE gesetzt"
fi

...

 
and so on (500 lines... so I don't show it all)

In the last if-loop LSEITE will be set if LSEITE isn't set.
This is for no parameters on command-line.
But how I wrote: It ever works but now it lost the -l 104 ... the -f is no 
Problem...

My question wasn't fokused on my wrong script. I think there is something 
wrong or limited by the System...
Maybe you can give me a tip I should search for...

Thanks a lot
Daniel :-)

-- 
Mit fitten und gesunden Grüßen
Daniel Bunzendahl

http://www.Bunzendahl.net

Umsatzsteuer-Identifikationsnummer: 08132/03257 

Unsere Empfehlung:
http://www.wsim.de/ref.php?id=16436 (Manager Game)

==
Hinweis: Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte 
Informationen. Wenn Sie nicht der richtige Adressat sind oder dieses E-Mail 
irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und 
vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte 
Weitergabe dieser Mail ist nicht gestattet. --- NOTICE: This e-mail may 
contain confidential or privileged material and is intended for use solely by 
the above-referenced recipient. Any review, copying, printing, disclosure, 
distribution, or other use by any other person or entity is strictly 
prohibited. If you are not the named recipient, or believe you have received 
this e-mail in error, please reply to the sender and delete
the copy you received. Thank you.

Re: Variable getopts lost

[Daniel Bunzendahl]

Am Dienstag, 23. Februar 2010 20:45:31 schrieb Greg Wooledge:
> On Tue, Feb 23, 2010 at 08:30:16PM +0100, Daniel Bunzendahl wrote:
> > if [ !$LSEITE ]; then
> 
> You want:  if [ ! "$LSEITE" ]

this dosn't work.

But I earsed the if-loop. And it works. (dont overwrite LSEITE)
But I need this check in case somebody will put last page to work with...

maybe I try thinks like

test  

If I don't find a way, I will ask again.
Thanks for your time :-)

Gratings
Daniel

> There are probably more errors.  This is just the line you mentioned
> in particular as not working.
> 

-- 
Mit fitten und gesunden Grüßen
Daniel Bunzendahl

http://www.Bunzendahl.net

Umsatzsteuer-Identifikationsnummer: 08132/03257 

Unsere Empfehlung:
http://www.wsim.de/ref.php?id=16436 (Manager Game)

==
Hinweis: Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte 
Informationen. Wenn Sie nicht der richtige Adressat sind oder dieses E-Mail 
irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und 
vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte 
Weitergabe dieser Mail ist nicht gestattet. --- NOTICE: This e-mail may 
contain confidential or privileged material and is intended for use solely by 
the above-referenced recipient. Any review, copying, printing, disclosure, 
distribution, or other use by any other person or entity is strictly 
prohibited. If you are not the named recipient, or believe you have received 
this e-mail in error, please reply to the sender and delete
the copy you received. Thank you.

shell-forward-word and shell-backward-word not symmetric

[Daniel Colascione]

Consider the string "word1 word2 word3 word4 word5" with point just
before "word3". In bash-4.1, shell-backword-word moves point to the
beginning of the string, but shell-forward-word just advances to the end
of word4. The two functions really should be symmetric --- either both
should break out of quoted strings, or neither should. The latter
behavior is probably better, and it's more consistent with Emacs.



signature.asc
Description: OpenPGP digital signature

Du wirst ausspioniert ....!

[daniel . frischkorn]

und weisst es nicht einmal:

http://www.heise.de/newsticker/meldung/58003
http://www.heise.de/newsticker/meldung/59304

http://www.heise.de/newsticker/meldung/58311


http://www.heise.de/newsticker/meldung/58351


___
Bug-bash mailing list
Bug-bash@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-bash

Re: broken pipe from func-output by changing network settings?

[Daniel Mills]

On Tue, Oct 17, 2017 at 1:37 PM, Ángel  wrote:

> bash converts < <(  dd_need_io "$if" "$of"  ...)  into a read
> from /dev/fd/62 in order to make readarray read file descriptor 62.
>
> Given that this the host OS doesn't provide them, the first thing I
> would verify would be: is cygwin, as setup on that system, providing
> such descriptors?
>
> As a simple test, you can do:
>  wc -l <(cat /etc/passwd)
>
> (change /etc/passwd to another file if that's not available)
>
> Regards
>
>
> Cygwin implements mkfifo, which bash will use in place of /dev/fd

Feature request: PROMPT_COMMANDS array variable

[Daniel Colascione]

Right now, PROMPT_COMMAND gives a shell command to run before displaying
the prompt. It's common these days to include in one's bash configuration
numerous packages from different sources that *all* want to run code at
PROMPT_COMMAND time. Can we add a new PROMPT_COMMANDS array variable that
stores a list of shell commands to execute? With a PROMPT_COMMANDS
variable, different packages can independently add their hooks without
stepping on each other.

Re: Feature request: PROMPT_COMMANDS array variable

[Daniel Colascione]

Sure, but you have a coordination problem. Everyone is going to use a
different array name, and since scripts have no way of knowing in advance
to what array they need to add their function, we end up the same spot.
Putting the array in the core would solve the coordination problem.

On Jan 23, 2018 7:05 PM, "Clark Wang"  wrote:

> On Wed, Jan 24, 2018 at 2:23 AM, Daniel Colascione 
> wrote:
>
>> Right now, PROMPT_COMMAND gives a shell command to run before displaying
>> the prompt. It's common these days to include in one's bash configuration
>> numerous packages from different sources that *all* want to run code at
>> PROMPT_COMMAND time. Can we add a new PROMPT_COMMANDS array variable that
>> stores a list of shell commands to execute? With a PROMPT_COMMANDS
>> variable, different packages can independently add their hooks without
>> stepping on each other.
>>
>
> I would define my own array var and go through it in PROMPT_COMMAND. (My
> PROMPT_COMMAND is just a function name.)
>

[PATCH] Add active mark, face support; activate mark on paste

[Daniel Colascione]

This patch teaches readline about two concepts from Emacs: 1) faces,
and 2) the mark being "active". Both exist in rudimentary form: we
support exactly two faces, normal and "standout", and use standout to
highlight the contents of the region when the mark is active. Readline
redisplay is now smart enough to consider not only text content, but
also face differences when computing how to refresh the screen.

The immediate motivation for this patch is to provide visual feedback
to users after a bracketed paste operation, but an active region
concept could be useful for all kinds of things, including shift-arrow
selection or integration with xterm mouse facilities.

The mark automatically deactivates after most commands, including C-g
and cursor motion. We do keep any active mark active across screen
refresh operations, however.

exchange-point-and-mark activates the mark just like in Emacs.
---
 lib/readline/display.c   | 478 +++
 lib/readline/kill.c  |   6 +-
 lib/readline/readline.c  |  19 ++
 lib/readline/readline.h  |   4 +
 lib/readline/rlprivate.h |   5 +-
 lib/readline/terminal.c  |  43 
 lib/readline/text.c  |  55 +++--
 lib/readline/util.c  |   1 +
 8 files changed, 398 insertions(+), 213 deletions(-)

diff --git a/lib/readline/display.c b/lib/readline/display.c
index 2d2e768a..f228a39e 100644
--- a/lib/readline/display.c
+++ b/lib/readline/display.c
@@ -63,13 +63,13 @@
 extern char *strchr (), *strrchr ();
 #endif /* !strchr && !__STDC__ */
 
-static void update_line PARAMS((char *, char *, int, int, int, int));
+static void update_line PARAMS((char *, char *, char *, char *, int, int, int, 
int));
 static void space_to_eol PARAMS((int));
 static void delete_chars PARAMS((int));
-static void insert_some_chars PARAMS((char *, int, int));
 static void open_some_spaces PARAMS((int));
 static void cr PARAMS((void));
 static void redraw_prompt PARAMS((char *));
+static void _rl_move_cursor_relative PARAMS((int, const char *, const char *));
 
 /* Values for FLAGS */
 #define PMT_MULTILINE  0x01
@@ -80,6 +80,7 @@ static char *expand_prompt PARAMS((char *, int, int *, int *, 
int *, int *));
 struct line_state
   {
 char *line;
+char *line_face;
 int *lbreaks;
 int lbsize;
 #if defined (HANDLE_MULTIBYTE)
@@ -102,7 +103,9 @@ static int line_structures_initialized = 0;
 #define vis_lbsize (line_state_visible->lbsize)
 
 #define visible_line   (line_state_visible->line)
+#define visible_face (line_state_visible->line_face)
 #define invisible_line (line_state_invisible->line)
+#define invisible_face (line_state_invisible->line_face)
 
 #if defined (HANDLE_MULTIBYTE)
 static int _rl_col_width PARAMS((const char *, int, int, int));
@@ -123,7 +126,10 @@ static int _rl_col_width PARAMS((const char *, int, int, 
int));
to use prompt_last_invisible directly. */
 #define PROMPT_ENDING_INDEX \
   ((MB_CUR_MAX > 1 && rl_byte_oriented == 0) ? prompt_physical_chars : 
prompt_last_invisible+1)
-  
+
+#define FACE_NORMAL '0'
+#define FACE_STANDOUT '1'
+#define FACE_INVALID ((char)1)
 
 /*  */
 /* */
@@ -208,8 +214,8 @@ static int msg_bufsiz = 0;
 /* Non-zero forces the redisplay even if we thought it was unnecessary. */
 static int forced_display;
 
-/* Default and initial buffer size.  Can grow. */
-static int line_size = 1024;
+/* Line buffer size. */
+static int line_size;
 
 /* Variables to keep track of the expanded prompt string, which may
include invisible characters. */
@@ -521,6 +527,64 @@ rl_expand_prompt (prompt)
 }
 }
 
+static void
+ensure_line_size (minsize)
+ int minsize;
+{
+  int minimum_size = 1024;
+  int new_line_size, delta;
+  if (minsize < minimum_size)
+minsize = minimum_size;
+  if (line_size >= minsize)
+return;
+  if (!new_line_size)
+new_line_size = minimum_size;
+  while (new_line_size < minsize)
+new_line_size *= 2;
+  visible_line = (char *)xrealloc (visible_line, new_line_size);
+  visible_face = (char *)xrealloc (visible_face, new_line_size);
+  invisible_line = (char *)xrealloc (invisible_line, new_line_size);
+  invisible_face = (char *)xrealloc (invisible_face, new_line_size);
+  delta = new_line_size - line_size;
+  memset (visible_line + line_size, 0, delta);
+  memset (visible_face + line_size, FACE_NORMAL, delta);
+  memset (invisible_line + line_size, 1, delta);
+  memset (invisible_face + line_size, FACE_INVALID, delta);
+  line_size = new_line_size;
+}
+
+static void
+invis_add (outp, c, face)
+ int *outp;
+ char c;
+ char face;
+{
+  ensure_line_size (*outp + 1);
+  invisible_line[*outp] = c;
+  invisible_face[*outp] = face;
+  *outp += 1;
+}
+
+static void
+invis_add_n (outp, str, n, face)
+ int *outp;
+ const char *str;
+ int n;
+ char face;
+{
+  int i;
+  for (i = 0; i < n; ++i)
+invis_add (outp, str[i], face);
+}
+

Re: [PATCH] Add active mark, face support; activate mark on paste

[Daniel Colascione]

Ping

On Fri, Mar 9, 2018 at 11:50 PM, Daniel Colascione 
wrote:

> This patch teaches readline about two concepts from Emacs: 1) faces,
> and 2) the mark being "active". Both exist in rudimentary form: we
> support exactly two faces, normal and "standout", and use standout to
> highlight the contents of the region when the mark is active. Readline
> redisplay is now smart enough to consider not only text content, but
> also face differences when computing how to refresh the screen.
>
> The immediate motivation for this patch is to provide visual feedback
> to users after a bracketed paste operation, but an active region
> concept could be useful for all kinds of things, including shift-arrow
> selection or integration with xterm mouse facilities.
>
> The mark automatically deactivates after most commands, including C-g
> and cursor motion. We do keep any active mark active across screen
> refresh operations, however.
>
> exchange-point-and-mark activates the mark just like in Emacs.
> ---
>  lib/readline/display.c   | 478 +++
>  lib/readline/kill.c  |   6 +-
>  lib/readline/readline.c  |  19 ++
>  lib/readline/readline.h  |   4 +
>  lib/readline/rlprivate.h |   5 +-
>  lib/readline/terminal.c  |  43 
>  lib/readline/text.c  |  55 +++--
>  lib/readline/util.c  |   1 +
>  8 files changed, 398 insertions(+), 213 deletions(-)
>
> diff --git a/lib/readline/display.c b/lib/readline/display.c
> index 2d2e768a..f228a39e 100644
> --- a/lib/readline/display.c
> +++ b/lib/readline/display.c
> @@ -63,13 +63,13 @@
>  extern char *strchr (), *strrchr ();
>  #endif /* !strchr && !__STDC__ */
>
> -static void update_line PARAMS((char *, char *, int, int, int, int));
> +static void update_line PARAMS((char *, char *, char *, char *, int, int,
> int, int));
>  static void space_to_eol PARAMS((int));
>  static void delete_chars PARAMS((int));
> -static void insert_some_chars PARAMS((char *, int, int));
>  static void open_some_spaces PARAMS((int));
>  static void cr PARAMS((void));
>  static void redraw_prompt PARAMS((char *));
> +static void _rl_move_cursor_relative PARAMS((int, const char *, const
> char *));
>
>  /* Values for FLAGS */
>  #define PMT_MULTILINE  0x01
> @@ -80,6 +80,7 @@ static char *expand_prompt PARAMS((char *, int, int *,
> int *, int *, int *));
>  struct line_state
>{
>  char *line;
> +char *line_face;
>  int *lbreaks;
>  int lbsize;
>  #if defined (HANDLE_MULTIBYTE)
> @@ -102,7 +103,9 @@ static int line_structures_initialized = 0;
>  #define vis_lbsize (line_state_visible->lbsize)
>
>  #define visible_line   (line_state_visible->line)
> +#define visible_face (line_state_visible->line_face)
>  #define invisible_line (line_state_invisible->line)
> +#define invisible_face (line_state_invisible->line_face)
>
>  #if defined (HANDLE_MULTIBYTE)
>  static int _rl_col_width PARAMS((const char *, int, int, int));
> @@ -123,7 +126,10 @@ static int _rl_col_width PARAMS((const char *, int,
> int, int));
> to use prompt_last_invisible directly. */
>  #define PROMPT_ENDING_INDEX \
>((MB_CUR_MAX > 1 && rl_byte_oriented == 0) ? prompt_physical_chars :
> prompt_last_invisible+1)
> -
> +
> +#define FACE_NORMAL '0'
> +#define FACE_STANDOUT '1'
> +#define FACE_INVALID ((char)1)
>
>  /*  */
>  /* */
> @@ -208,8 +214,8 @@ static int msg_bufsiz = 0;
>  /* Non-zero forces the redisplay even if we thought it was unnecessary. */
>  static int forced_display;
>
> -/* Default and initial buffer size.  Can grow. */
> -static int line_size = 1024;
> +/* Line buffer size. */
> +static int line_size;
>
>  /* Variables to keep track of the expanded prompt string, which may
> include invisible characters. */
> @@ -521,6 +527,64 @@ rl_expand_prompt (prompt)
>  }
>  }
>
> +static void
> +ensure_line_size (minsize)
> + int minsize;
> +{
> +  int minimum_size = 1024;
> +  int new_line_size, delta;
> +  if (minsize < minimum_size)
> +minsize = minimum_size;
> +  if (line_size >= minsize)
> +return;
> +  if (!new_line_size)
> +new_line_size = minimum_size;
> +  while (new_line_size < minsize)
> +new_line_size *= 2;
> +  visible_line = (char *)xrealloc (visible_line, new_line_size);
> +  visible_face = (char *)xrealloc (visible_face, new_line_size);
> +  invisible_line = (char *)xrealloc (invisible_line, new_line_size);
> +  invisible_face = (char *)xrealloc (inv

Re: Error message garbage when parameter expansion used inside (()) and variable unset

[Daniel Mills]

On Mon, Apr 2, 2018 at 5:16 PM, PRussell 
wrote:

>
> echo 4B
>   ( set -x;var=5;var1=var; (( var1 == $var2 )) && echo yes || echo no )
>
>
> It appears that 3A and 4A evaluate to 0 because of the arithmetic context.
> 3A echo's yes; 4A echo's no.
>
> The problem is what is happening with 3B and 4B.  I tested on bash 4.3.11
> and
> bash 4.4.19 and got a slightly different error message.
>
> Bash version 4.3.11:
>
> ./tt: line 18: var1: var1 ==  : syntax error: operand expected (error
> token is "==  ")
>
> Bash version 4.4.19 ( was garabage):
>
> ./tt: line 18: : var1 ==  : syntax error: operand expected (error
> token is "==  ")
>
> Peggy Russell
>
>
The problem here is quite simple. When using "$", the parameter is expanded
first. Since
it's empty, it expands to the empty string, and the (( keyword simply sees
(( var1 == )),
which is absolutely a syntax error. When not using "$", no expansions
happen before the
(( keyword sees the variable name, and so it uses the name correctly. So
the fix here is
simple: provide variable names without using "$" in an arithmetic context.

Re: [PATCH] Add active mark, face support; activate mark on paste

[Daniel Colascione]

Hey. I'd appreciate a quick peek at this patch. The lack of visual feedback
on paste is leading people to turn off bracketed paste mode, which is
unfortunate.

On Mon, Mar 19, 2018 at 11:06 AM, Chet Ramey  wrote:

> On 3/19/18 1:25 PM, Daniel Colascione wrote:
> > Ping
>
> I haven't looked at it closely yet; trying to get bugs and tests done
> before starting the 5.0-alpha release cycle.
>
> Chet
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>

INT still propagating to child processes when trapped

[Daniel Mills]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu'
-DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash'
-DSHELL -DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib  -D_FORTIFY_SOURCE=2
-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt
-DDEFAULT_PATH_VALUE='/usr/local/sbin:/usr/local/bin:/usr/bin'
-DSTANDARD_UTILS_PATH='/usr/bin' -DSYS_BASHRC='/etc/bash.bashrc'
-DSYS_BASH_LOGOUT='/etc/bash.bash_logout' -DNON_INTERACTIVE_LOGIN_SHELLS
-Wno-parentheses -Wno-format-security
uname output: Linux djdev 4.17.5-1-ARCH #1 SMP PREEMPT Sun Jul 8 17:27:31
UTC 2018 x86_64 GNU/Linux
Machine Type: x86_64-unknown-linux-gnu

Bash Version: 4.4
Patch Level: 23
Release Status: release

Description:
 After an INT trap, forked processes are still sent INT when ^C is used on
a running script.

Repeat-By:

#!/bin/bash
{
  trap 'echo "foo :("' EXIT
  sleep 30
  echo foo
} & foo=$!
{
  trap 'echo "bar :("' EXIT
  sleep 32
  echo bar
} & bar=$!
trap 'kill "$foo"' INT
read
sleep 60
###

Run this script in bash 3.x and hit ^C, and you get "foo :(", as expected.
Run it with bash 4.4, and you also get "bar :(". I would not expect this.

Re: Reading from a file by using its FD returns its contents only once

[Daniel Mills]

On Sun, Dec 30, 2018 at 8:37 PM mike b  wrote:

> Configuration Information:
> Machine: x86_64
> OS: linux-gnu
> Compiler: gcc
> Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
> -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu'
> -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL
> -DHAVE_CONFIG_H   -I.  -I../. -I.././include -I.././lib  -Wdate-time
> -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/bash-7fckc0/bash-4.4=.
> -fstack-protector-strong -Wformat -Werror=format-security -Wall -no-pie
> -Wno-parentheses -Wno-format-security
> uname output: Linux debian9 4.9.0-8-amd64 #1 SMP Debian 4.9.130-2
> (2018-10-27) x86_64 GNU/Linux
> Machine Type: x86_64-pc-linux-gnu
>
> Bash Version: 4.4
> Patch Level: 12
> Release Status: release
>
> I am not quite sure if this is a bug, but here's what I find as a bit odd
> behavior:
>
> # modprobe zram num_devices=0
> # exec {add} # read -r id <&"$add"; echo "$id"
> 0
> # read -r id <&"$add"; echo "$id" # <- $id ends up empty, no data is read
>
> # read -r id  1
> # read -r id  2
> # readlink -f "/proc/$$/fd/$add"
> /sys/class/zram-control/hot_add
>
> The above sysfs interface is used for creating a zram device by performing
> a read on the hot_add file. The value that should be returned is the id of
> the newly created device. In first instance file is opened by dynamically
> allocating the fd to use "$add" (the fd) across reads instead of
> referencing the file directly. But from the above example you can see that
> $id is assigned an actual value only on first read. On every next one, $id
> would become empty - when using $add, that is. However, when file is read
> in a standard way, by using it directly, everything works as it should.
>
> The above is just an example. Doing reads on any other regular file like
> this yields same behavior:
> # echo bla >./t
> # exec 10<./t
> # read -r <&10
> # echo $REPLY
> bla
> # read -r <&10
> # echo $REPLY
>
> #
> Playing with something like:
> # zram=/sys/class/zram/control/hot_add
> # c=0; while  ((++c <= 3)); do read -r; echo "${REPLY:-NULL}"; done
> <"$zram"
> 0
> NULL
> NULL
> #
> also gets same results. In contrast to:
> # c=0; while (( ++c <= 3 )); do read -r; echo "out: ${REPLY:-NULL}"; done
> foo
> out: foo
> foo
> out: foo
> foo
> out: foo
> #
> which keeps reading from default stdin (terminal in this case) without any
> hiccups (as expected).
>
> So, considering all the above, any hints on why subsequent reads in these
> scenarios don't get any data from a file would be really appreciated. :)
>
> Same behavior is seen in 4.3.30 and 5.0 versions.
>

Redirections treat all files as streams, it won't seek back to the
beginning once you hit the end of the file. You would need to close and
re-open it

Re: [PATCH] Add active mark, face support; activate mark on paste

[Daniel Colascione]

Any chance we can revive this patch now that Bash 5 is out? (The patch
itself, of course, would need to be rebased.)

On Fri, Jun 8, 2018 at 9:27 AM Chet Ramey  wrote:
>
> On 6/7/18 10:45 PM, Daniel Colascione wrote:
> > Hey. I'd appreciate a quick peek at this patch. The lack of visual feedback
> > on paste is leading people to turn off bracketed paste mode, which is
> > unfortunate.
>
> I've looked at it briefly. I have not had time to put it into the devel
> branch yet; probably after bash-5.0 is released.
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [PATCH] Add active mark, face support; activate mark on paste

[Daniel Colascione]

On Wed, Jan 9, 2019 at 12:37 PM Chet Ramey  wrote:
>
> On 1/9/19 2:39 PM, Daniel Colascione wrote:
> > Any chance we can revive this patch now that Bash 5 is out? (The patch
> > itself, of course, would need to be rebased.)
>
> Yes, I plan to.

Have you had a chance to look at the patch?

Re: [PATCH] Add active mark, face support; activate mark on paste

[Daniel Colascione]

On Mon, Sep 23, 2019 at 6:36 AM Chet Ramey  wrote:
>
> On 9/23/19 7:32 AM, Daniel Colascione wrote:
> > On Wed, Jan 9, 2019 at 12:37 PM Chet Ramey  wrote:
> >>
> >> On 1/9/19 2:39 PM, Daniel Colascione wrote:
> >>> Any chance we can revive this patch now that Bash 5 is out? (The patch
> >>> itself, of course, would need to be rebased.)
> >>
> >> Yes, I plan to.
> >
> > Have you had a chance to look at the patch?
>
> Sorry, I have not. I need a bigger chunk of time than I've had available
> to figure out the intent of the patches, and whether they're a good idea
> to include in readline, before looking closely at the code.

I've had a people tell me that this patch is very useful for helping
them understand editing operations and pasting. I sent this patch over
a year and a half ago. Should I expect another year and a half of
pinging this thread? :-)

Bug in Bash Version 5.0.11(1)-release

[Daniel Hannon]

the use of the "…" character results in the ability to delete the bash prompt 
string using backspace

[FR] save command times and exit status in history automatically

[Daniel Colascione]

Right now, bash history saves only the command line actually executed.
Why not also, optionally, save command execution times and exit
statuses? This information is practically free to collect.

Re: [FR] save command times and exit status in history automatically

[Daniel Colascione]

On Thu, Nov 7, 2019 at 12:09 PM Chet Ramey  wrote:
>
> On 11/5/19 12:49 PM, Daniel Colascione wrote:
> > Right now, bash history saves only the command line actually executed.
>
> This isn't quite the case. What it saves is the line returned from
> readline, before it's expanded or executed.

Fair enough.

> > Why not also, optionally, save command execution times and exit
> > statuses? This information is practically free to collect.
>
> Because by the time you gather this information, the command has already
> been saved completely.
>
> There have been various proposals to extend the timestamp with additional
> information, but it's all data you can gather when the timestamp is saved
> before the command is executed.

That's how history works today, yes. I'm wondering whether it'd be
possible to maintain an auxiliary history that included not only the
command lines read, but also their execution time and outcome. Doing
something in PROMPT_COMMAND doesn't seem quite accurate. Maybe what I
really want is something like $?, but instead of containing exit
status, it would contain information from a struct rusage (derived
from wait4) for the last command. Or something like that anyway. The
larger point is that the shell could gather this information
proactively almost for free and I don't think user code running in
existing shell extension points can do the same job.

Re: Request For Enhancement - TID variable

[Daniel Colascione]

On 12/26/19 7:37 AM, Eric S. Raymond wrote:

In attempting to use GNU parallel, with some bash scripts, I
discovered I had a problem with tempfile collisions due to
all of the thread having the same PID.

I was able to come up with a workaround, but...

RFE: bash should have a TID varuable that returns the vakue of gettid(2).

If the bash devs are too busy for this like the idea, I could write
the patch.


Are you sure that'd help? Parallel runs bash in a bunch of subprocesses, 
so looking at PID would suffice to distinguish jobs. Are you sure you 
weren't seeing an invariant PID because you were letting the PID 
variable expansion happen too early?

Re: [PATCH] Add active mark, face support; activate mark on paste

[Daniel Colascione]

On Tue, Sep 24, 2019 at 10:20 PM Daniel Colascione  wrote:
>
> On Mon, Sep 23, 2019 at 6:36 AM Chet Ramey  wrote:
> >
> > On 9/23/19 7:32 AM, Daniel Colascione wrote:
> > > On Wed, Jan 9, 2019 at 12:37 PM Chet Ramey  wrote:
> > >>
> > >> On 1/9/19 2:39 PM, Daniel Colascione wrote:
> > >>> Any chance we can revive this patch now that Bash 5 is out? (The patch
> > >>> itself, of course, would need to be rebased.)
> > >>
> > >> Yes, I plan to.
> > >
> > > Have you had a chance to look at the patch?
> >
> > Sorry, I have not. I need a bigger chunk of time than I've had available
> > to figure out the intent of the patches, and whether they're a good idea
> > to include in readline, before looking closely at the code.
>
> I've had a people tell me that this patch is very useful for helping
> them understand editing operations and pasting. I sent this patch over
> a year and a half ago. Should I expect another year and a half of
> pinging this thread? :-)

Is there anything I can do to get this patch looked at?

Re: "wait" loses signals

[Daniel Colascione]

> There are lots of programming languages around, they each have their
> particular
> niche - the reason their inventors created them in the first place.  Use
> an
> appropriate one, rather than attempting to shoehorn some feature that is
> needed
> into a language that was never intended for it - just because you happen
> to
> be a big fan of that language.   Spread your wings, learn a new one

That is a poor excuse for not fixing bugs. Maybe you can torture the
standards into confessing that this behavior isn't a bug. This behavior
nevertheless surprises people and nevertheless precludes various things
people want to do with a shell. Don't you think it's better that programs
work reliably than that they don't? Of course something working
intuitively 99.9% of the time and hanging 0.1% of the time is a bug. It's
not appropriate to treat that 0.1% hang as some kind of cosmic punishment
for using shell in a manner you find inappropriate: remember when we
believed in mechanism, not policy? Nor is the presence of the bug in other
shells adequate justification for leaving this one in a bad state. I've
never understood the philosophy of people who want to leave bugs unfixed.

No, it's not that much trouble to fix the bug. The techniques for fixing
this kind of signal race are well-known. In particular, instead of
waitpid, you use a self-pipe and signal the pipe in the signal handler,
and you have a signal handler for SIGCHLD. If we had a pwaitpid (like
pselect) we could use that too. If I could get Chet to look at my patches,
I'd fix it myself.

Re: "wait" loses signals

[Daniel Colascione]

> Date:Mon, 24 Feb 2020 04:58:31 -0800
> From:    "Daniel Colascione" 
> Message-ID:  <07d1441d41280e6f9535048d6485.squir...@dancol.org>
>
>   | That is a poor excuse for not fixing bugs.
>
> Only if they are bugs.

That executing traps except in case you lose one rare race is painfully
obvious.

>
>   | Maybe you can torture the standards into confessing that this
>   | behavior isn't a bug.
>
> No torture required.  Once again, the standard documents the way users
> can expect shells to behave.

I refuse to let the standard cap the quality of a shell's implementation.
Missing signals this way is pure negative. It doesn't add to any
capability or help any user. It can only make computing unreliable and
hurt real users trying to automate things with shell.

> That is what a standard is - a common set
> of agreed operations

A standard is a bare minimum.

> attempt to get shells to change this way of
> working, and if you can get a suitable set to agree, and implement
> something
> new, that more meets your needs, then perhaps that might one day become
> the
> standard,

This opposition to doing more than the bare minimum that the standard
requires makes this task all the much harder.

>   | This behavior nevertheless surprises people
>
> Lots of things surprise people.

Sometimes people deserve to be surprised. This isn't one of those times.

>   | and nevertheless precludes various things
>   | people want to do with a shell.
>
> That was my point, that you just labelled a poor excuse.   Not everything
> is suitable for implementation in sh.   Sometimes you simply have to go
> elsewhere.

Making people go elsewhere *on purpose* by refusing to fix bugs is not
good software engineering.

> Wanting to do it in shell doesn't make it reasonable or
> possible.

It is reasonable and possible. All that's needed is to make an existing
operation that's almost perfectly reliable in fact perfectly reliable, and
as I've mentioned, it's not that hard.

> I want the shell to feed my dog, where is the dogfood option?

We're talking about fixing an existing shell feature, not adding a new one.

>   | Don't you think it's better that programs
>   | work reliably than that they don't?
>
> Yes, when they are written correctly.

By fixing this bug, we make a class of programs correct automatically.

>
>   | Of course something working intuitively 99.9% of the time and
>   | hanging 0.1% of the time is a bug.
>
> Nonsense.   An alternative explanation is that your intuition is wrong,
> and that it often works that way is just by chance.

We're talking about a documented feature that users expect to work a
certain way and that almost always *does* work that way and that diverges
from this behavior only under rare circumstances. Not the same as spacebar
heating.

> The program is
> broken because it is making unjustified assumptions about how things are
> specified to work.

This moralistic outlook is not helpful. It doesn't *matter* whether a
program is right or wrong or making unjustified assumptions or not.
Punishing programs does not make the world does not make the world better.
When a piece of infrastructure can transform these programs from incorrect
to correct at next to zero cost, it behooves that infrastructure component
to do that.

> This is the kind of common error that people who
> program (in any language) by guesswork often make "I saw Fred did this,
> and I tried it, and it worked for me like I thought it would, so it
> must do this similar thing like I think it will too".   Rubbish.

Ever hear of the "pit of success"? It's the idea that software gets better
when you make the intuitive thing happen to be the correct thing. Why
should we require a degree of cleverness greater than what a domain
requires? Why *not* make it so that, to the greatest extent possible,
shouldn't we let "I saw Fred do this" lead people to good patterns? Like I
said before, making things difficult on purpose doesn't actually achieve
anything.

[1] https://docs.microsoft.com/en-us/archive/blogs/brada/the-pit-of-success

>
>   | I've never understood the philosophy of people who want to leave
>   | bugs unfixed.
>
> Nor have I, except sometimes perhaps when it comes to costs.   But the
> issue here is whether this is a bug.  Your belief that it is does not make
> it so.

Your belief that this behavior is acceptable doesn't make it so --- except
under a pointlessly literal interpretation of the standards.

>   | No, it's not that much trouble to fix the bug.
>
> It isn't, if it needs fixing - but any fix for this will slow the shell
> (for what that matters, but some people care)

Re: "wait" loses signals

[Daniel Colascione]

> Date:Mon, 24 Feb 2020 06:44:12 -0800
> From:    "Daniel Colascione" 
> Message-ID:  
>
>   | That executing traps except in case you lose one rare race is
> painfully
>   | obvious.
>
> Maybe you misunderstand the issue, no traps are lost, if they were
> that would indeed be a bug, the trap will always be executed in the
> cases in question, the only issue is when that happens.

They're not executed before the wait as is supposed to happen though, so
we can hang when we shouldn't.

>   | This opposition to doing more than the bare minimum that the standard
>   | requires makes this task all the much harder.
>
> I am not at all opposed to doing more than the standard requires, the
> shell I maintain does more (not nearly as many addons as bash, but
> considerably more than bash - and in some areas we're ahead, we already
> have a wait command where it is possible to wait for any one of a set
> of processes (or jobs) and be told which one completed, for example).
>
> I'm also not opposed to doing less when the standard is nonsense, which
> it is in a couple of places.
>
> But "I want x" or "I think it should be y" aren't good enough reasons to
> change something, and making the shell useful for (very primitive) IPC
> isn't a good reason for making updates.

Yes, it is, because people find this style of IPC useful today, and it's
worthwhile to make this use reliable.

>   | Making people go elsewhere *on purpose* by refusing to fix bugs is not
>   | good software engineering.
>
> Of course.   I don't see a bug.

You can interpret any random bit of brokenness as a feature. Whether the
behavior is a "bug" or not is irrelevant: bash _should_ be handling these
traps as early as possible, because that simplifies the programming model
without hurting anything else.

>   | We're talking about fixing an existing shell feature, not adding a new
> one.
>
> OK, here's an alternative, I want the shell to be able to do arithmetic on
> arbitrarily large (and small) numbers.   All that is needed to fix it is
> to link in the bignum library and use it (and extend the parser a little
> to
> handle real numbers).

This situation is more like bash supporting arbitrary-precision addition
and giving the wrong answer when the number is prime. "Oh, we never
promised support for _prime_ sums. It's not a bug. It's just a thing the
shell doesn't do."


>   | This moralistic outlook is not helpful. It doesn't *matter* whether a
>   | program is right or wrong or making unjustified assumptions or not.
>
> That is unbelievable.   That is all that matters.   If the program is
> wrong, the program needs to be fixed, not the world altered so that the
> program suddely works.

You want to increase the number of correct programs in the world.
Sometimes the fix is to declare incorrect programs broken and have people
fix them. Other times, in situations like this one, it's better to just
change the infrastructure so that the program is correct.

>
>   | Punishing programs does not make the world does not make the world
> better.
>
> It does.   The bad ones fail, and are replaced by better ones.

Computer security was even more of a horrible nightmare than it is today
back when people had this attitude. "Why should we use stack hardening? If
a program writes beyond the end of an array, that's a bug in the program."
Nice sentiment. Doesn't work.

Re: [PATCH] Add active mark, face support; activate mark on paste

[Daniel Colascione]

On Fri, Jan 10, 2020 at 5:34 AM Chet Ramey  wrote:
>
> On 1/8/20 2:38 PM, Daniel Colascione wrote:
> > On Tue, Sep 24, 2019 at 10:20 PM Daniel Colascione  
> > wrote:
> >>
> >> On Mon, Sep 23, 2019 at 6:36 AM Chet Ramey  wrote:
> >>>
> >>> On 9/23/19 7:32 AM, Daniel Colascione wrote:
> >>>> On Wed, Jan 9, 2019 at 12:37 PM Chet Ramey  wrote:
> >>>>>
> >>>>> On 1/9/19 2:39 PM, Daniel Colascione wrote:
> >>>>>> Any chance we can revive this patch now that Bash 5 is out? (The patch
> >>>>>> itself, of course, would need to be rebased.)
> >>>>>
> >>>>> Yes, I plan to.
> >>>>
> >>>> Have you had a chance to look at the patch?
> >>>
> >>> Sorry, I have not. I need a bigger chunk of time than I've had available
> >>> to figure out the intent of the patches, and whether they're a good idea
> >>> to include in readline, before looking closely at the code.
> >>
> >> I've had a people tell me that this patch is very useful for helping
> >> them understand editing operations and pasting. I sent this patch over
> >> a year and a half ago. Should I expect another year and a half of
> >> pinging this thread? :-)
> >
> > Is there anything I can do to get this patch looked at?
>
> I haven't taken the time to sit down and look at it.  That's my bad.

Ping? Anything I can do to help?

Re: [PATCH] Add active mark, face support; activate mark on paste

[Daniel Colascione]

> On 3/25/20 1:14 PM, Daniel Colascione wrote:
>
>> Ping? Anything I can do to help?
>
> OK, I sat down and looked at this code, since I'm homebound. I added the
> active mark/region features (rl_activate_mark/rl_deactivate_mark/etc.)
> and a couple of the smaller pieces (_rl_cr, the so/se sequences).

Thanks!

> Can you
> tell me why you decided to make the region management functions public?
> Did you anticipate external application functions wanting to manage the
> region?

I was imagining applications wanting to highlight certain regions, e.g., a
shell showing which command in a pipeline failed. You can also imagine a
readline user supporting something like xterm-mouse-mode. Making the
region functions public isn't essential right now though.

> I haven't touched the face code in display.c. I'd like to find a simpler
> way to do it: the patch seems to have a lot of overhead and adds more
> complexity than I'd like at a time when I'm trying to make the redisplay
> code simpler. I don't know of a better way to do that yet.

The redisplay code needs to track the intended attribute state of each
character in the buffer. I'm not sure what simpler approach might be
viable. Fat characters? You'd still have to support the old char*
interface, and you'd still need something like puts_face to "propertize"
any strings we add to the buffer. A face system preserves the existing
format of the buffer at least, and the redisplay update code is a logical
extension of the current diffing logic.

Re: Are there any plans for more readable, modern syntaxes for If statements?

[Daniel Colascione]

> I keep on wondering why these shells can't come up with something better
> than improving previous shell syntax by little by only providing poor
> better alternatives.
> I somehow think there is a need to rethink shells from scratch to make
> them
> less mentally demanding and readable in the command prompt and scripts
> themselves.
> I really haven't seen anything that I would like to use so far. Neither
> zsh, tcsh or any other of the more popular ones as far as I remember from
> the last weeks "personal research".
>

fish shell and powershell are two reasonable attempts. I think current
shell syntax is fine though.

Re: [PATCH] Add active mark, face support; activate mark on paste

[Daniel Colascione]

On 4/12/20 6:23 PM, Chet Ramey wrote:

On 4/12/20 2:15 PM, gentoo_esh...@tutanota.com wrote:


There is one more/different 'face' issue: if I paste a line and then press 
Enter (as opposed to any alphanumeric key or arrow keys) then the highlight 
remains(highlighted), possibly because the ^M is echoed and thus moves the 
cursor one line up(?) before the highlight is attempted to be removed. But I'm 
just guessing.


Unsurprising. The highlights are added and removed in readline's redisplay.
Once you enter newline (or any key bound to accept-line), readline returns
the line immediately without any redisplay, so the line remains as is.


Is that a regression relative to my original patch? I could have sworn I 
made command submission deactivate the mark and redisplay, but maybe I'm 
recalling incorrectly. In any case, isn't that the right thing to do?


FWIW, for debugging the kinds of issues we're discussing here, rr(1) is 
_incredibly_ helpful.

Re: [PATCH] Add active mark, face support; activate mark on paste

[Daniel Colascione]

> On 2020-04-11 at 18:04 +0200, gentoo_eshoes wrote:
>> $ echo ${PS1@A}
>> declare -x PS1=$'\\\n---\\n\\\n\\[\\a\\]\\\n\\[\\e[1;37m\
>> \e[42m\\]\\u@\\H\\[\\e[0m\\] \\\n\\[\\033[1;30m\\]$(date "+%Y/%m/%d %
>> H:%M:%S")\\[\\033[0m\\] \\\n\\[\\e[0;37m\\]\\s\\V t:\\l j:\\j \\\nd:
>> ${SHLVL} pp:${PPID} p:$$ ut`cat /proc/uptime | cut -f1 -d.`\\[\\e[0m\
>> \]\\n\\\n\\[\\e[0;37m\\]!\\!\\[\\e[0m\\] \\\n\\[\\033[0;36m\\]\\#\\[\
>> \033[0m\\] \\\n$(evalexitcode "${__earlyec[@]}" ) \\\n\\[\\e[0m\
>> \]$(uname -r) $(uname -v)\n$(ps_lepath "\\w")\\[ \\033];\\w\\a\\]\n\
>> \[\\e[1;32m\\]\\$\\[\\e[0m\\] \\\n'
>
>
> That was an… 'interesting' prompt.
>
> Note that there are several subprocesses that you could easily avoid:
>
> $(date "+%Y/%m/%d %H:%M:%S") is equivalent to \D{%Y/%m/%d %H:%M:%S}
>
>
> `cat /proc/uptime | cut -f1 -d.`  this could be simplified to `cut -f1 -d.
> /proc/uptime`
> it may be replaced with just builtins by `uptime=$( echo ${uptime/.*}`
>
> $(uname -r) $(uname -v)  is equivalent to $(uname -r -v)  I wonder why you
> need these fairly static values on every prompt line, though.

More generally, a loadable module command can do whatever you want, and
that's going to be more efficient than any subprocess fork and exec.

nameref pointing to array element

[Daniel Molina]

Hello,

Is the following an intended behavior? Undefined? A bug?

$ a[1]=2; declare -n ref=a[1]; echo $ref $((ref))
2 0

Thanks,

Daniel

Re: Proposed new feature for bash: unbuffered pipes

[Daniel Colascione]

On 4/23/20 4:39 PM, Dale R. Worley wrote:

Andreas Schwab  writes:

See stdbuf(1).


The limitation(s) of stdbuf are:  It can only be applied to the standard
I/O streams.  It doesn't affect statically-linked executables.  It only
applies to a single executable, so if the command is a shell function or
script, or creates subprocesses, it doesn't affect them.


If you want to define an environment variable for affecting stdout 
buffering, you can. (Good luck getting any libc to implement the support 
you're imagining: libc authors are *very* conservative about 
everything.) But whether or not you have this environment variable, you 
don't need special bash syntax to set it: you can just prefix the 
command you want with an environment variable assignment. I don't think 
the need you're discussing is widespread enough to warrant further 
complicating shell syntax.


A much simpler option might be just convincing libc people to make 
line-buffering the default whether isatty or not. Machines are very fast 
these days, and any program that _knows_ it's emitting non-line-oriented 
data can opt into full buffering.

Re: Proposed new feature for bash: unbuffered pipes

[Daniel Colascione]

On April 22, 2020 6:32:07 PM wor...@alum.mit.edu (Dale R. Worley) wrote:


The crux of the problem, IMHO, is to look at it from the right angle:

Occasionally, the user desires that I/O through certain pipes should be
unbuffered, that is, the stdio stream(s) that write into the pipe should
be unbuffered, rather than the default block-buffered.  These are
situations where the user can tell that it is desirable to sacrifice
efficiency to gain low latency, whereas the writer of the program that
the user is running could not predict the needs of this particular
invocation.  (This is where Ulrich Drepper was incorrect; the writer of
the program doesn't know when this is desirable.)

This facility is best activated by a feature in bash, as the need arises
from the relationship between two processes.

The ideal way to implement this would be for the kernel to provide two
flavors of pipe, one designated buffered and one designated unbuffered,
and according to the user's instructions, bash would allocate an
unbuffered pipe when it was needed.  The kernel itself would treat the
two types of pipe in the same way, but stdio would create buffered
streams for fd's that were opened on buffered pipes and it would create
unbuffered streams for fd's that were opened on unbuffered pipes.

However, getting a coordinated change into the kernel seems like it
would be quite difficult.

So an alternative implementation is to have a way for bash to notify
processes it creates that certain pipes (identified by their dev/ino
numbers) should have streams that are unbuffered.  Pretty much the only
way to do this is to have a conventional environment variable for
communicating this information.

Of course, either of these solutions requires changes to glibc, but
then, if you're going to modify buffering behavior of streams, you are
going to have to modify glibc.


Given the current political realities in the relevant FOSS ecosystem, it 
might be easier to add a new pipe flavor or attribute to the kernel and 
then get libc implementations to support it as a fait accompli than to do 
something purely in libc. Also, an environment variable is semantically 
wrong: its scope is too  broad, well beyond the specific pipe you care 
about. Imagine the variable leaking into a daemon that lives forever.

apparent inconsistencies in readline documentation

[Daniel Molina]

Hi,

I found some aspects of readline documentation that seem inconsistent to
me and I wanted to share them.

1. The difference between backward-kill-line and unix-line-discard
readline commands.

Documentation states:

   backward-kill-line (C-x Rubout)
  Kill backward to the beginning of the line.

   unix-line-discard (C-u)
  Kill backward from point to the  beginning  of  the 
line.   The
  killed text is saved on the kill-ring.

In both cases they kill from the point and killed text is saved in the
kill-ring.

2. Default key sequences vs. emacs key bindings [the default].

It is confusing to me that there are two defaults. Firstly, it can be read:

EDITING COMMANDS
   The  following  is  a list of the names of the commands and
the default
   key sequences to which they are bound.  Command names without
an accom‐
   panying key sequence are unbound by default.

On the other hand, emacs editing command are default:

readline  offers  editing  capabilities  while the user is entering the
   line.  By default, the line editing commands are similar  to 
those  of
   emacs.  A vi-style line editing interface is also available.

An explicit list of emacs commands is maintained and commands do not
always coincide (both being valid defaults in practice). For example,
instead of C-x Rubout for backward-kill-line, emacs has

 "C-XC-?"  backward-kill-line

3. Key-bindings in the emacs/vi list are written with capital letters
(C-A), but not in the section with the description (C-a).

Best regards,

Daniel

Re: apparent inconsistencies in readline documentation

[Daniel Molina]

On 27/7/20 22:09, Chet Ramey wrote:
> On 7/25/20 12:21 PM, Daniel Molina wrote:
>> Hi,
>>
>> I found some aspects of readline documentation that seem inconsistent to
>> me and I wanted to share them.
>>
>> 1. The difference between backward-kill-line and unix-line-discard
>> readline commands.
>>
>> Documentation states:
>>
>>    backward-kill-line (C-x Rubout)
>>   Kill backward to the beginning of the line.
>>
>>    unix-line-discard (C-u)
>>   Kill backward from point to the  beginning  of  the 
>> line.   The
>>   killed text is saved on the kill-ring.
>>
>> In both cases they kill from the point and killed text is saved in the
>> kill-ring.
> The difference is what happens with numeric arguments.


I see.


>  Maybe that is what
> should be added to the backward-kill-line description.

I think that would be useful for that an similar commands. I found the
question asked on the web too.


>> 2. Default key sequences vs. emacs key bindings [the default].
>>
>> It is confusing to me that there are two defaults. Firstly, it can be read:
>>
>> EDITING COMMANDS
>>    The  following  is  a list of the names of the commands and
>> the default
>>    key sequences to which they are bound.  Command names without
>> an accom‐
>>    panying key sequence are unbound by default.
>>
>> On the other hand, emacs editing command are default:
>>
>> readline  offers  editing  capabilities  while the user is entering the
>>    line.  By default, the line editing commands are similar  to 
>> those  of
>>    emacs.  A vi-style line editing interface is also available.
>>
>> An explicit list of emacs commands is maintained and commands do not
>> always coincide (both being valid defaults in practice). For example,
>> instead of C-x Rubout for backward-kill-line, emacs has
>>
>>  "C-XC-?"  backward-kill-line
> Rubout/DEL[ete]/C-? are the same key.

I see.


>> 3. Key-bindings in the emacs/vi list are written with capital letters
>> (C-A), but not in the section with the description (C-a).
> It's a writing convention. The behavior doesn't differ. Are there places
> where this convention is used inconsistently?
>
Convention is not followed strictly in the same document doc/readline.0,
but I suppose that it is intentional to keep a different convention in
the DEFAULT KEY BINDINGS (emacs and vi list) where uppercase is used,
and the rest of the document, where mainly lower case is used with very
few exceptions:

In SEARCHING:

If that variable has not been assigned a value the Escape  and
C-J characters will terminate an incremental search.  C-G will abort an
incremental search and restore the original line.

In INITIALIZATION FILE -> Variables

   isearch-terminators (``C-[ C-J'')
  The string of characters that should  terminate  an 
incremental
  search  without  subsequently  executing the character as
a com‐
  mand.  If this variable has not been given a value, the 
charac‐
  ters ESC and C-J will terminate an incremental search.

Maybe a confusing part is Section "1.3.3 Sample Init File" of
doc/readline.info where there are lines like

 # Arrow keys in keypad mode
         #
 #"\M-OD":    backward-char


 #
 # Arrow keys in ANSI mode
 #
 "\M-[D":    backward-char

and

"\C-xr": redraw-current-line

but in that case maybe it is not a convention but a parsing rule of
INITRC (?), I do not understand well the conventions in that section.

Regards,

Daniel

Re: execve E2BIG (Argument list too long)

[Daniel Colascione]

On September 30, 2020 8:24:01 AM Ilkka Virta  wrote:


On Wed, Sep 30, 2020 at 5:53 PM Michael Green  wrote:


The included short script when run with the following command results
in execve "E2BIG (Argument list too long) errors".

* The number of arguments can be tuned down to "seq 1 23694" and it
still occurs, but any lower and it disappears.


That sounds a lot like the 128 kB hard limit Linux puts on the size of a
single argument to exec. (I know it counts for command line arguments, I
expect it also counts for env variables. They're passed in pretty much the
same way anyway.)



It might be worth asking lkml to lift this restriction



seq 1 23694 | wc  gives 131058, just a bit less than 131072. Add the
variable name and it goes over. Workaround: use another OS, or pass big
data like that in files.

Re: execve E2BIG (Argument list too long)

[Daniel Colascione]

On September 30, 2020 8:44:40 AM Andreas Schwab  wrote:


On Sep 30 2020, Daniel Colascione wrote:


It might be worth asking lkml to lift this restriction


You will have bad luck with that.  The limit has been introduced for
security reasons.


What "security reasons"? You'd still be subject to the stack ulimit, which 
you can set to whatever you want.

Subject: Pressing Ctrl-C during any subshell evaluation terminates the shell

[Daniel Farina]

Configuration Informatio:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -O2 -g -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions
-fstack-protector-strong -grecord-gcc-switches
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
-Wno-parentheses -Wno-format-security
uname output: Linux shrike 5.8.9-200.fc32.x86_64 #1 SMP Mon Sep 14 18:28:45
UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-redhat-linux-gnu

Bash Version: 5.0
Patch Level: 17
Release Status: release

Description:

Pressing Ctrl-C during any subshell evaluation terminates the shell.  I
noticed this when using direnv and emacs together: Ctrl-C would not cancel
a subprocess started by the shell, but would exit the entire shell.

Relaying this from: https://github.com/direnv/direnv/issues/627

Repeat-By:

Per https://github.com/direnv/direnv/issues/627#issuecomment-635611930

$ cat bash.rc
eval "$(direnv hook bash)"

$ bash --rcfile bash.rc
bash$ echo $PROMPT_COMMAND
_direnv_hook
bash$ $(sleep 10) # pressing ^C during those 10 seconds will terminate the
shell
^C
$ # inner shell terminated

Fix:

No known good fix.  It does seem zsh manages the situation normally without
much difference in approach. Direnv 2.20.0 does not have this bug, but it
also has deficits in how it traps signals.

Re: Subject: Pressing Ctrl-C during any subshell evaluation terminates the shell

[Daniel Farina]

On Sun, Oct 11, 2020 at 12:59 PM Chet Ramey  wrote:
>
> On 10/9/20 7:23 PM, Daniel Farina wrote:
>
> > Bash Version: 5.0
> > Patch Level: 17
> > Release Status: release
> >
> > Description:
> >
> > Pressing Ctrl-C during any subshell evaluation terminates the shell.  I
> > noticed this when using direnv and emacs together: Ctrl-C would not cancel
> > a subprocess started by the shell, but would exit the entire shell.
> >
> > Relaying this from: https://github.com/direnv/direnv/issues/627
> >
> > Repeat-By:
> >
> > Per https://github.com/direnv/direnv/issues/627#issuecomment-635611930
> >
> > $ cat bash.rc
> > eval "$(direnv hook bash)"
>
> What commands does this `direnv' command output?

The source for it is at
https://github.com/direnv/direnv/blob/a4632773637ee1a6b08fa81043cacd24ea941489/shell_bash.go#L10-L20

And reads:

_direnv_hook() {
  local previous_exit_status=$?;
  trap -- '' SIGINT;
  eval "$("{{.SelfPath}}" export bash)";
  trap - SIGINT;
  return $previous_exit_status;
};
if ! [[ "${PROMPT_COMMAND:-}" =~ _direnv_hook ]]; then
  PROMPT_COMMAND="_direnv_hook${PROMPT_COMMAND:+;$PROMPT_COMMAND}"
fi

.selfPath is templated. This is embedded in a Go source file.

The version that does not exit looks like this. It is the version I use daily.

_direnv_hook() {
  local previous_exit_status=$?;
  eval "$("{{.SelfPath}}" export bash)";
  return $previous_exit_status;
};
if ! [[ "${PROMPT_COMMAND:-}" =~ _direnv_hook ]]; then
  PROMPT_COMMAND="_direnv_hook${PROMPT_COMMAND:+;$PROMPT_COMMAND}"
fi

https://github.com/direnv/direnv/blob/e2a7d51f31abf36d7983309a14f5036ea8aa6da2/shell_bash.go#L10-L17

> >
> > $ bash --rcfile bash.rc
> > bash$ echo $PROMPT_COMMAND
> > _direnv_hook
>
> What does `direnv_hook' do?
>
> > bash$ $(sleep 10) # pressing ^C during those 10 seconds will terminate the
> > shell
> > ^C
> > $ # inner shell terminated
>
> My guess is that it messes with the controlling terminal somehow, causing
> the shell to get EOF or -1/EIO when it attempts to read input.

direnv's purpose in life is to execute subshells that are then typed into.

Re: Possible race condition in bash

[Daniel Colascione]

On 11/21/20 1:15 PM, Chet Ramey wrote:

On 11/21/20 2:32 PM, Andreas Schwab wrote:

On Nov 21 2020, Chet Ramey wrote:


but since the shell always ignores SIGTERM,


Even a non-interactive shell?


No, you're right, it's SIGQUIT the shell always ignores. It catches SIGTERM
if there is an EXIT trap so it can run the trap on EXIT before exiting.


I'd also try it with one of the bash-5.1 testing releases, since I did
some reworking of how the shell handles SIGTERM back in April.


5.1-rc3 has the same issue.


OK, I instrumented bash a little bit, and discovered that the second
child doesn't start running, or at least doesn't get to the point in the
subshell initialization where it resets the signal handlers, until after
the parent sends SIGTERM.

That means the trap handler is still set to catch SIGTERM by the time
the signal arrives. However, the child is not allowed to run the trap
handler; it's supposed to reset the dispositions to the default. This
is the race condition.

It's a dilemma. I could block SIGTERM (all trapped signals, really) until
the child resets the signal dispositions, but that seems like a long time
to keep signals blocked. I'll look at it some more after bash-5.1 is
released.


Plenty of other programs keep signals blocked until reset. I don't see 
the problem with the delay: if the SIGTERM is ignored, the delay between 
SIGTERM and kill is infinite. :-) Since the delay between process-kill 
and actual process death can be arbitrarily long anyway (that's the 
contract with the kernel), I don't see a problem with the blocking.

Re: Feature Request: scanf-like parsing

[Daniel Colascione]

On 1/22/21 9:54 AM, Saint Michael wrote:

I vote for this new feature.


Personally, I've found that scanf underpowered for parsing modern data 
formats. Bash's existing regular expression support seems perfectly 
adequate to me, and it can handle everything that scanf can. I'd only 
suggest extending the regular expression syntax with support for named 
subgroups.

Re: Feature Request: scanf-like parsing

[Daniel Colascione]

On 1/25/21 11:59 AM, Greg Wooledge wrote:


On Mon, Jan 25, 2021 at 06:47:36PM +0200, Oğuz wrote:

25 Ocak 2021 Pazartesi tarihinde Chet Ramey  yazdı:

declare -A copy
eval copy=( "${assoc[@]@K}" )

So many reputable people contributed to the demonization of `eval' that I
don't think I can convince anyone that there's nothing wrong with it
anymore.

It's a tricky thing to deal with.  Eli referenced my wiki, which has a
page dedicated to it, with contributions from many different authors.
The resulting quasi-consensus is complex and perhaps even a little
bit self-contradictory as a result.

You'll want to use eval only when it's absolutely necessary, and only when
it's safe.  If bash's @K feature is designed to be fed to eval, then we
can assume it's safe.  It becomes one of the very small number of
green-lighted cases where eval is OK.

The problem with eval is that for every OK usage, there are a thousand
incorrect and dangerous uses.  Avoid those, by being absolutely sure
you know what you're doing, and why you're doing it.



And it's also worth systematically creating alternatives for all the 
safe and legitimate uses of "eval" so that eventually the whole 
mechanism can be discouraged without having to rely on the nuance you've 
just described.

Behaviour of test -v with assoc array and quote character in key

[Daniel Gröber]

Hi list,

I've found what I belive to be a bug in how `test -v` expands the key in an
associative array. The following minimal test case demonstrates the
problem:

declare -A array

mytest () {
array["$1"]=123
test -v array["$1"]
printf 'test -v array[%s]; $?=%s\n' "$1" "$?"
}

mytest 'a' # prints: test -v array[a]; $?=0
mytest '"' # prints: test -v array["]; $?=1
echo "keys: ${!array[*]}" # prints: keys: " a

With a regular alphanumeric key `test -v` will return zero, indicating the
array element exists. However if we specify a variable that expands to
something containing the double quote character the test doesn't succeeed
indicating the key doesn't exist though it does as we can observe by the
expansion of ${!array[*]}.

Do you think this intended behaviour or a real bug?

Tested on Debian buster, with bash_5.0-4 ($BASH_VERSION="5.0.3(1)-release")
and git master commit 76404c85d492c001f59f2644074333ffb7608532.

Thanks,
Daniel

PS: Please keep me in CC as I'm not subscribed.

[PATCH] bracketed paste support

[Daniel Colascione]

This patch adds support for "bracketed paste mode" to readline. In
this mode, readline instructs the terminal to wrap pasted strings in
special control sequences so that programs can distinguish them from
typed input. This patch makes readline insert each pasted string as
one big literal into the edit buffer; as there is no termcap
capability for bracketed paste support, the patch detects capable
terminals automatically by looking for support for similar escape
sequences.

diff --git a/lib/readline/bind.c b/lib/readline/bind.c
index 8acf4ac..7173552 100644
--- a/lib/readline/bind.c
+++ b/lib/readline/bind.c
@@ -1486,6 +1486,7 @@ static const struct {
   { "convert-meta",&_rl_convert_meta_chars_to_ascii, 0 },
   { "disable-completion",  &rl_inhibit_completion, 0 },
   { "echo-control-characters", &_rl_echo_control_chars,0 },
+  { "enable-bracketed-paste",   &_rl_enable_bracketed_paste,0 },
   { "enable-keypad",   &_rl_enable_keypad, 0 },
   { "enable-meta-key", &_rl_enable_meta,   0 },
   { "expand-tilde",&rl_complete_with_tilde_expansion, 0 },
diff --git a/lib/readline/doc/readline.3 b/lib/readline/doc/readline.3
index ca0a81a..808b4b7 100644
--- a/lib/readline/doc/readline.3
+++ b/lib/readline/doc/readline.3
@@ -578,6 +578,13 @@ following the cursor are not duplicated.
 If set to \fBOn\fP, a character denoting a file's type as reported  
 by \fIstat\fP(2) is appended to the filename when listing possible
 completions.
+.TP
+.B enable\-bracketed\-paste (On)
+If set to \fBOn\fP and the terminal supports bracketed
+paste mode, configure it to insert each paste into the
+editing buffer as a string instead of treating
+the characters pasted as normal input, preventing inadvertent
+execution of pasted commands.
 .PD
 .SS Conditional Constructs
 .PP
diff --git a/lib/readline/doc/rluser.texi b/lib/readline/doc/rluser.texi
index 577..0a26444 100644
--- a/lib/readline/doc/rluser.texi
+++ b/lib/readline/doc/rluser.texi
@@ -703,6 +703,13 @@ If set to @samp{on}, a character denoting a file's type
 is appended to the filename when listing possible
 completions.  The default is @samp{off}.
 
+@item enable-bracketed-paste
+@vindex enable-bracketed-paste
+If set to @samp{on} and the terminal supports bracketed paste mode,
+configure it to insert each paste into the editing buffer as a string
+instead of treating the characters pasted as normal input, preventing
+inadvertent execution of pasted commands.  The default is @samp{on}.
+
 @end table
 
 @item Key Bindings
diff --git a/lib/readline/funmap.c b/lib/readline/funmap.c
index 363507b..29f089b 100644
--- a/lib/readline/funmap.c
+++ b/lib/readline/funmap.c
@@ -101,6 +101,7 @@ static const FUNMAP default_funmap[] = {
   { "history-substring-search-backward", rl_history_substr_search_backward },
   { "history-substring-search-forward", rl_history_substr_search_forward },
   { "insert-comment", rl_insert_comment },
+  { "bracketed-paste-begin", rl_bracketed_paste_begin },
   { "insert-completions", rl_insert_completions },
   { "kill-whole-line", rl_kill_full_line },
   { "kill-line", rl_kill_line },
diff --git a/lib/readline/kill.c b/lib/readline/kill.c
index 3d23745..79de79f 100644
--- a/lib/readline/kill.c
+++ b/lib/readline/kill.c
@@ -656,6 +656,61 @@ rl_yank_last_arg (count, key)
   return retval;
 }
 
+/* We've recognized a terminal control sequence telling us to expected
+ * pasted content.  Read characters from the terminal until we read a
+ * bracketed paste end sequence, treating the characters read as
+ * literal text to insert.  */
+int
+rl_bracketed_paste_begin (count, key)
+ int count, key;
+{
+  static const char endseq[] = "\033[201~";
+  static const int endseqlen = sizeof (endseq) - 1;
+
+  int rv = -1;
+  int c;
+  size_t len = 0;
+  size_t cap = 16;
+  char *buf = xmalloc (cap);
+
+  RL_SETSTATE(RL_STATE_MOREINPUT);
+  while (0 <= (c = rl_read_key ()))
+{
+  if (RL_ISSTATE (RL_STATE_MACRODEF))
+_rl_add_macro_char (c);
+
+  if (c == '\r')
+c = '\n';
+
+  if (len == cap)
+buf = xrealloc (buf, (cap *= 2));
+
+  buf[len++] = c;
+  if (len >= endseqlen &&
+  !strncmp (&buf[len - endseqlen],
+endseq,
+endseqlen))
+{
+  len -= endseqlen;
+  break;
+}
+}
+  RL_UNSETSTATE(RL_STATE_MOREINPUT);
+
+  if (c >= 0)
+{
+  if (len == cap)
+buf = xrealloc (buf, (cap += 1));
+
+  buf[len++] = '\0';
+  rl_insert_text (buf);
+  rv = 0;
+}
+
+  xfree (buf);
+  return rv;
+}
+
 /* A special paste command for users of Cygnus's cygwin32. */
 #if defined (__CYGWIN__)
 #include 
diff --git a/lib/readline/readline.c b/lib/readline/readline.c
index f8211e7..d6e9775 100644
--- a/lib/readline/readline.c
+++ b/lib/readline/readline.c
@@ -302,6 +302,11 @@ int _rl_revert_all_at_newline = 0;
characters corresponding to keyboard-g

Re: [PATCH] bracketed paste support

[Daniel Colascione]

On 10/29/2014 09:35 PM, Pádraig Brady wrote:
> On 10/27/2014 10:35 PM, Daniel Colascione wrote:
> 
>> +@item enable-bracketed-paste
>> +@vindex enable-bracketed-paste
>> +If set to @samp{on} and the terminal supports bracketed paste mode,
>> +configure it to insert each paste into the editing buffer as a string
>> +instead of treating the characters pasted as normal input, preventing
>> +inadvertent execution of pasted commands.  The default is @samp{on}.
> 
> I see this defaults on.
> Does this mean one can't paste command sequences to readline now?

Well, I don't know whether Chet left the feature enabled by default. I hope he 
did, though, since preventing execution of pasted commands is one of the 
feature's key benefits. In bash, you should be able to execute a pasted command 
sequence by typing RET after the paste, but a paste by itself should never 
begin execution.

For better or worse, people copy and paste commands from websites all the time. 
Even if a bit of shell looks innocuous, a malicious bit of JavaScript could 
change the pasted text at the last second without the user being aware of the 
switch. (Tynt uses this technique to slightly less malicious ends.) If pasting 
in a terminal immediately begins execution, there's no opportunity to review 
pasted code. With bracketed paste support, the shell requires additional user 
interaction after a paste to begin execution, making this attack much less 
effective.



signature.asc
Description: OpenPGP digital signature

Re: [PATCH] bracketed paste support

[Daniel Colascione]

On 10/30/2014 06:05 AM, Bob Proulx wrote:
> Daniel Colascione wrote:
>> Well, I don't know whether Chet left the feature enabled by
>> default. I hope he did, though, since preventing execution of pasted
>> commands is one of the feature's key benefits. In bash, you should
>> be able to execute a pasted command sequence by typing RET after the
>> paste, but a paste by itself should never begin execution.
> 
> I use paste into the shell with an embedded newline in order to
> immediately execute a command *a lot*.  If that were removed I would
> be very unhappy.

I strongly doubt that your use case is typical. I've asked several of my
colleagues; all have complained about accidentally pasting a large
amount of text into the shell at one time or another. Nobody has
complained about losing automatic execution of code after paste.
Speaking from personal experience, I've been using terminal emulators of
various sorts for almost 20 years, and in that time, I've accidentally
pasted documents into my terminal orders of magnitude more often than
I've deliberately pasted multi-command sequences.

As far as I'm concerned, automatic execution of code on paste isn't a
feature: it's a bug and security hole. Users should have to opt into
security holes. We've only lived with the existing behavior so long
because it's only recently become possible to distinguish pastes from
other input.

> However I read from Chet's replies that this is not the case.  So I am
> not worried.  Just voicing a counter opinion that this shouldn't be
> removed.

Nobody is talking about removing the ability to select the present
behavior. We're talking about the default setting.

> If you want it then you should enable it.  This is like many
> other options available in the shell.  They are optional.

This feature significantly decreases the likelihood of user error, but
if it's not on by default, most users won't enable it and won't see any
benefits. I'd rather slightly power users like you with very specific
use cases than continue to see accidental code execution.

>> For better or worse, people copy and paste commands from websites
>> all the time. Even if a bit of shell looks innocuous, a malicious
>> bit of JavaScript could change the pasted text at the last second
>> without the user being aware of the switch. 
> 
> Then the browser shouldn't copy it.  The place to fix the problem is
> in the browser.  

Browser behavior is unlikely to change; even if it did, sequences of
text that look safe can contain control characters like TAB that invoke
various shell commands. Visually inspection is not a reliable way to
inspect a piece of code pasted from an otherwise-untrusted source.

Sure, you might argue that users should paste into a trusted
intermediate location --- say a text editor --- inspect the code, and
then paste into the shell. That would be the prudent thing to do, but
users don't actually *do* that and never will.

> A problem with a browser should not break cutting and
> pasting in general.

This change doesn't "break" pasting. It changes its behavior to one
that's safer, more natural, and more consistent with that of other programs.

>> (Tynt uses this technique to slightly less malicious ends.) If
>> pasting in a terminal immediately begins execution, there's no
>> opportunity to review pasted code. With bracketed paste support, the
>> shell requires additional user interaction after a paste to begin
>> execution, making this attack much less effective.
> 
> I am very happy this is a feature you like.  However I would hate that
> feature.  It should definitely not default to on or it will make
> people like me very unhappy.

You can disable the setting yourself. If you don't want to do that, all
you need to do is type RET after pasting. That's a very small tradeoff
for an increase in safety and predictability.



signature.asc
Description: OpenPGP digital signature

Re: [PATCH] bracketed paste support

[Daniel Colascione]

On 10/29/2014 08:49 PM, Chet Ramey wrote:
>> On 10/27/14, 6:35 PM, Daniel Colascione wrote:
>>> This patch adds support for "bracketed paste mode" to readline. In
>>> this mode, readline instructs the terminal to wrap pasted strings in
>>> special control sequences so that programs can distinguish them from
>>> typed input. 
>>
>> Thanks for the contribution.  I'll look at the code; the approach seems
>> sound.
> 
> It went in very easily, though I changed some things around.  This
> will be in the next release of bash and readline.

The code appeared in the snapshot, so I was able to look at how you
"changed some things around". You removed the code that tries to
determine whether a terminal actually supports the feature; instead, you
blindly send the enable sequence to *any* terminal when
enable-bracketed-paste is enabled, with potentially unknown consequences
on those terminals.

There's a comment in the code that indicates you expect users to enable
the feature only on terminals where it's supported. I don't think it's
reasonable to expect people to maintain a terminal database in their
inputrc files.

You also know very well that people are going to just set the feature to
enabled in their configurations, test it, and see it work, all without
considering whether they should enable this feature in a
terminal-type-dependent conditional. Then they'll wonder why parts of
their system silently break.

Please restore the part of my patch,
rl_bracketed_paste_probably_supported, that detects terminal support for
this feature.



signature.asc
Description: OpenPGP digital signature

Re: [PATCH] bracketed paste support

[Daniel Colascione]

On 11/06/2014 02:38 AM, Chet Ramey wrote:
> On 11/5/14 9:02 PM, Daniel Colascione wrote:
>> On 10/29/2014 08:49 PM, Chet Ramey wrote:
>>>> On 10/27/14, 6:35 PM, Daniel Colascione wrote:
>>>>> This patch adds support for "bracketed paste mode" to readline. In
>>>>> this mode, readline instructs the terminal to wrap pasted strings in
>>>>> special control sequences so that programs can distinguish them from
>>>>> typed input.
>>>>
>>>> Thanks for the contribution.  I'll look at the code; the approach seems
>>>> sound.
>>>
>>> It went in very easily, though I changed some things around.  This
>>> will be in the next release of bash and readline.
> 
>> The code appeared in the snapshot, so I was able to look at how you
>> "changed some things around". You removed the code that tries to
>> determine whether a terminal actually supports the feature; instead, you
>> blindly send the enable sequence to *any* terminal when
>> enable-bracketed-paste is enabled, with potentially unknown consequences
>> on those terminals.
> 
>> There's a comment in the code that indicates you expect users to enable
>> the feature only on terminals where it's supported. I don't think it's
>> reasonable to expect people to maintain a terminal database in their
>> inputrc files.
> 
> People rarely use more than one, maybe two, different terminals or
> terminal emulators.  I don't doubt that people who enable this feature
> know enough to figure out whether or not it's going to work.

You're assuming users are experts.  In reality, users are going to just
paste "set enable-bracketed-paste on" into their inputrc files; they'll
then see an inscrutable control sequence appear in places they didn't
expect and have no idea what's wrong with their system.

Please optimize for what real users actually do, not what perfectly
intelligent users with perfect understanding of their systems would
ideally do.

I'm already going to have to contact every readline and bash distributor
on earth and ask them to enable this feature by default. I don't want to
have to ask them to apply a code patch too.

>> Please restore the part of my patch,
>> rl_bracketed_paste_probably_supported, that detects terminal support for
>> this feature.
> 
> It doesn't `detect terminal support' as such.  It uses a heuristic to
> guess whether or not a terminal supports bracketed paste by checking for
> the presence of  a string in another termcap sequence.  Those kinds of
> heuristics always end up having an exception.

I don't think they will in this case, and at least it's a *conservative*
heuristic. All terminals I've tested (urxvt, xterm, linux, and vte)
either fail the heuristic, pass the heuristic and ignore the control
sequence (which is harmless), or pass the heuristic and enable bracketed
paste mode.

Terminal emulation at this point is very mature. I don't expect
significant development of new terminal types and new kinds of terminal
emulators. The heuristic should work well for a long time.



signature.asc
Description: OpenPGP digital signature

Re: pipefail with SIGPIPE/EPIPE

[Daniel Colascione]

On 02/15/2015 01:48 PM, Chet Ramey wrote:
> On 2/13/15 12:19 PM, Pádraig Brady wrote:
>> I was expecting bash to handle SIGPIPE specially here,
>> as in this context it's informational rather than an indication of error.
> 
> I don't agree.  It's a fatal signal whose default disposition is to
> terminate a process, which is exactly what happens in your example.

The purpose of pipefail is to make the shell indicate when something has
gone wrong anywhere in a pipeline. For most programs, SIGPIPE does not
indicate that something went wrong. Instead, SIGPIPE is expected
behavior. When pipefail spuriously reports expected behavior as an
error, Bash comes less useful.

> You might consider trapping or ignoring SIGPIPE in situations where it
> might be an issue.

If I were emperor of the world, I would make SIGPIPE's SIG_DFL action
terminate the process with exit status 0. But POSIX says we can't do
that. Even locally, I make my system do that without kernel surgery.
It's also not reasonable to modify every program that might be part of a
pipeline so that it exits successfully on EPIPE.

Making Bash treat SIGPIPE death as success is the next best option.



signature.asc
Description: OpenPGP digital signature

'[ --version' should give output, instead a bash error missing re: missing ']'

[Daniel Simeone]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu'
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL
-DHAVE_CONFIG_H   -I.  -I../. -I.././include -I.././lib
-D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat
-Werror=format-security -Wall
uname output: Linux simeone-thinkpad 3.19.0-23-generic #24-Ubuntu SMP Tue
Jul 7 18:52:55 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 4.3
Patch Level: 30
Release Status: release

Description:
According to the joint man page for '[' and 'test', '[ --version'
and '[ --help' should give appropriate output, while 'test' should not.
'test' appears to behave correctly, but when running '[ --version' or '[
--help', the following error appers:
bash: [: missing `]'

It appears that whatever code is checking for syntax failed to
take  the two options that may be passed to '[' into account.


Repeat-By:
[ --help
OUTPUT: bash: [: missing `]'

Re: '[ --version' should give output, instead a bash error missing re: missing ']'

[Daniel Simeone]

You are correct:

/usr/bin/\[ --help
gives the desired output.

 When I ran 'which ['  it stated that the /usr/bin/[ was what was running,
and so I presumed it was in bash.

Also, the error message says 'bash'

So, all in all, a bit counfusing.

Cheers,
Daniel



On 22 September 2015 at 10:39, Eric Blake  wrote:

> On 09/22/2015 08:22 AM, Daniel Simeone wrote:
> >
> > Description:
> > According to the joint man page for '[' and 'test', '[ --version'
> > and '[ --help' should give appropriate output, while 'test' should not.
>
> You're probably reading the coreutils man page, rather than the bash man
> page.
>
> Bash has not (yet) implemented support for ANY --options to its
> builtins, although there has been talk on the list of doing so for
> future versions (especially since ksh has already done it).  If you are
> executing the shell builtins instead of the coreutils versions, then the
> behavior you see is expected and not a bug.
>
> To see the behavior mentioned in the coreutils man page, be sure you run
> the coreutils version of [, as in:
>
> env [ --help
> or
> /bin/[ --help
>
> --
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
>


-- 
--Daniel Simeone
PhD candidate,
Candidat au doctorat,
Université McGill University

readline and bash disagree on $'' quoting

[Daniel Colascione]

Consider this command:

  foo $'foo \' bar'

As far as the bash core is concerned, this command has one argument
word. But readline, for completion, splits it up into three words:

  0: foo
  1: $'abc \'
  2: bar'

Shouldn't we be splitting the command line into the same number of words?



signature.asc
Description: OpenPGP digital signature

Re: forked before bash subshell

[Daniel Colascione]

On 10/13/2016 08:05 PM, Bob Proulx wrote:

XiaoBing Jiang wrote:

Greg Wooledge wrote:

If you want to ENSURE that the child shell process is replaced by the
external sleep 20, use an explicit exec.


yes, I want to know why bash not optimize this. or any strategy ?


Because it wouldn't save anything significant.  Since the parent shell
is bash the extra bash in memory is already in memory.  A forked
process uses almost the same OS memory as the original.  The OS will
share the code pages between the two processes at the OS memory page
level.  Data pages will split when they are individually modified.
Even if you use 'exec' to replace the forked and backgrounded copy
another one is still in memory as the script runs.  Only unique data
pages will go through the copy-on-write process as the two bash
processes diverge.  Therefore even if you *think* you are saving
something by optimizing it out the actual savings is insignificantly
small.

Plus this is really only a problem if the main part of the script
exits leaving the forked subshell running in the background.  However
that is not a good thing to do.  If you want to leave the process
running that is a daemon process.  Launching a daemon process needs
more setup than this to avoid other problems.  Since leaving processes
behind like that is a bad thing to do it again doesn't make much sense
to try to optimize the doing of it.

It is possible to contrive some examples where it makes sense to do
this optimization.  But mostly they are very unusual cases.  Not
typical cases.


One such case is Cygwin --- I'm not sure how "contrived" it is. Cygwin 
has an old-fashioned non-COW fork, and to add insult to injury, process 
creation generally is very slow (~100ms). It pays to eliminate subshells 
in that environment.

Re: forked before bash subshell

[Daniel Colascione]

On 10/15/2016 12:23 AM, L. A. Walsh wrote:



Daniel Colascione wrote:

One such case is Cygwin --- I'm not sure how "contrived" it is. Cygwin
has an old-fashioned non-COW fork, and to add insult to injury,
process creation generally is very slow (~100ms). It pays to eliminate
subshells in that environment.

Given what Cygwin has to work around in Windows -- it is very contrived.


Maybe the internals. From the perspective of programs like bash, Cygwin 
is a mostly ordinary POSIX environment, one with lots of users and that 
deserves full support.



MS has copy-on-write available to MS processes, that it's not usable in
Cygwin isn't surprising given that most of Windows is closed-source.


It has nothing to do with being closed source and everything to do with 
the win32 subsystem (under which Cygwin runs) not being designed to cope 
with process duplication.




But if you really want bash on your windows, use a native copy.  It's
not as likely to have the same problems
(https://techcrunch.com/2016/03/30/be-very-afraid-hell-has-frozen-over-bash-is-coming-to-windows-10/)



It doesn't do the same thing. Cygwin is an integrated POSIXish environment.



Apparently there is a Windows Subsystem for Linux that is currently in Beta
for Win10:
http://www.howtogeek.com/265900/everything-you-can-do-with-windows-10s-new-bash-shell/



SFU and similar do not interact with the win32 subsystem as closely as 
Cygwin and so can't provide the same experience. IMHO, you might as well 
run a VM.

Problem outputting a quote in a variable expansion which is in a here doc

[Daniel Einspanjer]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: darwin15.6.0
Compiler: clang
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='darwin15.6.0' -DCONF_MACHTYPE='x86_64-apple-darwin15.6.0'
-DCONF_VENDOR='apple'
-DLOCALEDIR='/usr/local/Cellar/bash/4.4.5/share/locale' -DPACKAGE='bash'
-DSHELL -DHAVE_CONFIG_H -DMACOSX   -I.  -I. -I./include -I./lib
-I./lib/intl -I/private/tmp/bash-20161112-61463-r79i9i/bash-4.4/lib/intl
 -DSSH_SOURCE_BASHRC -Wno-parentheses -Wno-format-security
uname output: Darwin den 15.6.0 Darwin Kernel Version 15.6.0: Thu Sep  1
15:01:16 PDT 2016; root:xnu-3248.60.11~2/RELEASE_X86_64 x86_64
Machine Type: x86_64-apple-darwin15.6.0

Bash Version: 4.4
Patch Level: 5
Release Status: release

Description:
Tested on OSX as well as Ubuntu GNU/Linux 14.4 and Debian Jessie 8
Tested with Bash 4.3 and 4.4

When trying to output a string from a variable expansion that is inside
a here doc, the quotes don't behave the same as when not in a here doc.

Repeat-By:
Here is a small test script that demonstrates the problem:

#!/bin/bash

export VARIABLE="test test"

echo "Trying to quote inside a variable expansion inside a here document"
echo
echo The desired output is:
echo "var = \"test test\""
echo
echo "First off, taking here docs out of the equation, unescaped quotes
don't work:"
echo ${VARIABLE:+var = "$VARIABLE"}
echo
echo But escaping the quotes does work:
echo ${VARIABLE:+var = \"$VARIABLE\"}

echo
echo But things get weird when the variable expansion is inside a here doc:

cat << EOF
Plain quotes don't work:
echo ${VARIABLE:+var = "$VARIABLE"}

Escaped quotes don't work:
echo ${VARIABLE:+var = \"$VARIABLE\"}

Double quotes within single quotes don't work:
echo ${VARIABLE:+'var = "$VARIABLE"'}

Even trying to single quote just the double quote is a no-go:
echo ${VARIABLE:+var = '"'$VARIABLE'"'}

I tried a couple of variants of \x22 and $'\x22', but I wasn't sure about
the syntax.

Here is the only thing I have found so far that does work:
${VARIABLE:+var = $(echo '"')$VARIABLE$(echo '"')}

EOF


-Daniel

Re: Problem outputting a quote in a variable expansion which is in a here doc

[Daniel Einspanjer]

Awesome, thanks for this tip!

On Wed, Dec 7, 2016 at 8:44 AM Greg Wooledge  wrote:

> On Wed, Dec 07, 2016 at 12:56:00AM +0000, Daniel Einspanjer wrote:
> > echo The desired output is:
> > echo "var = \"test test\""
>
> imadev:~$ q=\"
> imadev:~$ var="a variable"
> imadev:~$ cat < > ${var:+var = $q$var$q}
> > EOF
> var = "a variable"
> imadev:~$ echo "$BASH_VERSION"
> 4.4.0(1)-release
>
> Putting literal quotes (to be echoed) inside variables helps work around
> a fairly large number of tricky situations like this.  It's a useful
> general-purpose hack to keep in the back of your mind for the future.
>

bash: /home/greenhornet/.bashrc: line 120: syntax error: unexpected end of file

[Daniel Bullard]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu'
-DCONF_VENDOR='pc' -DL$
uname output: Linux udontknowme 4.4.0-59-generic #80-Ubuntu SMP Fri Jan 6
17:47:47 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 4.3
Patch Level: 46
Release Status: release

Description: openConnection: connect: No such file or directory
cannot connect to brltty at :0
upstart: indicator-bluetooth main process (1573) killed by TERM signal
upstart: indicator-power main process (1577) killed by TERM signal
upstart: indicator-datetime main process (1579) killed by TERM signal
upstart: indicator-printers main process (1591) killed by TERM signal
upstart: indicator-session main process (1595) killed by TERM signal
upstart: indicator-application main process (1606) killed by TERM sign

Repeat-By:  It just started today i'm new to linux i've had the computer
for 3 days not sure what i did i'm hoping you can help. I apologize if I
haven't explained myself well enough .


these are other issues that popped up

 xsession-errors” changed on disk

.bash_history
Ubuntu
sudo pactl load-module module-bluetooth-discover
sudo taskbar
gsettings set com.canonical.Unity.Launcher launcher-position Bottom
sudo apt-get install unity-tweak-tool
Unable to lock the administration directory (/var/lib/dpkg/), is another
process using it?
sudo apt-get install unity-tweak-tool
sudo no
no
sudo lsof /var/lib/dpkg/lock
sudo apt-get update
sudo apt-get install unity-tweak-tool
sudo add-apt-repository ppa:mpstark/elementary-tweaks-daily
sudo add-apt-repository ppa:moka/daily
sudo apt-get update
sudo apt-get install moka-icon-theme faba-icon-theme faba-mono-icons
sudo apt-get install unzip;
sudo apt-get install vlc
sudo apt-get install ubuntu-restricted-extras ubuntu-restricted-addons
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo apt-get install bluez
dpkg –status bluez | grep ‘^Version:’
sudo /etc/init.d/networking restart
[ok]
hciconfig
sudo bluetooth/audio.conf
sudo bluetooth
bluetooth
pulseaudio
apt-get install pulseaudio pulseaudio-module-bluetooth pavucontrol
bluez-firmware
hcitoolscan
sudo apt-get installbluez-firmware
sudo apt-get install bluez-firmware
dmesg | grep -i blue
sudo pactl load-module module-bluetooth-discover
sudo apt-get install bluez-utils
bluez:i386 bluez
sudo apt-get install bluez:i386 bluez
remove bluez
dpkg --list
bluez:i386
sudo apt-get clean
sudo apt-get autoclean
sudo apt-get -f install
sudo apt autoremove
sudo restart
sudo reboot
lsb_release -a command
lsb_release -a
dpkg –status bluez | grep ‘^Version:’
dpkg -Dhelp
bitwise-or bluez
dpkg -Dhelp bluez
dpkg --status bluez | grep '^Version:'
dpkg bluez
dpkg -- help bluez
dpkg --help
-V bluez
-V
sudo apt install unattended-upgrades
Unattended-Upgrade::Allowed-Origins {
//  "Ubuntu xenial-updates";
};sudo Unattended-Upgrade::Allowed-Origins {
//  "Ubuntu xenial-updates";
};sudo apt install unattended-upgrades
sudo apt install apticron
EMAIL="bullard.dan...@gmail.com"
EMAIL=bullard.dan...@gmail.com
sleep
sleep --help
HandleLidSwitch=hibernate
sudo HandleLidSwitch=hibernate
uname -r
lsb_release -a
dpkg -l openssl
./configure
sudo service bluetooy
sudo service bluetooth start
sudo rfkill list
sudo hciconfg hci0 up hcitool scan
sudo apt-get install bluez-tools
bt-device -1
bt-device --help
-c 8c:04:22:BB:C851
8c:04:22:BB:C851
apt-get install showbox apk
sudo apt-get install showbox apk
8C:64:22:BB:C8:51
apt-get 8C:64:22:BB:C8:51
sudo apt-get 8C:64:22:BB:C8:51
bt-audio -c  8C:64:22:BB:C8:51
bt-device -1
bt-device --help
-c
bt-device  -c
bt-device  --c
--connect=8C:64:22:BB:C8:51
--connect= 8C:64:22:BB:C8:51
-c 8C:64:22:BB:C8:51
sudo apt-get 8C:64:22:BB:C8:51
hcitool scan
hciconfig scan
bluez-simple-agent hci0 8C:64:22:BB:C8:51
bluez-simple-agent hcio 8C:64:22:BB:C8:51
sudo apt-get install bluez-tools
apt-rdepends
sudo apt install apt-rdepends
apt- rdepends bluez
rdepends bluez
-s
rdepends --build-depends
reverse-depends bluez
rdepends
apt-rdepends pkgs
apt-rdepends pkgs bluez
apt-rdepends pkgs blueztools
apt-rdepends pkgs bluez tools
apt-cache rdepends
bluez-tools/testing
apt-bluez-tools/testing
make check
apt-make check
ls -la
dpkg --status bluez | grep '^Version:'
bluetoothd -v
sudo apt-get dist-upgrade
sudo reboot
pkgs
apt-get pkgs
lsb_release -a command
lsb_release -a
sudo dpkg -l | grep bluez
sudo dpkg -l | grep bluez tools
sudo dpkg -l | more
cat /var/log/dpkg.log | grep " install "
cat /var/log/dpkg.log | grep " install " | grep bluez
cat /var/log/dpkg.log | grep " install " | grep google
pkgdetails
alias
alias apt get= apt
.bashrc.
bash
bt-adapter
bt --list
--list
-l
bt-device
bashrc
google
chrome
getent passwd $(whoami) | awk -F: '{print $NF}'
ps -p $$ -o cmd=
/etc/skel/.bashrc
gksu gnome-terminal
sudo apt install gksu
apt-get up

Re: Is it normal for `bash -s foo` not to make 1=foo available from ~/.bashrc?

[Daniel Mills]

On Mon, Mar 27, 2017 at 8:32 AM, Torka Noda  wrote:

> For any particular reason?
>
> Why are they not all made available anyway? with an alternative
> array for the arguments sent to the commands fed to Bash stdin
> with "-s", so we don't have to handle all possible arguments if
> we just want the non-option arguments.
>
>
> There definitely are other relatively clean ways (`env` and
> '--rcfile', most notably), but using `bash -s foo bar` and
> handling the positional parameters from ~/.bashrc, would be the
> cleanest for small per-shell customizations (although it sure is
> not what '-s' is meant to be used for).
>
>
> Examples of people trying stuffs related to this:
>
> "Open gnome terminal programmatically and execute commands
> after bashrc was executed":
> https://superuser.com/questions/198015
>
> "Open gnome terminal programmatically and execute commands
> after bashrc was executed":
> http://stackoverflow.com/questions/3896882
>
> "Custom environment with gnome-terminal":
> http://askubuntu.com/questions/600139
>
> "Opening multiple tabs with gnome-terminal":
> http://askubuntu.com/questions/277543
>
>
Because you want the positional parameters set with bash -s to take
precedence over anything set in the startup files. Otherwise anything in
.bashrc would simply override what you set with bash -s.

Re: Infinite loop in bash glob matching

[Daniel Mills]

The '[:' may be messing with it, 'a[[:alpha:]:abm[]' should work. It won't
match 'amm' because the range only matches a single character, you'd need
'a+([[:alpha:]:abm[])'

On Tue, May 16, 2017 at 10:49 AM, Eduardo Bustamante 
wrote:

> On Tue, May 16, 2017 at 2:48 AM, Zoltán Herczeg 
> wrote:
> > Hi,
> >
> > bash enter an infinite loop for this glob:
> >
> > ls @(@()).
>
> It works fine for me. What version of Bash are you using? And, what
> files are in the directory you're testing in?
>
> dualbus@debian:~/t$ ls
> a.1  a.2  a.3  a.4  b.1  b.2  b.3  b.4  c.1  c.2  c.3  c.4  d.1  d.2  d.3
> d.4
> dualbus@debian:~/t$ ls @(@()).
> a.1  a.2  a.3  a.4  b.1  b.2  b.3  b.4  c.1  c.2  c.3  c.4  d.1  d.2  d.3
> d.4
> dualbus@debian:~/t$
> GNU bash, version 4.4.11(1)-release (x86_64-pc-linux-gnu)
>
>
> > I have been trying to create a bash glob regex converter. It would be
> great if somebody (privately) could explain me  how !() and invalid []
> expressions exactly work. I have questions such as:
> >
> > ls a[[:alpha:][:abm]
> >
> > Why does this match to a: and nothing else (e.g. am or a[mm )?
>
> The shell globbing library seems to be interpreting this pattern
> weirdly. I don't know the answer for this, but the glob() call in
> glibc does what I expect:
>
> dualbus@debian:~/t$ ls
> a:  aa  ab
> dualbus@debian:~/t$ ls a[[:alpha:][:abm]
> a:
> dualbus@debian:~/t$ ../glob
> a:
> aa
> ab
> dualbus@debian:~/t$ cat ../glob.c
> #include 
> #include 
> #include 
>
> int main() {
> int i;
> glob_t pglob;
> if(glob("a[[:alpha:][:abm]", 0, NULL, &pglob))
> perror(NULL);
> for(i = 0; i < pglob.gl_pathc; i++) {
> puts(pglob.gl_pathv[i]);
> }
> return 0;
> }
>
>

Re: printf fails in version 5.2.026-3

[Daniel Lublin]

Apparently the patch needed to fix this was the one Florian Weimer
posted in november 2023, in "C compatibility issue in the configure
script" <8734oqnlou@gentoo.org> (if you don't have the mail history:
https://lists.gnu.org/archive/html/bug-bash/2023-11/msg00104.html)

Maybe some compiler change triggered this now.

-- 
Daniel
lublin.se

Re: printf fails in version 5.2.026-3

[Daniel Lublin]

The message ID with the patch in question was of course
<87leasmvoo@oldenburg.str.redhat.com>

-- 
Daniel
lublin.se

Re: Env var feature request

[Daniel Colascione]

Greg Wooledge  writes:

> On Tue, Jul 09, 2024 at 20:14:27 +, Erik Keever wrote:
>> A --debug-envvars flag which will, when passed to bash, catch every
>> time an environment variable is set and print the file/line that is
>> setting it. To restrict it, "--debug-envvars FOO,BAR" to catch only
>> instances of FOO or BAR being set.
>
> It's not *exactly* what you're asking for, but you can get most of
> this by invoking bash in xtrace mode with PS4 set to a custom value:
>
> PS4='+ $BASH_SOURCE:$FUNCNAME:$LINENO:' bash -ilxc : 2>&1 | grep WHATEVER
>
> That will show you where WHATEVER is being set during an interactive
> shell login, for example.  Omit the "l" flag if you want to debug a
> non-login shell instead.
>
> Note that if bash is being run as UID 0, it will ignore PS4 coming from
> the environment, for security reasons.  So, this only works as a non-root
> user.

That's a cool trick. It should be in the manual.

Consolidate recommended options?

[Daniel Colascione]

I wonder whether it would be possible to add a simple catch-all command
that would encompass a set of options that can't be defaults but that
make writing shell scripts easier, e.g. set -euo pipefail, nullglob,
inherit_errexit, extglob, assoc_expand_once, and various other settings.

It's annoying to enable all these settings individually, but enabling
them by default would break existing scripts.  How about something like
"shopt -s bash2025" that would succinctly set these options all at once
so that script writers can more easily opt into modern behavior?

Re: Consolidate recommended options?

[Daniel Colascione]

On December 29, 2024 3:50:36 PM EST, "Lawrence Velázquez"  
wrote:
>On Sun, Dec 29, 2024, at 3:44 PM, Daniel Colascione wrote:
>> I wonder whether it would be possible to add a simple catch-all command
>> that would encompass a set of options that can't be defaults but that
>> make writing shell scripts easier, e.g. set -euo pipefail, nullglob,
>> inherit_errexit, extglob, assoc_expand_once, and various other settings.
>>
>> It's annoying to enable all these settings individually, but enabling
>> them by default would break existing scripts.  How about something like
>> "shopt -s bash2025" that would succinctly set these options all at once
>> so that script writers can more easily opt into modern behavior?
>
>No, the shell should not endorse any options like this.  The
>notion that *any* of the options you mentioned is "modern" or
>even best practice is ... contentious.
>

Are you really indifferent as to whether array indexes should be evaluated once 
or twice? Have you seen 
https://yossarian.net/til/post/some-surprising-code-execution-sources-in-bash/?

Bash has some unfortunate unfixable defaults avoidable if you know how. Don't 
you think it should be easier to avoid them? I have a hard time seeing how 
anyone would choose some of these old behaviors on purpose.

Error building mkbuiltins on ia64-hp-hpux11.23

[Daniel Richard G.]

Building bash 4.2 on an HP-UX/Itanium system:

gmake[1]: Entering directory `/tmp/bash-4.2.build/builtins'
rm -f mkbuiltins.o
cc -c  -DHAVE_CONFIG_H -DSHELL  -I. -I..  -I/tg/freeport/src/bash/bash--4.2 
-I/tg/freeport/src/bash/bash--4.2/include -I/tg/freeport/src/bash/bash--4.2/lib 
-I/tg/freeport/src/bash/bash--4.2/builtins  -DHPUX  -D_HPUX_API_LEVEL=20040821 
-D_HPUX_SOURCE  -g  /tg/freeport/src/bash/bash--4.2/builtins/mkbuiltins.c
cc  +DD64 +DO11.00 +DSblended +Olit=all -mt +w -z +p -Wl,+k +O1 +Onofltacc 
+Onolimit +Onosize  -g  -o mkbuiltins mkbuiltins.o -ldl 
ld: Can't find library or mismatched ABI for -ldl
Fatal error.
gmake[1]: *** [mkbuiltins] Error 1
gmake[1]: Leaving directory `/tmp/bash-4.2.build/builtins'
gmake: *** [builtins/builtext.h] Error 1

The attached patch (against the 4.2 source) fixes the problem for me.


--Daniel


-- 
NAME = Daniel Richard G. _\|/_Remember, skunks
MAIL = sk...@iskunk.org (/o|o\) _- don't smell bad---
MAIL+= sk...@alum.mit.edu   < (^),> it's the people who
WWW  = (not there yet!)  /   \  annoy us that do!
diff -ru bash-4.2/builtins/Makefile.in bash--4.2/builtins/Makefile.in
--- bash-4.2/builtins/Makefile.in	2010-12-21 08:37:18.0 -0500
+++ bash--4.2/builtins/Makefile.in	2011-04-20 10:51:16.0 -0400
@@ -56,7 +56,7 @@
 
 PROFILE_FLAGS = @PROFILE_FLAGS@
 CFLAGS = @CFLAGS@
-CFLAGS_FOR_BUILD = @CFLAGS_FOR_BUILD@ @CROSS_COMPILE@
+CFLAGS_FOR_BUILD = @CFLAGS_FOR_BUILD@ @CROSS_COMPILE@ $(CFLAGS)
 CPPFLAGS = @CPPFLAGS@
 CPPFLAGS_FOR_BUILD = @CPPFLAGS_FOR_BUILD@
 LOCAL_CFLAGS = @LOCAL_CFLAGS@ ${DEBUG}

Re: Error building mkbuiltins on ia64-hp-hpux11.23

[Daniel Richard G.]

On Thu, 2011 Apr 21 16:36-0400, Chet Ramey wrote:
>
> What is the part of CFLAGS that fixes this, and why not just add it to
> CFLAGS_FOR_BUILD?  CFLAGS_FOR_BUILD is for the `build tools' part of
> the builtins library.

I believe the "+DD64" bit is salient. (Generate 64-bit instead of
32-bit code.)

The problem here is that the makefile is compiling without CFLAGS
(a.k.a. with CFLAGS_FOR_BUILD), and then linking with CFLAGS. Whether
it's CFLAGS or CFLAGS_FOR_BUILD isn't the issue; it's that the makefile
is using one and then the other on the same object.

That said, I'm not sure why CFLAGS and CFLAGS_FOR_BUILD should be
different, given that this is a "simple" build where build = host =
target.


--Daniel


-- 
NAME = Daniel Richard G. _\|/_Remember, skunks
MAIL = sk...@iskunk.org (/o|o\) _- don't smell bad---
MAIL+= sk...@alum.mit.edu   < (^),> it's the people who
WWW  = (not there yet!)  /   \  annoy us that do!

Re: Error building mkbuiltins on ia64-hp-hpux11.23

[Daniel Richard G.]

On Fri, 2011 Apr 22 10:08-0400, Chet Ramey wrote:
>
> That's not what actually happens.  CCFLAGS and CCFLAGS_FOR_BUILD share
> a common set of options but differ in their use of CPPFLAGS and
> CFLAGS. The link is done with LDFLAGS_FOR_BUILD.  It's hard to tell
> where the DD64 is coming from, since it doesn't get added by anything
> in the bash configure scripts, but I suspect it gets put into
> LOCAL_DEFS somehow.

+DD64 is from CFLAGS in the environment prior to configuration.

Broadly speaking, linking needs to be done with CFLAGS + LDFLAGS (not
just LDFLAGS) precisely because of flags like this. This is already how
Automake does things; the bug comes down to bash's build system not
following existing convention. It's not reasonable to have to duplicate
CFLAGS in LDFLAGS to avoid the reported link error.


--Daniel


-- 
NAME = Daniel Richard G. _\|/_Remember, skunks
MAIL = sk...@iskunk.org (/o|o\) _- don't smell bad---
MAIL+= sk...@alum.mit.edu   < (^),> it's the people who
WWW  = (not there yet!)  /   \  annoy us that do!

"here strings" and tmpfiles

[Daniel Kahn Gillmor]

hi bash developers--

I ran the following command to get a sense of how bash deals with here
strings under the hood:

strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"'

(i'm testing with bash 5.0-2 on debian testing/unstable).

It turns out that this creates a temporary file, actually touching the
underlying filesystem:

[…]
18557 openat(AT_FDCWD, "/home/dkg/tmp/sh-thd.UCPAvB", 
O_RDWR|O_CREAT|O_EXCL, 0600) = 3
18557 fchmod(3, 0600)   = 0
18557 fcntl(3, F_SETFD, FD_CLOEXEC) = 0
18557 write(3, "hello there", 11)   = 11
18557 write(3, "\n", 1) = 1
18557 openat(AT_FDCWD, "/home/dkg/tmp/sh-thd.UCPAvB", O_RDONLY) = 4
18557 close(3)  = 0
18557 unlink("/home/dkg/tmp/sh-thd.UCPAvB") = 0
18557 fchmod(4, 0400)   = 0
18557 dup2(4, 0)= 0
18557 close(4)  = 0
18557 execve("/bin/cat", ["cat"], 0x5577ec52b8e0 /* 49 vars */) = 0
[…]

I could find no mention in the bash(1) manpage of any risk of either
here documents or here strings touching the underlying filesystem.

I know that some systems use heredocs or herestrings explicitly to avoid
things like:

 * writing to the filesystem,
 * invoking extra processes, or
 * making sensitive data avaialble to the process table.

For example, sending a password or secret key material from the
environment to stdin would be a typical way to use a herestring.

So writing this stuff to the filesystem, where it is likely to touch the
underlying disk, seems particularly problematic. And of course there is
the potential for weird race conditions around filename selection common
to all tmpfile-style shenanigans.

A few possible options for trying to improve the situation:

 a) use socketpair(2) or pipe(2) instead of making a tmpfile.  this has
the potential downside that the semantics of access to the remaining
file descriptor would be subtly different from "regular file"
semantics.

 b) On systems that support O_TMPFILE, try something like
open("/dev/shm", O_RDWR|O_CREAT|O_EXCL|O_TMPFILE).  /dev/shm tends
to be a globally-writable tmpfs, so that avoids touching any disk,
and O_TMPFILE avoids tmpfile-style race conditions.  This might need
a fallback (to the current tmpdir selection mechanics?) in case
/dev/shm isn't available.

 c) Just use O_TMPFILE with the current tmpdir selection mechanics, if
it's supported.  This isn't quite as clever as trying to use
/dev/shm first, and it won't fix the herestrings hitting the disk,
but it at least avoids tmpfile races.

 d) If none of the above can be done, at the very least, bash(1)'s
section on here docs and here strings should warn that the contents
of these documents are likely to get written to the disk
unprotected.

Does this make sense?  If this has been raised and discussed elsewhere,
please don't hesitate to point me to any archives of that discussion.

Please keep me in cc during any replies, i'm not subscribed to
bug-bash@gnu.org.

Regards,

--dkg


signature.asc
Description: PGP signature

Re: "here strings" and tmpfiles

[Daniel Kahn Gillmor]

On Mon 2019-03-18 17:18:10 -0400, Daniel Kahn Gillmor wrote:
> A few possible options for trying to improve the situation:
>
>  a) use socketpair(2) or pipe(2) instead of making a tmpfile.  this has
> the potential downside that the semantics of access to the remaining
> file descriptor would be subtly different from "regular file"
> semantics.
>
>  b) On systems that support O_TMPFILE, try something like
> open("/dev/shm", O_RDWR|O_CREAT|O_EXCL|O_TMPFILE).  /dev/shm tends
> to be a globally-writable tmpfs, so that avoids touching any disk,
> and O_TMPFILE avoids tmpfile-style race conditions.  This might need
> a fallback (to the current tmpdir selection mechanics?) in case
> /dev/shm isn't available.
>
>  c) Just use O_TMPFILE with the current tmpdir selection mechanics, if
> it's supported.  This isn't quite as clever as trying to use
> /dev/shm first, and it won't fix the herestrings hitting the disk,
> but it at least avoids tmpfile races.
>
>  d) If none of the above can be done, at the very least, bash(1)'s
> section on here docs and here strings should warn that the contents
> of these documents are likely to get written to the disk
> unprotected.

One more possibility for an implementation fix occurs to me (at least on
systems with Linux >= 3.17 and glibc >= 2.27):

 e) bash could use use memfd_create(2) for heredocs and herestrings --
that should preserve "regular file" semantics, avoid tmpfile races,
and avoid hitting the disks.

--dkg


signature.asc
Description: PGP signature

Re: "here strings" and tmpfiles

[Daniel Kahn Gillmor]

Thanks for the feedback, Eduardo--

On Mon 2019-03-18 17:40:17 -0700, Eduardo A. Bustamante López wrote:
> I don't think the implementation details of herestrings are documented 
> anywhere,
> and I'm not too sure if they should (i.e. IMO if you need that degree of 
> control
> over the implementation details, then you should use something other than
> shell).

I hear you in general -- i also don't want the documentation to be as
detailed as the source code.  But casually sending ephemeral data to
disk is a risk that i think ought to be avoided or at least avoidable.
If bash was in the habit of writing the environment to disk, i think
users would rightly complain.

> Having said that, have you tried process substitution as an option?

sure, that's an option (as long as process substitution is enabled on
the platform -- apparently that's not universal either).  Also possible
(for stdin in particular) is sending data via a pipeline using bash
builtins.  Both of these require users of bash to rewrite their scripts
though.

It seems like it'd be preferable for the shell itself to avoid these
problems automatically, at least on platforms where it's possible to do
so.  Otherwise, we *require* users of the shell to know which things are
"safe" and which things aren't before they can use the shell safely.

  --dkg

Re: "here strings" and tmpfiles

[Daniel Kahn Gillmor]

On Tue 2019-03-19 08:25:50 -0400, Greg Wooledge wrote:
> On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote:
>> strace -o tmp/bash.herestring.strace -f bash -c 'cat <<<"hello there"'
>> It turns out that this creates a temporary file, actually touching the
>> underlying filesystem:
>
> Yes, just like here documents do.  And have always done, in all shells.

Apologies for being unaware of the history.  It looks like there are a
handful of possible approaches today that minimize these fixes, which
may not have been possible on older systems, which i listed upthread.
And they work on arbitrary file descriptors, not just stdin.

Do you think that bash should not improve the situation, at least on
platforms that support these other approaches?

--dkg


signature.asc
Description: PGP signature