bug-bash@gnu.org

[Reuben]

$ echo cat /dev/stderr > bug
$ bash bug 2>&-
cat /dev/stderr

calling bash script 2>&- on linux
seems to make /dev/stderr refer to script,
though &2 seems unaffected.
using 2>&- inside script does not trigger this bug.
i assume it is a bug and not 'historical compatibility'.
the bug does not seem to appear in bash on openbsd.
the bug does not seem to appear in dash or pdksh.

---

$ bash -version
GNU bash, version 5.2.26(1)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later


This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

$ uname -srvmo
Linux 6.7.6-200.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 23 18:27:29
UTC 2024 x86_64 GNU/Linux

---

$ bash -version
GNU bash, version 5.1.4(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later


This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Linux 5.10.0-28-amd64 #1 SMP Debian 5.10.209-2 (2024-01-31) x86_64
GNU/Linux


---

Misleading text at start of manual

[Reuben Thomas]

(Checked against bash 4.2; for some reason, the manual on gnu.org is only 4.1.)

The top node, "Bash Features", says:

"The following menu breaks the features up into categories based upon which
one of these other shells inspired the feature."

But the following menu doesn't seem to bear that out. Perhaps it's
simplest just to remove this sentence, as the top-level menu is
perfectly logical and self-explanatory.

-- 
http://rrt.sc3d.org

When a hashed pathname is deleted, search PATH

[Reuben Thomas]

Tested in bash 4.3.

$ foo
... a command is run
$ hash
hits command
   0 /home/rrt/bin/foo
$ rm `which foo`
$ which foo
/usr/bin/foo
$ foo
bash: /home/rrt/bin/foo: No such file or directory

Why doesn't bash just remove the hashed path and do a normal PATH search? I
have to remove it manually.

-- 
http://rrt.sc3d.org

Re: When a hashed pathname is deleted, search PATH

[Reuben Thomas]

On 14 March 2014 18:23, Chet Ramey  wrote:

> On 3/14/14 12:11 PM, Reuben Thomas wrote:
> > Tested in bash 4.3.
> >
> > $ foo
> > ... a command is run
> > $ hash
> > hits command
> >0 /home/rrt/bin/foo
> > $ rm `which foo`
> > $ which foo
> > /usr/bin/foo
> > $ foo
> > bash: /home/rrt/bin/foo: No such file or directory
> >
> > Why doesn't bash just remove the hashed path and do a normal PATH
> search? I
> > have to remove it manually.
>
> Look at the description of the `checkhash' option to `shopt'.  It does what
> you want; it's just not the default.
>

Thanks. Why is it not the default? Shouldn't an optimisation (hashing PATH
lookup) be transparent to the user by default?

-- 
http://rrt.sc3d.org

Re: When a hashed pathname is deleted, search PATH

[Reuben Thomas]

On 15 March 2014 18:23, Chet Ramey  wrote:

> It's not been a problem, really.  The existence of the `checkhash' option
> has been enough.  How often do you remove binaries in directories in $PATH?
>

Fairly often: I frequently rename or retire scripts in my per-user bin
directory.

Since bash is happy to switch this on in POSIX mode, why not do so by
default? It's just another example of making something Just Work rather
than inflict a minor annoyance on the user which can only be resolved by
manual configuration (whose very existence escaped at least this
individual). This is precisely the sort of problem that I regularly fix in
my own systems and upstream, because in the big picture, it's just one of
many.

Is there a downside to making checkhash the default?

-- 
http://rrt.sc3d.org

Re: When a hashed pathname is deleted, search PATH

[Reuben Thomas]

On 17 March 2014 14:12, Chet Ramey  wrote:

> On 3/15/14 2:44 PM, Reuben Thomas wrote:
> > On 15 March 2014 18:23, Chet Ramey  > <mailto:chet.ra...@case.edu>> wrote:
> >
> > It's not been a problem, really.  The existence of the `checkhash'
> option
> > has been enough.  How often do you remove binaries in directories in
> $PATH?
> >
> >
> > Fairly often: I frequently rename or retire scripts in my per-user bin
> > directory.
>
> OK.  Based on the number of reports in the last few years, you are in the
> minority.
>

It's a minor annoyance, I bet a lot of users just live with it. I'm an
inveterate reporter of minor annoyances, and even I took many years to
report this problem. To my mind that doesn't indicate it shouldn't be
fixed: minor annoyances can easily be "rounded down" to zero, but the total
pain can still be large.

Only the minor performance hit it would extract on every command lookup.
>

I don't understand, surely it only has a performance impact when a hashed
file or a directory on PATH is (re)moved?

How about removing both pain *and* complexity: I'd happily prepare a patch
to make checkhash always on, remove its documentation, but of course keep
the option name supported for backwards compatibility. Then no-one ever
need think about it again.

Failing that, I'd also happily prepare a patch to simply switch the default
to on.

-- 
http://rrt.sc3d.org

Re: When a hashed pathname is deleted, search PATH

[Reuben Thomas]

On 17 March 2014 20:30, Chet Ramey  wrote:

> On 3/17/14 10:17 AM, Dave Rutherford wrote:
> > On Mon, Mar 17, 2014 at 10:12 AM, Chet Ramey 
> wrote:
> >> On 3/15/14 2:44 PM, Reuben Thomas wrote:
> >>> On 15 March 2014 18:23, Chet Ramey  >>> <mailto:chet.ra...@case.edu>> wrote:
> >>> Is there a downside to making checkhash the default?
> >>
> >> Only the minor performance hit it would extract on every command lookup.
> >
> > Only on commands that were found in the hash table, correct?
>
> Yes, of course.  I should have said every *successful* command hash table
> lookup.  This is much less of an issue now than when the option was first
> added.
>

OK, I misread the description, and thought it would only look the command
up if the execution fails. Why can't that be done?

-- 
http://rrt.sc3d.org

Re: When a hashed pathname is deleted, search PATH

[Reuben Thomas]

On 17 March 2014 20:46, Chet Ramey  wrote:

> Because the execution fails in a child process.  You'd be able to fix it
> for that process, but would do nothing about the contents of the parent
> shell's hash table.
>

Thanks for the explanation.

-- 
http://rrt.sc3d.org

Re: When a hashed pathname is deleted, search PATH

[Reuben Thomas]

On 18 March 2014 08:04, Linda Walsh  wrote:

>
>
> Chet Ramey wrote:
>
>> Because the execution fails in a child process.  You'd be able to fix it
>> for that process, but would do nothing about the contents of the parent
>> shell's hash table.
>>
>> The way the option works now is to check the hash lookups and delete
>> anything that is no longer an executable file, then redo the lookup and
>> hash the new value.
>>
> 
> Wouldn't bash notice that the child exited in <.1 seconds (
> or is it less?
>

Timing is not something you could rely on, though.

The child could report failure by e.g. writing through a pipe to the
parent, as in
http://stackoverflow.com/questions/2719499/c-signal-parent-process-from-childthereby
letting the parent update its hash table, but it's not clearly
worth the extra complexity to remove the small overhead.

Misleading syntax in manual

[Reuben Thomas]

The syntax for the "for" command is misleading, as although correct for 
bash, it is not POSIX-compliant.


I am using GNU bash, version 3.2.48(1)-release (i486-pc-linux-gnu)

The man page says:

for name [ in word ] ; do list ; done

which conflicts with the POSIX syntax definition, given in

http://www.opengroup.org/onlinepubs/95399/utilities/xcu_chap02.html#tag_02

which says:

for name [ in [word ... ]]do
compound-list


done

Note that in the POSIX definition, a semi-colon is only needed, and indeed 
permitted, after the word list when an "in" clause is used. Hence:


for i; do

is not correct in POSIX, although it is accepted by bash. (I found this 
while trying to build GNU Zile on Solaris.) bash does, of course, accept the 
POSIX syntax.


The easiest fix seems to be to put the semicolon above in square brackets, 
making it optional, though this risks giving the impression that the syntax


for i in foo bar;; do

would be acceptable, when it's not (even by bash). So, you could give two 
explicit definitions:


for i [;] do list ; done

and

for i in word ; do list ; done

--
http://rrt.sc3d.org/
My dark path darkens; rise over the mountain, distant moon (Murasaki)
Listening to: 'Sanctus and Benedictus' from 'Mass of the Children and other
sacred music' (The Cambridge Singers / John Rutter)

Please improve documentation of "history -a"

[Reuben Thomas]

At least in bash 4.3, the documentation for history -a says:

  Append the new history lines (history lines entered since the
  beginning of the current Bash session) to the history file.

This is unfortunately misleading, since it suggests that the technique of
adding "history -a" to one's PROMPT_COMMAND, as suggested here:

http://heyrod.com/snippet/s/always-append-bash-history.html

​would append to one's history file many copies of the lines entered since
the beginning of the current session. Fortunately it doesn't do that, but
rather appends lines that have not yet been appended. I suggest making this
clear, and also mentioning the PROMPT_COMMAND trick, as I look forward to
losing far fewer useful commands from my global history by having them
instantly saved, and not lost when a long-running terminal hangs and must
be killed, or the machine running out of power etc.

-
​​
-
http://rrt.sc3d.org

Re: Default time for unmarked history lines

[Reuben Thomas]

On 7 January 2016 at 20:07, Eduardo A. Bustamante López 
wrote:

> (2) The history should be ordered monotonically (increasing?)
>

​Yes, and it's not at the moment (or wasn't, until I added timestamps to
every line in my history), because the lines at the start of the history,
with no timestamp, were given the current date and time, and lines at the
end of the history, with timestamps, were given​ the earlier date and time
at which they were added to the history. That's why the epoch is better.
Also, it's a better clue that the timestamp is being faked.

Re: Default time for unmarked history lines

[Reuben Thomas]

On 8 January 2016 at 04:21, Eduardo A. Bustamante López 
wrote:

>
> I now understand your points.
>

​Thanks very much for taking a look at this.​


> dualbus@hp ...src/gnu/bash % cat ~/.bash_history
> echo 1
> #1452197044
> echo a; sleep 1
> #1452197045
> echo b; sleep 1
>
> dualbus@hp ...src/gnu/bash % ./bash -i <<<
> "HISTTIMEFORMAT='%Y-%m-%dT%H:%M:%S ' history"
> dualbus@hp:~/local/src/gnu/bash$ HISTTIMEFORMAT='%Y-%m-%dT%H:%M:%S '
> history
> 1  1969-12-31T18:00:01 echo 1
> 2  2016-01-07T14:04:04 echo a; sleep 1
> 3  2016-01-07T14:04:05 echo b; sleep 1
> 4  1969-12-31T18:00:01 HISTTIMEFORMAT='%Y-%m-%dT%H:%M:%S ' history
>
> So, it seems that new entries are created with the localtime, unless
> there's a
> history comment followed by a timestamp. I'm not sure what changes are
> needed
> to adjust this, but I guess that is not a simple fix.
>

​I'm puzzled by this. What do you mean by "unless there's a history comment
followed by a timestamp"? The history file you dump ends with a command,
not a timestamp. I am guessing that the bash you then invoke does not have
HISTTIMEFORMAT set in its environment, so no timestamp is added for the
last line.


> BTW, the timestamp = 0 thing is a bug in readline.


​Aha! Thanks.​

--
​ ​

http://rrt.sc3d.org

Re: Default time for unmarked history lines

[Reuben Thomas]

On 11 January 2016 at 14:22, Chet Ramey  wrote:

> For a history file without any timestamps, using
> the current default and setting the history entry timestamp to the current
> time is more appropriate.
>

​Why is that? The only similar thing I can think of is file systems, where
if you zero the metadata you get timestamps of the epoch, not the current
time.​
​


> You probably also have histappend set or didn't use something like
> `history -w' to force the entire history file to be written when the shell
> exits.
>

​That's right.

--
​ ​

http://rrt.sc3d.org

Fwd: Default time for unmarked history lines

[Reuben Thomas]

[
​Forwarding reply erroneously not sent to the list.]​


On 15 January 2016 at 15:26, Chet Ramey  wrote:

> On 1/11/16 11:54 AM, Reuben Thomas wrote:
> > On 11 January 2016 at 14:22, Chet Ramey  > <mailto:chet.ra...@case.edu>> wrote:
> >
> > For a history file without any timestamps, using
> > the current default and setting the history entry timestamp to the
> current
> > time is more appropriate.
> >
> >
> > ​Why is that? The only similar thing I can think of is file systems,
> where
> > if you zero the metadata you get timestamps of the epoch, not the current
> > time.​
>
> OK, consider the situation when a user has no timestamps in his history
> file.  There seem to be three choices for the timestamp: the epoch time,
> which to me is clearly incorrect ("what? what is this 1970s stuff?"), the
> last modified time on the history file itself, and the current time.  I
> left the default as the current time, so commands from the history file
> are the same as commands entered at the terminal.
>

​I agree with the user story thought in principle.

The trouble is that this is an experience users have anyway with
filesystems and with programs that deal with metadata in files, such as
dates in JPEGs: zero (and sometimes absent) metadata result in epoch
times/dates.

​By fixing it just in bash you're actually confusing users (like me) who've
worked out what tends to happen with 0 dates in general.

So, how about instead interpreting a missing/0 date as a NaD (Not A Date),
rather as readline does anyway with time 0, and providing a slightly more
meaningful message than the current "??". Then a) I would be able to remove
all my bogus "1" timestamps, and b) both "0" and missing timestamps would
give the user a clue that data was missing and/or zero?


> > You probably also have histappend set or didn't use something like
> > `history -w' to force the entire history file to be written when the
> shell
> > exits.
> >
> >
> > ​That's right.​
>
> Then you have presented the history library with an ambiguous situation.
>

Correct, and it's entirely natural: for years I didn't use timestamps, now
I do.

Again, with the solution I propose above, I would get a monotonic history
for the part where time is defined, which is good enough.​

-- 
http://rrt.sc3d.org

Re: Fwd: Default time for unmarked history lines

[Reuben Thomas]

On 18 January 2016 at 22:21, Chet Ramey  wrote:

> On 1/18/16 11:53 AM, Reuben Thomas wrote:
>
> > So, how about instead interpreting a missing/0 date as a NaD (Not A
> Date),
> > rather as readline does anyway with time 0, and providing a slightly more
> > meaningful message than the current "??". Then a) I would be able to
> remove
> > all my bogus "1" timestamps, and b) both "0" and missing timestamps would
> > give the user a clue that data was missing and/or zero?
>
> I think that a more meaningful error string would be useful.  We'll try
> reporting on invalid dates if the timestamp string in the history entry is
> non-empty.
>

​Sounds good, thanks.

-- 
http://rrt.sc3d.org

Time to increase the defaults for history size?

[Reuben Thomas]

The default HISTSIZE and HISTFILESIZE have been set for as long as I can
remember (and the repo only seems to back to bash 3.2).

For those concerned about privacy or security, 500 lines is probably too
much.

For those concerned about disk space, it's hard to come up with a sensible
default. My roughly 500,000-command long ~/.bash_history is getting on for
10 years old or so, but only about 12Mb.

I suggest therefore that the default be set to unlimited history (both
HISTSIZE and HISTFILESIZE), so that more users benefit out of the box from
the amazing powers unleashed by being able to get any command you ever
typed.

-- 
http://rrt.sc3d.org

Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=849517

​I can reproduce this also in bash 4.3 as supplied with Ubuntu 16.04, and
in a build of 4.4 from source on my Ubuntu system.

​As stated in the bug report, the bug causes problems beyond bash, as it
causes build systems to think that valgrind is not working, and hence run
tests without it.

I searched the bug-bash archives and couldn't find a previous report of
this particular problem (I found mentions of other valgrind diagnostics).

-- 
http://rrt.sc3d.org

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

​O​
n 12 April 2017 at 14:50, Chet Ramey  wrote:

> On 4/12/17 8:57 AM, Reuben Thomas wrote:
> > See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=849517
> >
> > ​I can reproduce this also in bash 4.3 as supplied with Ubuntu 16.04, and
> > in a build of 4.4 from source on my Ubuntu system.
> >
> > ​As stated in the bug report, the bug causes problems beyond bash, as it
> > causes build systems to think that valgrind is not working, and hence run
> > tests without it.
>
> Look at it this way: if the "bug" can't be reproduced on another OS, or
> can't be reproduced using a debugging malloc library other than valgrind,
> that points the finger at valgrind or Debian.
>

​Apologies, I didn't think of reproducibility on other OSes. (Though the
point about valgrind is not a strong one: ​
​
​frequently, it's the only tool that shows up bugs of this sort, as it's
rather more powerful than a debugging malloc library.)

Apologies also for not noticing that the backtrace was lacking. I've
installed debugging symbols here and obtained:

==19358== Invalid free() / delete / delete[] / realloc()
==19358==at 0x4C2ED5B: free (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==19358==by 0x45E1D0: unwind_frame_run_internal.constprop.3
(unwind_prot.c:301)
==19358==by 0x45E37F: without_interrupts (unwind_prot.c:107)
==19358==by 0x45E37F: run_unwind_frame (unwind_prot.c:135)
==19358==by 0x47B664: parse_and_execute (evalstring.c:421)
==19358==by 0x4209D6: run_one_command (shell.c:1348)
==19358==by 0x41F893: main (shell.c:695)
==19358==  Address 0x423b6e8 is in the brk data segment 0x4228000-0x423bfff
​
Do let me know if there's anything else useful I can do.

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

​O​
n 12 April 2017 at 15:49, Chet Ramey  wrote:

>
> It's a false positive, or a bug in valgrind. I took a quick look.  There's
> one place in this code path where free() gets called.  Here's the trace:
>

​[analysis snipped]

Thanks very much, looks like it's time for me to file a Valgrind bug report.

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

On Apr 12, 2017 4:56 PM, "Chet Ramey"  wrote:

[snip]

> Maybe update the Debian bug report you cited as well.  There's still stuff
> there from 2005.

The report is from December 2016. I can't find "2005" in the page.

I'll certainly send an update to point to the upstream bug I file.

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

On 12 April 2017 at 17:58, Hanno Böck  wrote:

> On Wed, 12 Apr 2017 14:59:26 +0100
> Reuben Thomas  wrote:
>
> > ​frequently, it's the only tool that shows up bugs of this sort, as
> > it's rather more powerful than a debugging malloc library.)
>
> Try address sanitizer, it's pretty reliable and finds even more bug
> classes compared to valgrind. Just add -fsanitize=address to the CFLAGS.
>
> If you can't reproduce the bug with asan I'd bet on a valgrind bug.


​Thanks very much for the recommendation (and to Chet); I'd not heard of
it. Once I've fixed all the bugs I'm finding with valgrind, I'll go back
and see what asan finds, and also see if it finds anything else, so I can
see whether to run it as well or instead of valgrind.

Having confirmed Chet's analysis with a few printfs added to bash (i.e.
just to check the address being allocated and the one complained about were
the same) I've filed a bug report against valgrind:
https://bugs.kde.org/show_bug.cgi?id=378732

-- 
http://rrt.sc3d.org

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

On 13 April 2017 at 09:15, Reuben Thomas  wrote:

>
> Having confirmed Chet's analysis with a few printfs added to bash (i.e.
> just to check the address being allocated and the one complained about were
> the same) I've filed a bug report against valgrind:
> https://bugs.kde.org/show_bug.cgi?id=378732
>

​Julian Seward (valgrind author) correctly​

​guessed that it is a problem with malloc/free interception: when bash uses
its own malloc (as by default on my GNU/Linux system), the malloc is not
intercepted by valgrind, but the free is.

Obviously a bug in valgrind, but I am interested: why does bash not use the
glibc malloc by default on GNU/Linux systems?

-- 
http://rrt.sc3d.org

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

On 12 April 2017 at 15:49, Chet Ramey  wrote:

>
> It's a false positive, or a bug in valgrind. I took a quick look.  There's
> one place in this code path where free() gets called.


​Julian Seward (valgrind author) pointed out:​

"
​…​
what you report is symptomatic of bash
​ ​
using its own malloc to allocate a block but the system free to release
​ ​
it.  Could that be the case?
​"

I had a look. Certainly at xmalloc.c:148 where free is called by xfree from
the cleanup function called at unwind_prot.c:333, gdb reports:

p free
$7 = {void (void *)} 0x77df0d80 

This is glibc free.

If I put a breakpoint on xmalloc and rerun, it is not hit.

If I put a breakpoint on shell.c:1399, and trace into savestring, I find it
is running sh_xmalloc.

So it seems that the malloc and free calls are mismatched.

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

​O​
n 13 April 2017 at 15:33, Reuben Thomas  wrote:

> On 12 April 2017 at 15:49, Chet Ramey  wrote:
>
>>
>> It's a false positive, or a bug in valgrind. I took a quick look.  There's
>> one place in this code path where free() gets called.
>
>
> ​Julian Seward (valgrind author) pointed out:​
>
> "
> ​…​
> what you report is symptomatic of bash
> ​ ​
> using its own malloc to allocate a block but the system free to release
> ​ ​
> it.  Could that be the case?
> ​"
>
> I had a look. Certainly at xmalloc.c:148 where free is called by xfree
> from the cleanup function called at unwind_prot.c:333, gdb reports:
>
> p free
> $7 = {void (void *)} 0x77df0d80 
>
> This is glibc free.
>
> If I put a breakpoint on xmalloc and rerun, it is not hit.
>
> If I put a breakpoint on shell.c:1399, and trace into savestring, I find
> it is running sh_xmalloc.
>
> So it seems that the malloc and free calls are mismatched.
>

​Trying to understand this at a source level, if I capture the compilation
command for evalstring.c​ and replace -c with -E to preprocess it, I see
that the relevant line has become:

add_unwind_protect (xfree, orig_string)

where xfree is now the function name (in xmalloc.c).

So this still looks wrong (it should be sh_xfree, surely?).

-- 
http://rrt.sc3d.org

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

On 13 April 2017 at 16:11, Chet Ramey  wrote:

>
> I see no reason why, since all of these things are defined in the same
> file and are statically linked, `free' would resolve to the glibc free
> when malloc resolves to the bash malloc.


So this is the real problem?​

​Do you have any suggestions for how I might investigate this?​

-- 
http://rrt.sc3d.org

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

​O​
n 13 April 2017 at 16:27, Chet Ramey  wrote:

> On 4/13/17 11:18 AM, Reuben Thomas wrote:
> > On 13 April 2017 at 16:11, Chet Ramey  > <mailto:chet.ra...@case.edu>> wrote:
> >
> >
> > I see no reason why, since all of these things are defined in the
> same
> > file and are statically linked, `free' would resolve to the glibc
> free
> > when malloc resolves to the bash malloc.
> >
> >
> > So this is the real problem?​
>
> If it is, it's a valgrind artifact.  Contrary to Julian's (?) assumption,
> free() resolves to the bash free implementation when compiled normally. I
> tested this on a Fedora 25 VM I was using for something else.  Putting a
> breakpoint in xfree, running bash -c '', and stepping through it ends up
> in the free() defined in lib/malloc/malloc.c.
>

​This is not the result I obtained. I simply ran gdb on the bash binary,
valgrind was not involved.​

​Indeed, building --without-bash-malloc fixed it.​

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

On 13 April 2017 at 16:46, Chet Ramey  wrote:

> On 4/13/17 11:41 AM, Reuben Thomas wrote:
>
> > ​This is not the result I obtained. I simply ran gdb on the bash binary,
> > valgrind was not involved.​
>
> If you didn't build the binary yourself,


​I did, from git master. Sorry that wasn't clear.​

​Thanks for your gdb traces.

Here's mine on Ubuntu 16.04:

$ ./bash --version
GNU bash, version 4.4.12(1)-release (x86_64-unknown-linux-gnu)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ gdb ./bash
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./bash...done.
(gdb) b xfree
Breakpoint 1 at 0x47ca80: file xmalloc.c, line 147.
(gdb) r -c ''
Starting program: /home/rrt/.local/var/repo/bash/bash -c ''
s is  0x721368
s is  0x721368
string is  0x721368

Breakpoint 1, xfree (string=0x721368) at xmalloc.c:147
147  if (string)
(gdb) s
148free (string);
(gdb) s
internal_free (mem=0x721368, file=0x0, line=0, flags=) at
malloc.c:846
846malloc.c: No such file or directory.
(gdb) s
858in malloc.c
(gdb) p free
$1 = {void (void *)} 0x77df0d80 
(gdb)

So it seems I misunderstood what gdb was telling me: "p free" gives the
address of glibc free (even if I do it inside xfree). But it is running
bash's free.

(There are some extra diagnostics there which are debugging statements I
put in.)

Meanwhile, in the valgrind bug report, Mark Wielaard observed:

"I think the problem is that bash not only has its own malloc/free
implementation (valgrind should intercept that just fine). But also has
malloc wrappers for some malloc functions that call their internal
counterparts directly (so valgrind won't know that is a matching
allocation/deallocation function)."

Mark manages to fix the problem by defining DISABLE_MALLOC_WRAPPERS, which
is at least less drastic than configuring --without-bash-malloc.

I shall pursue the matter on the Valgrind side. Mark also says that
compiled thus he finds some small memory leaks in bash, so I'll report back
if I can confirm.

Thanks very much again for your help with this problem.

-- 
http://rrt.sc3d.org

Leaks detected by valgrind

[Reuben Thomas]

Here they are. I guess you probably won't care about them as as far as I
can see they are all one-off allocations at initialization:

==1289== 276 bytes in 1 blocks are definitely lost in loss record 230 of 249
==1289==at 0x4C2DB2F: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1289==by 0x4793CF: xmalloc (xmalloc.c:112)
==1289==by 0x472F48: set_default_locale (locale.c:81)
==1289==by 0x420900: main (shell.c:411)
==1289==
==1289== 1,008 bytes in 1 blocks are definitely lost in loss record 232 of
249
==1289==at 0x4C2DB2F: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1289==by 0x47939A: xmalloc (xmalloc.c:112)
==1289==by 0x496FDD: sh_setlinebuf (setlinebuf.c:45)
==1289==by 0x41F83C: shell_initialize (shell.c:1799)
==1289==by 0x4210BB: main (shell.c:570)
==1289==
==1289== 1,008 bytes in 1 blocks are definitely lost in loss record 233 of
249
==1289==at 0x4C2DB2F: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1289==by 0x47939A: xmalloc (xmalloc.c:112)
==1289==by 0x496FDD: sh_setlinebuf (setlinebuf.c:45)
==1289==by 0x41F848: shell_initialize (shell.c:1800)
==1289==by 0x4210BB: main (shell.c:570)
==1289==

-- 
http://rrt.sc3d.org

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

On 13 April 2017 at 21:17, Chet Ramey  wrote:

> On 4/13/17 3:57 PM, Reuben Thomas wrote:
>
> > Meanwhile, in the valgrind bug report, Mark Wielaard observed:
> >
> > "I think the problem is that bash not only has its own malloc/free
> > implementation (valgrind should intercept that just fine). But also has
> > malloc wrappers for some malloc functions that call their internal
> > counterparts directly (so valgrind won't know that is a matching
> > allocation/deallocation function)."
>
> Yes, if he wants the details, feel free to forward my message from earlier
> today.
>

​Thanks, I
​ already linked to it in the valgrind bug report
.​

​​
-- 
http://rrt.sc3d.org

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

On 13 April 2017 at 22:05, Reuben Thomas  wrote:

>
> On 13 April 2017 at 21:17, Chet Ramey  wrote:
>
>> On 4/13/17 3:57 PM, Reuben Thomas wrote:
>>
>> > Meanwhile, in the valgrind bug report, Mark Wielaard observed:
>> >
>> > "I think the problem is that bash not only has its own malloc/free
>> > implementation (valgrind should intercept that just fine). But also has
>> > malloc wrappers for some malloc functions that call their internal
>> > counterparts directly (so valgrind won't know that is a matching
>> > allocation/deallocation function)."
>>
>> Yes, if he wants the details, feel free to forward my message from earlier
>> today.
>>
>
> ​Thanks, I
> ​ already linked to it in the valgrind bug report
> .​
>

​I've closed the Valgrind bug report WONTFIX. Mark Wielaard there kindly
provides a patch to bash to make it play nicely with Valgrind, though I
doubt you'll want to take it (just to be clear, I wouldn't if I were a bash
maintainer!).

​Since RedHat and Fedora build --without-bash-malloc, that seems an
acceptable solution for Debian/Ubuntu. (Not least, performance there is
less of an issue since the default /bin/sh is no longer bash.)

As Mark says, and if it hasn't already been tried, it would be good if any
benchmarks could be brought to the attention of the glibc maintainers if
there's potential for improvement in glibc's malloc.

-- 
http://rrt.sc3d.org

Confusing documentation of `ENV` in section "Bash variables"

[Reuben Thomas via Bug reports for the GNU Bourne Again SHell]

The documentation says:

'ENV'
 Similar to 'BASH_ENV'; used when the shell is invoked in POSIX Mode
 (*note Bash POSIX Mode::).

However, as described elsewhere in the manual, BASH_ENV is used
specifically for non-interactive shells, whereas ENV is used specifically
for interactive shells. From the quoted section above, one might get the
erroneous impression that ENV is just like BASH_ENV but for POSIX mode.

If I've not misunderstood, maybe a little more explanation in the quoted
section would be a good idea? Even just changing "invoked" to "invoked
interactively". Although that then runs the risk of implying that BASH_ENV
is also used in interactive mode. Alternatively, remove "Similar to
'BASH_ENV';".

-- 
https://rrt.sc3d.org

Re: Confusing documentation of `ENV` in section "Bash variables"

[Reuben Thomas via Bug reports for the GNU Bourne Again SHell]

On Wed, 23 Sep 2020 at 14:24, Chet Ramey  wrote:


> "Expanded and executed similarly to BASH_ENV when an interactive shell is
> invoked in POSIX Mode."
>

Yes, that's better than my suggestions, thanks!

-- 
https://rrt.sc3d.org