The clean opt can not delete all objs(like *.o)

[chrlis]

I find a special case:
->cd bash-4.4
->mkdir build 
->cd build
->...
->make clean

When I make clean current building directory, it never remove the object 
'man2html.o' in the path /path/to/build/support.
Is this a problem ? 

The released bash source code version is: 
https://ftp.gnu.org/gnu/bash/bash-4.4.tar.gz

Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=849517

​I can reproduce this also in bash 4.3 as supplied with Ubuntu 16.04, and
in a build of 4.4 from source on my Ubuntu system.

​As stated in the bug report, the bug causes problems beyond bash, as it
causes build systems to think that valgrind is not working, and hence run
tests without it.

I searched the bug-bash archives and couldn't find a previous report of
this particular problem (I found mentions of other valgrind diagnostics).

-- 
http://rrt.sc3d.org

Re: The clean opt can not delete all objs(like *.o)

[Chet Ramey]

On 4/12/17 3:27 AM, chrlis wrote:
> I find a special case:
> ->cd bash-4.4
> ->mkdir build
> ->cd build
> ->...
> ->make clean
> 
> When I make clean current building directory, it never remove the object
> 'man2html.o' in the path /path/to/build/support.
> Is this a problem ?

Thanks for the report.  I'll take a look.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Running bash under valgrind gives "invalid free()"

[Chet Ramey]

On 4/12/17 8:57 AM, Reuben Thomas wrote:
> See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=849517
> 
> ​I can reproduce this also in bash 4.3 as supplied with Ubuntu 16.04, and
> in a build of 4.4 from source on my Ubuntu system.
> 
> ​As stated in the bug report, the bug causes problems beyond bash, as it
> causes build systems to think that valgrind is not working, and hence run
> tests without it.

Look at it this way: if the "bug" can't be reproduced on another OS, or
can't be reproduced using a debugging malloc library other than valgrind,
that points the finger at valgrind or Debian.

For instance, here's what happens when you run valgrind on bash-4.4.12
under RHEL6:

$ valgrind --leak-check=full ./bash -c ''
==12098== Memcheck, a memory error detector
==12098== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==12098== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==12098== Command: ./bash -c
==12098==
==12098==
==12098== HEAP SUMMARY:
==12098== in use at exit: 0 bytes in 0 blocks
==12098==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==12098==
==12098== All heap blocks were freed -- no leaks are possible
==12098==
==12098== For counts of detected and suppressed errors, rerun with: -v
==12098== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 6 from 6)

Running the same command using bash's builtin malloc library, which catches
things like invalid frees, or the MacOS X debugging malloc, doesn't produce
that error.

I'll take another look, but it would help if there were some more detail
in the report besides `???'.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

​O​
n 12 April 2017 at 14:50, Chet Ramey  wrote:

> On 4/12/17 8:57 AM, Reuben Thomas wrote:
> > See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=849517
> >
> > ​I can reproduce this also in bash 4.3 as supplied with Ubuntu 16.04, and
> > in a build of 4.4 from source on my Ubuntu system.
> >
> > ​As stated in the bug report, the bug causes problems beyond bash, as it
> > causes build systems to think that valgrind is not working, and hence run
> > tests without it.
>
> Look at it this way: if the "bug" can't be reproduced on another OS, or
> can't be reproduced using a debugging malloc library other than valgrind,
> that points the finger at valgrind or Debian.
>

​Apologies, I didn't think of reproducibility on other OSes. (Though the
point about valgrind is not a strong one: ​
​
​frequently, it's the only tool that shows up bugs of this sort, as it's
rather more powerful than a debugging malloc library.)

Apologies also for not noticing that the backtrace was lacking. I've
installed debugging symbols here and obtained:

==19358== Invalid free() / delete / delete[] / realloc()
==19358==at 0x4C2ED5B: free (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==19358==by 0x45E1D0: unwind_frame_run_internal.constprop.3
(unwind_prot.c:301)
==19358==by 0x45E37F: without_interrupts (unwind_prot.c:107)
==19358==by 0x45E37F: run_unwind_frame (unwind_prot.c:135)
==19358==by 0x47B664: parse_and_execute (evalstring.c:421)
==19358==by 0x4209D6: run_one_command (shell.c:1348)
==19358==by 0x41F893: main (shell.c:695)
==19358==  Address 0x423b6e8 is in the brk data segment 0x4228000-0x423bfff
​
Do let me know if there's anything else useful I can do.

Re: Running bash under valgrind gives "invalid free()"

[Chet Ramey]

On 4/12/17 9:59 AM, Reuben Thomas wrote:
> ​O​
> n 12 April 2017 at 14:50, Chet Ramey  > wrote:
> 
> On 4/12/17 8:57 AM, Reuben Thomas wrote:
> > See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=849517
> 
> >
> > ​I can reproduce this also in bash 4.3 as supplied with Ubuntu 16.04, 
> and
> > in a build of 4.4 from source on my Ubuntu system.
> >
> > ​As stated in the bug report, the bug causes problems beyond bash, as it
> > causes build systems to think that valgrind is not working, and hence 
> run
> > tests without it.
> 
> Look at it this way: if the "bug" can't be reproduced on another OS, or
> can't be reproduced using a debugging malloc library other than valgrind,
> that points the finger at valgrind or Debian.
> 
> 
> ​Apologies, I didn't think of reproducibility on other OSes. (Though the
> point about valgrind is not a strong one: ​

It's a false positive, or a bug in valgrind. I took a quick look.  There's
one place in this code path where free() gets called.  Here's the trace:

1. the -c option causes run_one_command() to call parse_and_execute() with
   a dynamically-allocated string. Let's say its address is 0x7bd218

2. parse_and_execute calls parse_prologue with that string as the "string"
   argument. The value of "string" is still 0x7bd218.

3. parse_prologue assigns the value of string to "orig_string". orig_string
   still has the value 0x7bd218.

4. parse_prologue calls add_unwind_protect to arrange for that memory to be
   freed.  The unwind-protect structure that gets created has (simplified)
uwp->cleanup = xfree
uwp->argument = orig_string (still with the value 0x7bd218)

5. parse_and_execute runs the command, and calls run_unwind_frame. That
   calls each cleanup function in turn. xfree() gets called with a pointer
   whose value is 0x7bd218, frees it, and returns.

Before you ask, the string in step 1 is allocated by savestring(), which is
defined as

define savestring(x) (char *)strcpy (xmalloc (1 + strlen (x)), (x))

There's no place where that value is part of a data segment.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

​O​
n 12 April 2017 at 15:49, Chet Ramey  wrote:

>
> It's a false positive, or a bug in valgrind. I took a quick look.  There's
> one place in this code path where free() gets called.  Here's the trace:
>

​[analysis snipped]

Thanks very much, looks like it's time for me to file a Valgrind bug report.

Re: Running bash under valgrind gives "invalid free()"

[Chet Ramey]

On 4/12/17 10:59 AM, Reuben Thomas wrote:
> ​O​
> n 12 April 2017 at 15:49, Chet Ramey  > wrote:
> 
> 
> It's a false positive, or a bug in valgrind. I took a quick look.  There's
> one place in this code path where free() gets called.  Here's the trace:
> 
> 
> ​[analysis snipped]
> 
> Thanks very much, looks like it's time for me to file a Valgrind bug report.

Maybe update the Debian bug report you cited as well.  There's still stuff
there from 2005.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Running bash under valgrind gives "invalid free()"

[Reuben Thomas]

On Apr 12, 2017 4:56 PM, "Chet Ramey"  wrote:

[snip]

> Maybe update the Debian bug report you cited as well.  There's still stuff
> there from 2005.

The report is from December 2016. I can't find "2005" in the page.

I'll certainly send an update to point to the upstream bug I file.

Re: Running bash under valgrind gives "invalid free()"

[Hanno Böck]

On Wed, 12 Apr 2017 14:59:26 +0100
Reuben Thomas  wrote:

> ​frequently, it's the only tool that shows up bugs of this sort, as
> it's rather more powerful than a debugging malloc library.)

Try address sanitizer, it's pretty reliable and finds even more bug
classes compared to valgrind. Just add -fsanitize=address to the CFLAGS.

If you can't reproduce the bug with asan I'd bet on a valgrind bug.

-- 
Hanno Böck
https://hboeck.de/

mail/jabber: ha...@hboeck.de
GPG: FE73757FA60E4E21B937579FA5880072BBB51E42

Re: Running bash under valgrind gives "invalid free()"

[Chet Ramey]

On 4/12/17 12:58 PM, Hanno Böck wrote:
> On Wed, 12 Apr 2017 14:59:26 +0100
> Reuben Thomas  wrote:
> 
>> ​frequently, it's the only tool that shows up bugs of this sort, as
>> it's rather more powerful than a debugging malloc library.)
> 
> Try address sanitizer, it's pretty reliable and finds even more bug
> classes compared to valgrind. Just add -fsanitize=address to the CFLAGS.

It's a valgrind false positive. I run the entire bash test suite through
a version built with address sanitizer.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/