[Bug debug/93888] Incorrect DW_AT_location generated for copy-constructed function argument

[mgulick at mathworks dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93888

--- Comment #5 from Mike Gulick  ---
Thanks Jakub!  I can confirm that this fixes the issue for me.

[Bug debug/93888] New: Incorrect DW_AT_location generated for copy-constructed function argument

[mgulick at mathworks dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93888

Bug ID: 93888
   Summary: Incorrect DW_AT_location generated for
copy-constructed function argument
   Product: gcc
   Version: 10.0
Status: UNCONFIRMED
  Severity: normal
  Priority: P3
 Component: debug
  Assignee: unassigned at gcc dot gnu.org
  Reporter: mgulick at mathworks dot com
  Target Milestone: ---

Created attachment 47889
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=47889&action=edit
reproducer code, gdb test script, makefile and README

Hi,

I have observed that gcc produces an incorrect DW_AT_location list for a
function argument when both of the following are true:

a) The argument is passed by value with the help of a copy constructor
b) The function is inlined

When this occurs, the DW_AT_location list actually ends up pointing to the
address of the data, not the data itself.  Thus, it is either missing a
DW_OP_deref, or contains a gratuitous DW_OP_stack_value.

I have turned off all other optimizations and the issue is still reproducible.

I have observed this issue on all versions of GCC I have available to test,
including the current git master, 8.3, 6.3, 4.9, 4.7, and 4.4.

Here is a sample program:
class K
{
public:
  K() {}
  // Commenting out this copy-constructor makes the test pass
  K ( K const& rhs ) {
m_storage[0] = 'C';  // 0x43
  }
  // Initialize this array to some known values (0x42)
  char m_storage[ 8 ] = {'B','B','B','B','B','B','B','B'};
};

// Not inlining also fixes the issue
__attribute__((always_inline))
bool func1(const K func1_k) {
  return func1_k.m_storage[0] == 'C';
}

int main() {
  K my_k;
  return func1(my_k);
}

I am compiling this with 'g++ -g -O0 -fvar-tracking ...'.

In gdb, if I break inside 'func1' and print the value of 'func1_k', the wrong
value is printed.  If I instead print '*(K*)func1_k', I get the expected value.

If I ask gdb what the address of func1_k is, I get:

(gdb) info address func1_k
Symbol "func1_k" is multi-location:
  Range 0x40113f-0x401148: a complex DWARF expression:
 0: DW_OP_breg6 -8 [$rbp]
 2: DW_OP_stack_value

I verified with dwarfdump that this matches the value in the .o file, so it
does not appear to be a gdb bug.  I am using the latest gdb 9.1 release.

I have attached a tarball containing the reproduction code, along with a gdb
test script, and a README with additional details.

While this issue itself is relatively minor, there isn't an easy workaround if
you need to debug optimized code.  Also, in the production code where this
issue was found, the value being passed happens to be a boost::optional which
we have a gdb pretty-printer setup for, and this causes the unfortunate
side-effect that running a 'backtrace' command in gdb any time this function is
in the stack (which is pretty much all the time) causes the boost
pretty-printer to try to dereference an invalid value, and gdb segfaults.

I spent some time trying to debug gcc itself, but I got lost very quickly and
was unable to localize where the issue is.  Hopefully this reproduction case is
small-enough to help you determine the source of the error.

Thank you!

-Mike

[Bug preprocessor/83173] C preprocessor generates incorrect linemarkers

[mgulick at mathworks dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83173

--- Comment #6 from Mike Gulick  ---
(In reply to Pádraig Brady from comment #5)
> Seeing this also with GCC 7.3. Will try proposed patches

Latest version of patch and test:

Patch: https://gcc.gnu.org/ml/gcc-patches/2018-02/msg00557.html
Test: https://gcc.gnu.org/ml/gcc-patches/2018-01/msg01993.html

[Bug preprocessor/83173] C preprocessor generates incorrect linemarkers

[mgulick at mathworks dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83173

--- Comment #10 from Mike Gulick  ---
In hopes of seeing some progress on this bug, I will rebase the patches on the
latest gcc master branch and re-test.  I will also refactor the main patch to
separate out the functional fix from the diagnostics change, which will
hopefully make reviewing easier.  I'll reply here with a link to the
gcc-patches archive once I have posted them.

[Bug preprocessor/83173] C preprocessor generates incorrect linemarkers

[mgulick at mathworks dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83173

--- Comment #11 from Mike Gulick  ---
(In reply to Mike Gulick from comment #10)
> In hopes of seeing some progress on this bug, I will rebase the patches on
> the latest gcc master branch and re-test.  I will also refactor the main
> patch to separate out the functional fix from the diagnostics change, which
> will hopefully make reviewing easier.  I'll reply here with a link to the
> gcc-patches archive once I have posted them.

https://gcc.gnu.org/ml/gcc-patches/2018-11/msg00025.html

[Bug preprocessor/83173] C preprocessor generates incorrect linemarkers

[mgulick at mathworks dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83173

--- Comment #16 from Mike Gulick  ---
(In reply to Eric Gallager from comment #15)

> So can this be closed as FIXED now?

Yes, fixed by r266516.

[Bug preprocessor/83173] New: C preprocessor generates incorrect linemarkers

[mgulick at mathworks dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83173

Bug ID: 83173
   Summary: C preprocessor generates incorrect linemarkers
   Product: gcc
   Version: 7.2.0
Status: UNCONFIRMED
  Severity: normal
  Priority: P3
 Component: preprocessor
  Assignee: unassigned at gcc dot gnu.org
  Reporter: mgulick at mathworks dot com
  Target Milestone: ---

Created attachment 42724
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=42724&action=edit
invalid linemarker reproduction source code

GCC is generating incorrect line markers when processing source files with a
large number of includes (e.g. where the internal line map's source location is
greater than LINE_MAP_MAX_LOCATION_WITH_COLS.  This results in warnings from
gcc when it compiles this previously preprocessed output.  It also results in
gcc generating incorrect line mappings in the debug symbols, as well as
producing compiler warnings and errors that are associated with the wrong file
name and line number.

This invalid line marker is generated for include files that have a '#include'
as the last line in the file.  Simply adding an additional newline after the
last include will eliminate this issue.

This issue only occurs when the preprocessor is run as a separate phase, as
when using -save-temps, -no-integrated-cpp, or -E (as is used by distcc).

See also: https://gcc.gnu.org/ml/gcc-help/2017-11/msg00073.html

I have tested gcc 6.3, 7.2, as well as the latest git master (as of yesterday).
 These all exhibit this bug.  gcc 5.4 does *not* exhibit this bug.

The attached reproducer is able to reproduce this issue on a trivial source
file by using a gcc plugin that artificially increases the line map's starting
location before the preprocessor runs.  See README.txt included in the
reproduction tarball for instructions on running the reproducer.

[mgulick@mgulick2-deb9-64:/local-ssd/mgulick/src/gcc/linemarker_repro] ...
$ make
/local-ssd/mgulick/src/gcc/git/debug/gcc/xgcc -B
/local-ssd/mgulick/src/gcc/git/debug/gcc -E repro.c -o repro.i
-fplugin=./location_overflow_plugin.so
-fplugin-arg-location_overflow_plugin-value=0x6000
setting line_table location offset: 1610612736 was (32)
/local-ssd/mgulick/src/gcc/git/debug/gcc/xgcc -B
/local-ssd/mgulick/src/gcc/git/debug/gcc -c repro.i -o repro.o -Werror
In file included from repro_1.h:3,
 from repro.c:1:
repro_2.h:2:16: error: file "repro.h" linemarker ignored due to incorrect
nesting [-Werror]

^
repro_2.h:3:16: error: file "repro.c" linemarker ignored due to incorrect
nesting [-Werror]
 #define REPRO_2_H
^
cc1: all warnings being treated as errors
Makefile:45: recipe for target 'test_compile' failed
make: *** [test_compile] Error 1

[mgulick@mgulick2-deb9-64:/local-ssd/mgulick/src/gcc/linemarker_repro] ...
$ cat repro.i
# 1 "repro.c"
# 1 ""
# 1 ""
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "" 2
# 1 "repro.c"
# 1 "repro.h" 1
# 1 "repro_1.h" 1

# 2 "repro.h" 2
# 3 "repro_1.h"
# 1 "repro_2.h" 1

# 2 "repro.h" 2
# 2 "repro.c" 2

The line "# 3 repro_1.h" is incorrect and should not appear.

[Bug preprocessor/83173] C preprocessor generates incorrect linemarkers

[mgulick at mathworks dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83173

--- Comment #1 from Mike Gulick  ---
Just a minor update.  I re-tested the reproducer on gcc 5.4 as well as 4.9.2,
and the issue is present in both of those.  I had originally thought the bug
was not present in gcc 5.4 or earlier, however this is likely because the
source code I was testing it on was not pushing the line map source location >
0x6000 in these versions of the compiler.

[Bug preprocessor/83173] C preprocessor generates incorrect linemarkers

[mgulick at mathworks dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83173

--- Comment #2 from Mike Gulick  ---
I have made some progress in determining the cause of this bug.  This issue
occurs when the current source_location is > LINE_MAP_MAX_LOCATION_WITH_COLS
and when a #include is the last line in the file (with a terminating newline).

The corruption occurs when _cpp_stack_include decrements
ptable->line_table->highest_location.  It does this so that highest_location
refers to the *current* line in the file, not the next line.  For the case
where a #include is *not* the last line in the file, this works correctly. 
However when the the source location is > LINE_MAP_MAX_LOCATION_WITH_COLS and
the current #include line being processed is the last line in the file, the
highest_location value already refers to the current line in the file, as there
is no next line.  Thus this decrement sets highest_location to the previous
line in the file, which causes the corruption.

Consider an include file with two #includes:

  #include "foo.h"
  #include "bar.h"
  EOF

Consider when do_include_common() processes the final '#include "bar.h"'.  This
initially calls parse_include().  This calls check_eol(), which eventually
calls _cpp_lex_direct() via the following call stack:

  0 _cpp_lex_direct
  1 _cpp_lex_token
  2 cpp_get_token_1
  3 cpp_get_token
  4 check_eol_1
  5 check_eol
  6 parse_include
  7 do_include_common
  8 do_include
  9 _cpp_handle_directive
  10 _cpp_lex_token
  11 cpp_get_token_1
  12 cpp_get_token_with_location
  13 scan_translation_unit
  14 preprocess_file

_cpp_lex_direct parses the current buffer one character at a time.  In the case
of the line "#include bar.h", the buffer looks like:

  #include "bar.h"\n\n

Note that the second '\n' is added to the buffer when the file is initially
read in.  It doesn't exist in the file.

After parsing the '#include "bar.h", the buffer is sitting at the first '\n'.

  #include "bar.h"\n\n
   ^ ^
  buffer.cur---/ |
 |
  buffer.rlimit--/ 

buffer.rlimit is a pointer to the end of the buffer.  It points to the final
newline that was added to the end of the buffer when the file was read.

_cpp_lex_direct() reads the buffer one character at a time, e.g.

  c = *buffer->cur++
  ...
  switch (c)
{
...
case '\n':
  if (buffer->cur < buffer->rlimit)
CPP_INCREMENT_LINE (pfile, 0)
  buffer->need_line = true;
  goto fresh_line;
...

Under normal circumstances (i.e. if the #include is *not* the last line in the
file), when the '\n' is detected, CPP_INCREMENT_LINE increments the
line_maps->highest_line.  However for this last #include, buffer->cur ==
buffer->rlimit, so CPP_INCREMENT_LINE is not called.

Thus if the #include token has source_location 1610612807, highest_location in
the line_maps structure also has 1610612807.  Remember that we are past
LINE_MAP_MAX_LOCATION_WITH_COLS, so column numbers are not tracked, thus each
increment of a source_location value refers to a new line number and
potentially a new source file.

Continue stepping through do_include_common to _cpp_stack_include.  This
function has the following comment:

  /* Compensate for the increment in linemap_add that occurs if
  _cpp_stack_file actually stacks the file.  In the case of a
 normal #include, we're currently at the start of the line
 *following* the #include.  A separate source_location for this
 location makes no sense (until we do the LC_LEAVE), and
 complicates LAST_SOURCE_LINE_LOCATION.  This does not apply if we
 found a PCH file (in which case linemap_add is not called) or we
 were included from the command-line.  */

Under normal circumstances, the comment stating "we're currently at the start
of the line *following* the include is correct.  However in this case, this is
not true because we did not increment highest_line, thus highest_location still
refers to the current line.  Thus when we decrement highest_line, this makes
highest_line actually refer to the *previous* line map location, not the
current.  _cpp_stack_file then ultimately calls linemap_add, which sets
start_location to highest_location + 1.  This is assumed to be a new, unused
location, but in this case it actually already refers to an existing line map. 
Note that the linemap_assert in linemap_add will not catch this even if linemap
assertions are enabled.  This is because it only asserts if the new
start_location is less than the source_location of last line in the line map,
however in this case it is equal to the source_location of the last line.

We fix this by no longer decrementing pfile->line_table->highest_location if it
is less than or equal to the source_location of the current include header. 
The purpose of this decrement is to ensure that
pfile->line_table->highest_location still refers to the current line, so if it
already refers to the current line, there is no need to decrement it (and doing
so would be wrong).