Re: [Dwarf-discuss] ISSUE: CPU vector types.

[Kyle Huey via Dwarf-discuss]

On Mon, Mar 27, 2023 at 11:52 PM Cary Coutant via Dwarf-discuss <
dwarf-discuss@lists.dwarfstd.org> wrote:

> > Vector registers
> >
> > It has been the long standing existing practice to treat hardware
> > vector registers as arrays of a fundamental base type. To deliniate
> > these hardware register arrays from arrays in the language source they
> > have been given the DW_AT_GNU_vector attribute. This proposal simply
> > standardizes the existing behavior.
> >
> > In Section 2.2 Attribute Types, DW_AT_vector and
> > DW_AT_variable_vector_width shall be added to Table 2.2
> >
> > 
> > DW_AT_vector| A hardware vector register
> > DW_AT_variable_vector_width | Array bound for hardware
> > | implementation defined vector register
> > | width
> > 
>
> I don't understand what tags this DW_AT_vector attribute would apply
> to. Vector registers aren't *types*, they're *locations*, so it
> doesn't really make sense to me to put this attribute on a
> DW_TAG_array_type. We don't have DW_TAGs that describe registers; the
> ABI defines the registers and DWARF producers and consumers should
> understand and agree on the sizes and shapes of the various registers.
>
> In Tony's proposal, the new attribute modifies a base type, thus
> introducing a vector type, which might get placed in a vector
> register. But there, I don't see how the vector base type is
> fundamentally different from an array type. It seems it's just a dodge
> to make it a base type so that we can put whole vectors on the stack.
>
> Maybe what we're looking for is a DW_TAG_vector_type, whose DW_AT_type
> attribute gives the base type for each element of the vector. This
> seems to be more DWARF-like, and if we decide there's a reason to
> allow stack entries with vector types, we can do that.
>
> -cary
> --
> Dwarf-discuss mailing list
> Dwarf-discuss@lists.dwarfstd.org
> https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss


(Caveat this is all amd64 specific, I haven't looked at ARM.)

Last I looked into this I concluded that DW_AT_GNU_vector is a big hack to
make debuggers capable of handling function entries and exits that involve
the intrinsic types that hardware vendors have added for their SIMD
features.

In the SYSV AMD64 ABI (download link
https://gitlab.com/x86-psABIs/x86-64-ABI/-/jobs/artifacts/master/raw/x86-64-ABI/abi.pdf?job=build)
in Section 3.2.3 there is a subsection "Classification". Immediately after
the base types, _m64/_m128/_m256/_m512 are special cased to be part of the
"SSE" class. If you follow the algorithm described you'll note that
non-special cased arrays are treated as aggregates of the element type and
are not "promoted" to the "SSE" class. So _m256d and double[4], despite
having identical layouts in memory, will be classified as SSE/SSEUP and
MEMORY respectively by that algorithm (in particular note footnote 15
here). This is also easily verifable by compiling a trivial function that
takes _m256d and double[4] parameters.

A debugger usually doesn't care about any of this because the DIEs for
local/global variables will contain location information specifying where
to get the bytes from to render the variable. DWARF does not, however,
specify the locations of function return values at exit or parameters at
function entry[0]. These locations are needed, in turn, to display the
function return value upon function exit (think `ret` in gdb)[1] and to
invoke a function with user supplied parameters (think `call` in gdb).
Debuggers generally infer these locations from their knowledge of the
platform ABI[2]. The DW_AT_GNU_vector attribute informs the debugger that
these types are among the ones special-cased in the ABI and prevents the
need to do something even more unprincipled such as name matching.

DW_AT[_GNU]_vector is best understood not as "a hardware vector register"
but rather as a marker that "this type is eligible to be passed in hardware
vector registers at function boundaries according to the platform ABI".

- Kyle

[0] DWARF has DW_TAG_formal_parameter and those can contain location
information *but* DWARF does not specify that the location information must
be valid at function entry. For debug builds, compilers often emit
locations that are only valid after the function prologue has completed.
Post-prologue locations are useless for invoking the function in a debugger.
[1] Inferring the function return value location causes issues in other
contexts and I have a DWARF issue on that (221105.1)
[2] See e.g. amd64_classify in gdb/amd64-tdep.c
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] ISSUE: CPU vector types.

[Kyle Huey via Dwarf-discuss]

On Tue, Mar 28, 2023 at 1:17 PM David Blaikie  wrote:

> > DW_AT[_GNU]_vector is best understood not as "a hardware vector
> register" but rather as a marker that "this type is eligible to be passed
> in hardware vector registers at function boundaries according to the
> platform ABI".
>
> My 2c would not be to describe these in terms of
> hardware/implementations (that gets confusing/blurs the line between
> variable/types and locations - as you say, these things can be stored
> in memory, so they aren't uniquely in registers - you might have a
> member of this type in a struct passed in memory and need to know the
> ABI/struct layout for that, etc), but at the source level - which the
> ABI is defined in those same terms. Overloading, for instance, still
> applies if these are different types - so other debugger features need
> to work based on this type information.
>
> So it seems like a simpler question is:
>
> How should DWARF producers/consumers expect to encode the source
> example Ben provided (well, simplified a bit):
>
> #include 
>
> void f( __m128 a){
> }
>
> What DWARF should be used to describe the type of 'a'? And how does
> this encoding scale to all the other similar intrinsic types?
>

Stepping back even further I'd ask what information does the compiler need
to communicate to the debugger or other tools that's not communicated by
the DWARF for e.g. double[4]? The only information I'm personally aware of
is the ABI stuff I mentioned before, but there may very well be other
things lurking.

- Kyle
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] ISSUE: CPU vector types.

[Kyle Huey via Dwarf-discuss]

On Mon, Apr 3, 2023 at 12:42 PM Pedro Alves via Dwarf-discuss <
dwarf-discuss@lists.dwarfstd.org> wrote:

> Hi Ben,
>
> On 2023-03-24 6:19 p.m., Ben Woodard via Dwarf-discuss wrote:
>
> > I will admit that in its current state it is a work in progress. My
> original intent was to simply
> > codify the existing behavior as implemented by GCC and LLVM (and
> probably other compilers) moving it
> > from a vendor attribute to something that was in the standard.
>
> In light of the recent discussion, I was thinking how it would help if the
> proposal was laid out in terms of vector types instead of hardware
> registers.
>

I think the proposed language here is a bit too C/C++ centric.


> Please see comments inline below.
>
> > Vector registers
> >
> > It has been the long standing existing practice to treat hardware
> > vector registers as arrays of a fundamental base type. To deliniate
> > these hardware register arrays from arrays in the language source they
> > have been given the DW_AT_GNU_vector attribute. This proposal simply
> > standardizes the existing behavior.
>
> I'd suggest replacing the above with something like this:
>
> 
> Vector types
>
> As an extension to C and C++, GCC supports defining vector data types.
>

DW_AT_GNU_vector is used for languages beyond C/C++ today.


> On some targets, operations on these vector types make use of SIMD
> vector registers and instructions.
>
> Vector types are similar to arrays, and you can index and initialize
> them similarly, but they have some important differences.  For
> example:
>

These statements are not true in general for non C/C++ languages, and
DW_TAG_array_type represents a variety of array types across many languages.


> - Arrays automatically decay to pointers.  Vector types do not.
>

Array decay is only a C/C++ "feature" as far as I know.

- You can pass vector types by value to functions, and likewise
>   functions can return vector types by value.  You can not do either
>   with C arrays.
>

Arrays in other languages can be passed and returned by value at the
language level (e.g. Ada) and sometimes even at the machine level (e.g.
Rust).


> - Vector types can be used with a subset of normal C operations: +, -,
>   *, /, unary minus, ^, |, &, ~, %.  For example, addition is defined as
>   the addition of the corresponding elements of the operands.
>

The implicit statement here that non-vector arrays cannot be used with any
normal operations is also not true in general (e.g. in Fortran 90 A = B + C
works).


> A debugger that supports expression evaluation will want to be able to
> support these vector operations on vector objects too.
>

Supporting expression evaluation requires very deep knowledge of the
semantics of the program's language, to the point where I think saying much
beyond "vector types may have different semantics than standard array
types" in the DWARF spec isn't worth the effort.


> Vector types may appear on function prototypes, so they affect calling
> convention.
>
> To distinguish vector types from regular C arrays, GCC's DWARF
> describes the vector types as array with the DW_AT_GNU_vector
> attribute, a GNU extension.  Clang also supports the GCC vector
> language extension, and supports the same DWARF attribute as well.
> Vector types have supported by GDB for well over a decade, using said
> DWARF extension.  This proposal standardizes the existing behavior.
> 
>
> >
> > In Section 2.2 Attribute Types, DW_AT_vector and
> > DW_AT_variable_vector_width shall be added to Table 2.2
> >
> > 
> > DW_AT_vector| A hardware vector register
>
> Here, I think it would be better to say "A vector type".  An object of
> vector
> type could well have a memory location, for instance.
>
> > DW_AT_variable_vector_width | Array bound for hardware
> > | implementation defined vector register
> > | width
>
> Does GCC have a corresponding GNU extension for this?  I don't recall ever
> seeing it, and my grepping isn't finding it.
>
> > 
> >
> > The hyperlink in the "Identifies or Specifies" column shall point to
> > the paragraph added to Section 5.5 below for DW_AT_vector and the
> > paragraph added to Section 5.13 below for
> > DW_AT_variable_vector_width.
> >
> > In Section 2.5.1.2 Register values replace the description of
> > DW_OP_regval_type with the following:
> >
> > 
> > The DW_OP_regval_type operation provides the contents of a given
> > register interpreted as a value of a given type. The first operand
> > is an unsigned LEB128 number, which identifies a register whose
> > contents is to be pushed onto the stack. The second operand is an
> > unsigned LEB128 number that represents t

Re: [Dwarf-discuss] ISSUE: CPU vector types.

[Kyle Huey via Dwarf-discuss]

On Tue, Apr 4, 2023 at 3:09 AM Pedro Alves  wrote:

> This is introducing the need for the attribute.  The audience is the
> committee, not DWARF spec readers,
> because none of this text would end up in the spec.  It is just setting
> the ground.
>

Ah, ok, never mind then.

- Kyle
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] performance tools and inverted location lists

[Kyle Huey via Dwarf-discuss]

On Fri, Jun 16, 2023 at 8:28 AM Ben Woodard via Dwarf-discuss <
dwarf-discuss@lists.dwarfstd.org> wrote:

> I was looking at 2 level location tables
> https://dwarfstd.org/issues/140906.1.html and can see how it could
> improve things there. One thing that I noticed about it was that it was
> based on some prior art done in HP-UX.
>
> A nut that I have been trying to crack for a few years but haven't really
> figured out comes from the performance tools, binary analysis people and I
> could even make use of it for my ABI work. They would really like something
> that I've been calling "inverted location lists". Location lists are
> basically a function (yes I know it is technically a relation but let's not
> quibble about that right now):
>
> f( PC, "variable") -> location description
>
> This totally makes sense in the context of a debugger where the context of
> the lookup includes the PC, and the user inputs the symbolic name of the
> variable. The point that I try to make is DWARF is not just for debuggers
> anymore. A much broader range of consumers is emerging.
>
> In the context of performance tools, they need:
>
> f( PC,  location) -> symbolic name
>
> The reason why I specify "concrete" location is because it is never going
> to be a literal constant or an implicit or a DWARF expression or any of the
> other things along those lines. It is going to be an address or a register,
> things that have concrete existence.
>

lldb has a version of this function implemented in the functions
StackFrame::GuessValueForAddress/GuessValueForRegisterAndOffset.

- Kyle

> Here are the use cases for this that have come up:
>
>- For performance tools, they have some piece of code which is not
>performing as it should. The tools show them that they are getting a large
>number of cache misses between this PC and another PC. They can disassemble
>the machine code and they see a handful of instructions. To be able to
>figure out what is wrong they need to associate the operands of these
>instructions to what they refer to at the source level. Doing this
>disassembled optimized machine code and then associating the data accesses
>within it back to constructs within the source language really puts this
>level of analysis is the domain of experts and must be done by hand.
>Because of the limited information currently provided by DWARF it is not
>something that can be coded into a tool.
>- Within ABI analysis there are a few cases where compilation options
>can change the calling convention of a function sufficiently enough that it
>can create an ABI mismatch. The simple example that illustrates the point
>is SSE. You can pass vector operands by value in vector registers. However,
>if you compile a program without SSE then it is passed by reference. So in
>the ABI testing program, if I take the locations of the formal parameters
>at the call site and then use an inverted location list to look up what
>variables exist at those locations in the called function, if they don't
>match then I have an ABI problem.
>- One binary analysis use case is quite similar to the ABI use case,
>they see a call instruction, they can look up the target of the call to
>find the function prototype. From that they can figure out where the
>parameters to the function call are at the time of the call. However, they
>want to know in the calling scope what symbolic name in the source language
>for the parameter in a particular location at the time of the call. Simple
>example: find all the examples of "dlopen" calls - what library name is
>being opened, is it something that we can determine from static analysis or
>is it something read from a config file...
>
> What all three of us have tried doing with limited success is taking the
> location list data and then inverting the mappings. The problem is that
> this is far from complete and generating that data is very computationally
> expensive.
>
> I've looked for prior art to see if someone had come up with a solution to
> this but I haven't found any. The allusion to the HP-UX two level line maps
> is something that I had never come across before. Since the combined
> experience of this group is much deeper than what I have, does anybody know
> of any prior art that I can use as a basis for a DWARF feature enhancement
> issue?
>
> -ben
>
>
> --
> Dwarf-discuss mailing list
> Dwarf-discuss@lists.dwarfstd.org
> https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss
>
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Proposal: Error: DW_OP_entry_value description and examples

[Kyle Huey via Dwarf-discuss]

On Tue, Aug 8, 2023 at 9:22 AM Metzger, Markus T via Dwarf-discuss <
dwarf-discuss@lists.dwarfstd.org> wrote:

> >### Section 2.5.1.7 Special Operations, p.37
> >
> >The first sentence of the description of DW_OP_entry_value reads:
> >
> >The DW_OP_entry_value operation pushes the value that the described
> >location held upon entering the current subprogram.
> >
> >A DWARF expression does not describe a location, so this should read:
> >
> >The DW_OP_entry_value operation evaluates an expression or register
> >location description as if it had been evaluated upon entering the
> >current subprogram, and pushes the value of the expression or content
> >of the register, respectively.
>
> Aren't register location descriptions the only thing that, given
> appropriate
> unwind rules, can actually be implemented?
>
> Should this be restricted to only allow register location descriptions?
>

There exist debuggers (e.g. Pernosco https://pernos.co/about/overview/)
that are capable of handling non-register location descriptions in the
past, so I would oppose any spec limitation on what DW_OP_entry_value can
evaluate.

- Kyle


> Regards,
> Markus.
> Intel Deutschland GmbH
> Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
> Tel: +49 89 99 8853-0, www.intel.de 
> Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva
> Chairperson of the Supervisory Board: Nicole Lau
> Registered Office: Munich
> Commercial Register: Amtsgericht Muenchen HRB 186928
>
> --
> Dwarf-discuss mailing list
> Dwarf-discuss@lists.dwarfstd.org
> https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss
>
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Facing a issue in contents of dwarf debug info section in an ELF file

[Kyle Huey via Dwarf-discuss]

Most likely the .debug_info section is SHF_COMPRESSED. Try compiling with
-gz=none.

- Kyle

On Mon, Sep 25, 2023, 1:36 PM Rishi Raj via Dwarf-discuss <
dwarf-discuss@lists.dwarfstd.org> wrote:

> 
>
> I am trying to implement a dwarf parser in C++ without using any external
> dependencies. As mentioned in dwarf5 standard, debug info first 4 bytes or
> 12 bytes denotes the unit length Basically this:
>
> unit_length (initial length)
> A 4-byte or 12-byte unsigned integer representing the length of the3
> .debug_info contribution for that compilation unit, not including the length 
> field itself. In the 32-bit DWARF format, this is a 4-byte unsigned integer 
> (which must be less than 0xfff0); in the 64-bit DWARF format, this 
> consists of the 4-byte value 0x followed by an 8-byte unsigned 
> integer that gives the actual length (see Section 7.4 on page 196).
>
> When I am dumping the .debug_info section hexadecimally using objdump I am
> getting this(see readelf output below).
>
> objdump -s -j .debug_info hello.o
>
> hello.o: file format elf64-x86-64
>
> Contents of section .debug_info:
>
>   0100  9a00   
>  0010 0100  789c9bc6 c0c0c0ca  x...
>  0020 c0c801a4 18984084 2c031a10 42623372  ..@.,...Bb3r
>  0030 b0832916 0805d1c6 c804e5b1 4178ac20  ..).Ax.
>  0040 8a998535 33af04a8 8115498e 05aa2002  ...53.I... .
>  0050 8bf18c73 58131918 99394172 4c137318  ...sX9ArL.s.
>  0060 180011e5 0560
>
> So according to this, the length should be 0x0100, but the actual
> length is 0x96.(see readelf output below)
> readelf -wi hello.o
> Contents of the .debug_info section:
>
> Compilation Unit @ offset 0:
> Length: 0x96 (32-bit)
> Version: 5 Unit Type: DW_UT_compile (1) Abbrev Offset: 0
> Pointer Size: 8
>
> I know I am missing something basic, but even after reading the standards
> for many times. I am unable to find my mistake. One more thing, I searched
> for some basic dwarf parsers so that I could understand what they are doing
> but could not find any. All of the parsers were big libraries, which was
> difficult to understand. If any of you can at least provide some readable
> and understandable parser code, It will be helpful too.
>
> ELF header dump:
>
> ELF Header:
>   Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
>   Class: ELF64
>   Data:  2's complement, little endian
>   Version:   1 (current)
>   OS/ABI:UNIX - System V
>   ABI Version:   0
>   Type:  REL (Relocatable file)
>   Machine:   Advanced Micro Devices X86-64
>   Version:   0x1
>   Entry point address:   0x0
>   Start of program headers:  0 (bytes into file)
>   Start of section headers:  3048 (bytes into file)
>   Flags: 0x0
>   Size of this header:   64 (bytes)
>   Size of program headers:   0 (bytes)
>   Number of program headers: 0
>   Size of section headers:   64 (bytes)
> --
> Dwarf-discuss mailing list
> Dwarf-discuss@lists.dwarfstd.org
> https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss
>
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

[Dwarf-discuss] Coroutines

[Kyle Huey via Dwarf-discuss]

Has anyone proposed or discussed DWARF structures to represent coroutines
or similar constructs? I skimmed the open issues list and didn't see
anything relevant.

- Kyle
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Coroutines

[Kyle Huey via Dwarf-discuss]

On Wed, Feb 21, 2024 at 9:12 AM Adrian Prantl  wrote:

> Can you clarify what kind of information you are interested in? Are you
> talking about representing variables inside of coroutines, function
> pointers to coroutines, linetables for coroutines, ...?
>
> LLDB's Swift plugin for example supports Swift async functions (which are
> effectively coroutines) but does so without adding any new extensions to
> DWARF. Here's a recent talk from last year's LLVM dev meeting on that
> topic: https://www.youtube.om/watch?v=g4fei6Vl7o8
> <https://www.youtube.com/watch?v=g4fei6Vl7o8>
>

The async control flow parts of this (backtraces and stepping) are what I
am particularly interested in. Variables seem straightforward since DWARF
location expressions are already generic enough to let you point to heap
allocated storage for variables that survive across yield points. Your talk
is a good demo of the behavior I'd like to achieve (for Rust specifically).
A few technical questions:

How do you "identify" a particular chain of funclet invocations that go
together? I assume this is done based on the pointer identity of the
async_context.

And the logic to understand what the async_context is and where it's found
at function entry points is hardcoded into the Swift plugin?

How do you handle pointer reuse on async_contexts?

- Kyle


> -- adrian
>
>
>
>
>
> > On Feb 20, 2024, at 11:47 AM, Kyle Huey via Dwarf-discuss <
> dwarf-discuss@lists.dwarfstd.org> wrote:
> >
> > Has anyone proposed or discussed DWARF structures to represent
> coroutines or similar constructs? I skimmed the open issues list and didn't
> see anything relevant.
> >
> > - Kyle
>
>
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Coroutines

[Kyle Huey via Dwarf-discuss]

On Tue, Feb 27, 2024 at 1:07 PM Adrian Prantl  wrote:

>
>
> On Feb 27, 2024, at 10:32 AM, Kyle Huey  wrote:
>
> On Wed, Feb 21, 2024 at 9:12 AM Adrian Prantl  wrote:
>
>> Can you clarify what kind of information you are interested in? Are you
>> talking about representing variables inside of coroutines, function
>> pointers to coroutines, linetables for coroutines, ...?
>>
>> LLDB's Swift plugin for example supports Swift async functions (which are
>> effectively coroutines) but does so without adding any new extensions to
>> DWARF. Here's a recent talk from last year's LLVM dev meeting on that
>> topic: https://www.youtube.om/watch?v=g4fei6Vl7o8
>> <https://www.youtube.com/watch?v=g4fei6Vl7o8>
>>
>
> The async control flow parts of this (backtraces and stepping) are what I
> am particularly interested in. Variables seem straightforward since DWARF
> location expressions are already generic enough to let you point to heap
> allocated storage for variables that survive across yield points. Your talk
> is a good demo of the behavior I'd like to achieve (for Rust specifically).
> A few technical questions:
>
> How do you "identify" a particular chain of funclet invocations that go
> together? I assume this is done based on the pointer identity of the
> async_context.
>
>
> Each async_context has a continuation pointer pointing to the next funclet
> that is going to be executed when the current coroutine returns.
>

Sure, but if you have multiple async_contexts executing the same sequence
of funclets you have to condition stopping at the next funclet on having
the right async_context, no?

Put another way (forgive my pseudo code), if you have

function silly_print() {
delay = rand();
// Point 1
async_sleep(delay).await;
// Point 2
printf("Hello" );
}

function do_stuff() {
runtime.async_dispatch(silly_print());
runtime.async_dispatch(silly_print());
}

When stepping from point 1 to point 2, the breakpoint on the post-await
funclet at point 2 has to be conditional on being on the async_context
being the same async_context the debugger was using at point 1, right? And
then you have to be able to determine what the "same async_context" is.

- Kyle


> And the logic to understand what the async_context is and where it's found
> at function entry points is hardcoded into the Swift plugin?
>
>
> Yes, there is a Swift-specific unwinder plugin in LLDB that does this.
> There is also a custom stepping plan there.
>
>
> How do you handle pointer reuse on async_contexts?
>
>
> If I'm understanding the question correctly: there is a heap async_context
> object for every continuation that is on the schedule at any time. I don't
> think we can prevent a later continuation to be allocated in the same place
> as a previous one, but since they can never appear at the same time, this
> also isn't a problem.
>
> -- adrian
>
>
> - Kyle
>
>
>> -- adrian
>>
>>
>>
>>
>>
>> > On Feb 20, 2024, at 11:47 AM, Kyle Huey via Dwarf-discuss <
>> dwarf-discuss@lists.dwarfstd.org> wrote:
>> >
>> > Has anyone proposed or discussed DWARF structures to represent
>> coroutines or similar constructs? I skimmed the open issues list and didn't
>> see anything relevant.
>> >
>> > - Kyle
>>
>>
>
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

[Dwarf-discuss] Location expressions for partially-optimized-out structs

[Kyle Huey via Dwarf-discuss]

This originates from https://github.com/rust-lang/rust/issues/46698

This Rust program has a function that takes a struct type as an
argument. The compiler is clever enough to rewrite the function so
that a subset of the struct's fields are passed as arguments at the
machine code level (in this case, the struct  consists of two 8 byte
fields and only the second is needed). LLVM emits the following
DW_AT_location expression for the argument:

DW_OP_piece 0x8, DW_OP_reg5 RDI, DW_OP_piece 0x8

The DWARF spec does not explicitly contemplate this sequence, merely saying:

The DW_OP_piece operation takes a single operand, which is an unsigned
LEB128 number. The number describes the size in bytes of the piece of the
object referenced by the preceding simple location description. If the piece is
located in a register, but does not occupy the entire register, the placement of
the piece within that register is defined by the ABI.

The text for DW_OP_bit_piece, on the other hand, does explicitly
contemplate this, containing the language:

If the location description is empty, the offset doesn’t matter and
the DW_OP_bit_piece operation describes a piece consisting of the
given number of bits whose values are undefined.

I propose adding similar language to the description of DW_OP_piece. e.g.

The DW_OP_piece operation takes a single operand, which is an unsigned
LEB128 number. The number describes the size in bytes of the piece of the
object referenced by the preceding simple location description. If the
location description is empty the DW_OP_piece operation describes a
piece consisting of the given number of bytes whose values are
undefined. If the piece is
located in a register, but does not occupy the entire register, the placement of
the piece within that register is defined by the ABI.

(In practical use, gdb simply errors out on that location expression,
while lldb substitutes the undefined bytes with zeroes, neither of
which are correct IMO.)

- Kyle
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Location expressions for partially-optimized-out structs

[Kyle Huey via Dwarf-discuss]

On Tue, Feb 27, 2024 at 9:35 PM Cary Coutant  wrote:
>
> > The text for DW_OP_bit_piece, on the other hand, does explicitly
> > contemplate this, containing the language:
> >
> > If the location description is empty, the offset doesn’t matter and
> > the DW_OP_bit_piece operation describes a piece consisting of the
> > given number of bits whose values are undefined.
> >
> > I propose adding similar language to the description of DW_OP_piece. e.g.
>
> This has been clarified for DWARF 6; see:
>
> https://dwarfstd.org/issues/181205.1.html
>
> -cary

Mmm, the location where that was inserted seems odd. I specifically
looked at the latest working draft for something like this before
sending my email and didn't find it, although I do see if now that I
know the exact text to search for. Is there a reason it's not in the
DW_OP_piece specific subsection (especially since the language in the
DW_OP_bit_piece subsection is still there), or in the first paragraph
in 2.6.1.2?

- Kyle
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Location expressions for partially-optimized-out structs

[Kyle Huey via Dwarf-discuss]

On Tue, Feb 27, 2024 at 9:35 PM Cary Coutant  wrote:
>
> > The text for DW_OP_bit_piece, on the other hand, does explicitly
> > contemplate this, containing the language:
> >
> > If the location description is empty, the offset doesn’t matter and
> > the DW_OP_bit_piece operation describes a piece consisting of the
> > given number of bits whose values are undefined.
> >
> > I propose adding similar language to the description of DW_OP_piece. e.g.
>
> This has been clarified for DWARF 6; see:
>
> https://dwarfstd.org/issues/181205.1.html
>
> -cary

gmail claims this was rejected from your inbox last night even though
it did go through to the list, so resending below

~~~

Mmm, the location where that was inserted seems odd. I specifically
looked at the latest working draft for something like this before
sending my email and didn't find it, although I do see if now that I
know the exact text to search for. Is there a reason it's not in the
DW_OP_piece specific subsection (especially since the language in the
DW_OP_bit_piece subsection is still there), or in the first paragraph
in 2.6.1.2?

- Kyle
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Coroutines

[Kyle Huey via Dwarf-discuss]

On Wed, Feb 28, 2024 at 10:02 AM Adrian Prantl  wrote:
>
>
>
> On Feb 27, 2024, at 1:53 PM, Kyle Huey via Dwarf-discuss 
>  wrote:
>
> Put another way (forgive my pseudo code), if you have
>
> function silly_print() {
> delay = rand();
> // Point 1
> async_sleep(delay).await;
> // Point 2
> printf("Hello" );
> }
>
> function do_stuff() {
> runtime.async_dispatch(silly_print());
> runtime.async_dispatch(silly_print());
> }
>
> When stepping from point 1 to point 2, the breakpoint on the post-await 
> funclet at point 2 has to be conditional on being on the async_context being 
> the same async_context the debugger was using at point 1, right? And then you 
> have to be able to determine what the "same async_context" is.
>
>
>
> LLDB implements this in a custom stepping plan that creates a conditional 
> breakpoint where two async contexts are compared: First, there's the active 
> async context at the time of the step. Next, the active async context at the 
> time the breakpoint is hit. If the two async contexts are equal, then the 
> breakpoint is discarded and the step has completed.
>
> -- adrian

Right, that makes sense. What is "are equal" here? Just a pointer comparison?

- Kyle
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Coroutines

[Kyle Huey via Dwarf-discuss]

Hi Felipe,

Any chance you could point me in the right direction?

- Kyle

On Tue, Oct 1, 2024 at 9:41 AM Adrian Prantl  wrote:
>
> CC'ing Felipe, who is currently reworking a lot of this code and is much 
> deeper in the subject matter than me.
>
> -- adrian
>
> > On Oct 1, 2024, at 8:42 AM, Kyle Huey  wrote:
> >
> > On Wed, Feb 28, 2024 at 10:02 AM Adrian Prantl  wrote:
> >> LLDB implements this in a custom stepping plan that creates a conditional 
> >> breakpoint where two async contexts are compared: First, there's the 
> >> active async context at the time of the step. Next, the active async 
> >> context at the time the breakpoint is hit. If the two async contexts are 
> >> equal, then the breakpoint is discarded and the step has completed.
> >
> > Out of curiosity where does this code actually live?
> >
> > - Kyle
>
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Coroutines

[Kyle Huey via Dwarf-discuss]

On Wed, Feb 28, 2024 at 10:02 AM Adrian Prantl  wrote:
> LLDB implements this in a custom stepping plan that creates a conditional 
> breakpoint where two async contexts are compared: First, there's the active 
> async context at the time of the step. Next, the active async context at the 
> time the breakpoint is hit. If the two async contexts are equal, then the 
> breakpoint is discarded and the step has completed.

Out of curiosity where does this code actually live?

- Kyle
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Representing captured `this` in C++ lambdas

[Kyle Huey via Dwarf-discuss]

On Wed, Apr 16, 2025 at 2:10 PM David Blaikie  wrote:
>
>
>
> On Wed, Apr 16, 2025 at 8:59 AM Kyle Huey via Dwarf-discuss 
>  wrote:
>>
>> This may be more of an implementation question than a spec question,
>> but this seems like the place to have the discussion regardless.
>>
>> C++ debuggers that support expression evaluation need to identify the
>> `this` pointer argument of member functions because C++ permits
>> unqualified access to member variables. DWARF 3 introduced the
>> DW_AT_object_pointer attribute for this purpose, and modern versions
>> of clang and gcc emit it. lldb actually uses this attribute when
>> present to find the this pointer argument, falling back to checking to
>> see if the first formal parameter to the function looks "this-like",
>> while gdb appears to use a name lookup at all times. Regardless, these
>> end up being interoperable for normal member functions.
>>
>> lambdas that capture the `this` of a member function present the same
>> need for debuggers, as unqualified accesses within the lambda can also
>> access member variables of `this`. Both clang and gcc desugar lambdas
>> to an anonymous struct type whose members are the captured variables,
>> with an operator() implementation that contains the body of the
>> lambda. They, however, different in their representation of a captured
>> `this` pointer.
>>
>> In clang's case the DWARF for the operator() contains a
>> DW_TAG_formal_parameter for a `this` parameter which is a pointer to
>> the anonymous struct type. That anonymous struct then contains its own
>> `this` member which is the captured `this`. lldb then contains code in
>> GetLambdaValueObject that recognizes this "double this" pattern and
>> deals with it appropriately.
>>
>> In gcc's case the DWARF for the operator() contains a
>> DW_TAG_formal_parameter for a `__closure` parameter which is a pointer
>> to the anonymous struct type. That anonymous struct contains a
>> `__this` member which is the captured `this`. Additionally, gcc emits
>> a DW_TAG_variable inside the operator() named `this` with the
>> appropriate DW_AT_location that traverses the anonymous struct (as it
>> does for all captured variables).
>>
>> In both cases the compilers emit a DW_AT_object_pointer on the
>> operator() pointing to the anonymous struct pointer parameter.
>>
>> This results in neither debugger being able to understand the output
>> of the opposite compiler. gdb cannot understand what clang has emitted
>> because it looks at the `this` parameter (which points to the
>> anonymous struct) and lldb cannot understand what gcc has emitted
>> because it expects the "double this" pattern. This is also annoying
>> for third party debuggers (like the one I maintain) because we need to
>> recognize and explicitly support both patterns.
>>
>> I haven't done any research into why the compilers chose to emit what
>> they do, but it seems to me[0] like things would be better if clang
>> copied gcc's "repeat the captured variables as locals inside
>> operator()" behavior (which would make gdb understand clang binaries)
>> and then both compilers switched their DW_AT_object_pointers to point
>> to the captured `this` if and only if it exists (which would make lldb
>> understand gcc binaries), ignoring the anonymous compiler-generated
>> struct entirely. Then lambdas that capture `this` would look like
>> member functions to debuggers and "just work" without any special
>> lambda-aware code.
>>
>> This would require at least some clarification in the spec since the
>> subprogram's DW_AT_object_pointer would point to a local, not a
>> parameter, and would point to an object of a different type than the
>> DW_TAG_class_type containing the subprogram (or would exist on a
>> subprogram not contained in a class at all if compilers elided the
>> anonymous struct from DWARF entirely).
>>
>> Any thoughts?
>
>
> As a clang developer, I've some bias for the Clang representation here

Sure.

>  - and lambdas are classes (per the C++ spec) so it still makes sense to me 
> that op() is a member function, though, yeah, having its "this" pointer given 
> another name since users can't refer to it by that name.

Do you object to anything I proposed other than removing the
representation of the anonymous class compilers generate for lambdas?
Because that's not really essential to what I want to do if we're fine
with DW_AT_object_pointer pointing to a variable that doesn't s

Re: [Dwarf-discuss] Representing captured `this` in C++ lambdas

[Kyle Huey via Dwarf-discuss]

On Wed, Apr 23, 2025 at 3:14 PM David Blaikie  wrote:
>
>
>
> On Wed, Apr 23, 2025 at 2:53 PM Kyle Huey  wrote:
>>
>> On Wed, Apr 23, 2025 at 2:20 PM David Blaikie  wrote:
>> >>
>> >> Do you object to anything I proposed other than removing the
>> >> representation of the anonymous class compilers generate for lambdas?
>> >
>> >
>> > I'm not a /super/ fan of introducing a bunch of locals in addition to the 
>> > member descriptions - it'll be a bunch of extra DWARF that'd be nice to 
>> > avoid if we can...
>>
>> Yeah, that would be the reason to get rid of the representation of the
>> anonymous struct. Then we're just converting members into locals
>> rather than duplicating anything. Even if you really want to keep the
>> class itself, the members could be dropped.
>
>
> Except it's likely users will want to inspect the state of a lambda in some 
> situations. They get passed around, stored (in std::functions or similar 
> type-erased things, often), etc in many cases & may be important to know what 
> they represent when not near/in a call to the lambda. So having the members 
> described seems important.

Mmm, yes, ok.

- Kyle

>>
>> > But putting object_pointer on the class member that stores "this" seems 
>> > problematic since that's effectively at the same scope as the real object 
>> > pointer - it'd be awkward to say there's two "this" at the same scope and 
>> > have to say that the member variable "this" shadows the real "this" in 
>> > some way.
>> >
>> > And then you want the captured variables to be in a scope that is inside 
>> > the "this" scope so they override unqualified lookup for any names that 
>> > are also members of "this"...
>> >
>> > So, yeah, I get why you/gcc developers arrived where they did. I wouldn't 
>> > mind some size analysis to see how bad the regression/cost is, that might 
>> > help inform whether it's worth trying to address it.
>>
>> Hmm. I could look at hacking something up to measure but it's not at
>> the top of my priority list.
>
>
> Fair.
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Representing captured `this` in C++ lambdas

[Kyle Huey via Dwarf-discuss]

On Wed, Apr 23, 2025 at 2:20 PM David Blaikie  wrote:
>>
>> Do you object to anything I proposed other than removing the
>> representation of the anonymous class compilers generate for lambdas?
>
>
> I'm not a /super/ fan of introducing a bunch of locals in addition to the 
> member descriptions - it'll be a bunch of extra DWARF that'd be nice to avoid 
> if we can...

Yeah, that would be the reason to get rid of the representation of the
anonymous struct. Then we're just converting members into locals
rather than duplicating anything. Even if you really want to keep the
class itself, the members could be dropped.

> But putting object_pointer on the class member that stores "this" seems 
> problematic since that's effectively at the same scope as the real object 
> pointer - it'd be awkward to say there's two "this" at the same scope and 
> have to say that the member variable "this" shadows the real "this" in some 
> way.
>
> And then you want the captured variables to be in a scope that is inside the 
> "this" scope so they override unqualified lookup for any names that are also 
> members of "this"...
>
> So, yeah, I get why you/gcc developers arrived where they did. I wouldn't 
> mind some size analysis to see how bad the regression/cost is, that might 
> help inform whether it's worth trying to address it.

Hmm. I could look at hacking something up to measure but it's not at
the top of my priority list.

- Kyle

>>
>> Because that's not really essential to what I want to do if we're fine
>> with DW_AT_object_pointer pointing to a variable that doesn't share a
>> containing type.
>
>
> Yeah, this general idea I think I'm down with - if it generalizes to 
> scope-based "this" in other languages/features, etc, that seems good.
>
>>
>>
>> Now that I've thought about it a bit more I think that is not
>> particularly weird. Rust for example allows
>>
>> fn member(self: Rc, arg1: Foo, ...)
>>
>> where the type of the receiver can be certain things that are
>> convertible to the containing object but are definitely not the
>> containing object. rustc doesn't emit DW_AT_object_pointer today but
>> it should.
>>
>> - Kyle
>>
>>
>> > Introducing a bunch of locals that expose the right names/types - seems OK 
>> > to me.
>> >
>> > Using object_pointer to refer to a local "this" could have other uses too 
>> > - some languages (I forget which ones) have a scope based object usage, 
>> > like "foo f; f.x();" -> "using (foo f) { x(); }" and so using 
>> > object_pointer on the scope to refer to the local could be used to support 
>> > that feature.
>>
>> - Kyle
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Representing vtables in DWARF for downcasting

[Kyle Huey via Dwarf-discuss]

On Wed, Apr 23, 2025 at 8:45 AM Michael Buch  wrote:
>
> Sounds like this is what
> https://github.com/llvm/llvm-project/pull/130255 is trying to achieve?

Yes, though that may be trying to achieve other things too (there's
some discussion of trying to go from the class definition in the DWARF
to the vtable pointer which I haven't considered and don't immediately
see a use case for).

- Kyle

> If we could simplify that part of LLDB that'd be great!
>
> On Wed, 23 Apr 2025 at 16:32, Kyle Huey via Dwarf-discuss
>  wrote:
> >
> > Consider the following C++ program
> >
> > #include 
> >
> > class Base {
> > public:
> >   virtual const char* method1() = 0;
> >   void method2() {
> > printf("%s\n", method1());
> >   }
> > };
> >
> > class DerivedOne : public Base {
> >   virtual const char* method1() override {
> > return "DerivedOne";
> >   }
> > };
> >
> > template
> > class DerivedTwo : public Base {
> > public:
> >   DerivedTwo(T t) : t(t) {}
> > private:
> >   virtual const char* method1() override {
> > return t;
> >   }
> >   T t;
> > };
> >
> > template
> > class DerivedThree : public Base {
> > public:
> >   DerivedThree(T t) : t(t) {}
> > private:
> >   virtual const char* method1() override {
> > return t();
> >   }
> >   T t;
> > };
> >
> > int main() {
> >   DerivedOne d1;
> >   DerivedTwo d2("DerivedTwo");
> >   DerivedThree d3([]() {
> > return "DerivedThree";
> >   });
> >   d1.method2();
> >   d2.method2();
> >   d3.method2();
> >   return 0;
> > }
> >
> > If a debugger stops at method1, the DW_TAG_formal_parameter will tell
> > the debugger the type of `this` is Base. Downcasting to the derived
> > type is very useful for the programmer though, so both gdb and lldb
> > contain a feature to downcast based on the vtable pointer (the "print
> > object" and the "target.prefer-dynamic" settings in the respective
> > debuggers).
> >
> > The first part of this is straightforward. The DWARF for Base will
> > contain a member for the vtable pointer, and that plus knowledge of
> > how the ABI lays out vtables allows the debugger to effectively do a
> > dynamic_cast to obtain a pointer to the most derived object.
> > From there the vtable address is compared against the ELF symbol table
> > to find the mangled name of the vtable symbol.
> >
> > Then things begin to get hairy, the debugger demangles the mangled
> > name that exists in the ELF symbol table, chops off the "vtable for "
> > prefix on the demangled name, and searches for the type by name in the
> > DWARF. If it finds the type, it adjusts the type of the value and
> > prints it accordingly. But this text based matching doesn't always
> > work. There are no mangled names for types so the debugger's
> > demangling has to match the compiler's output character for character.
> >
> > In the example program I've provided, when using the respective
> > compilers, gdb can successfully downcast DerivedOne and DerivedThree
> > but not DerivedTwo. gdb fails because gcc emits the DW_TAG_class_type
> > with a DW_AT_name "DerivedTwo >" but libiberty
> > demangles the vtable symbol to "vtable for
> > DerivedTwo" and those do not match. lldb can only
> > successfully downcast DerivedOne. lldb appears to not handle classes
> > with template parameters correctly at all. And even if all of that
> > were fixed, libiberty and llvm disagree about how to demangle the
> > symbol for DerivedTwo's vtable, so the two ecosystems would not be
> > interoperable.
> >
> > Perhaps these are merely quality of implementation issues and belong
> > on the respective bug trackers, however, better representations are
> > possible. Rustc, for example, does not rely on the ELF symbol table
> > and demangled string matching. It emits a global variable in the DWARF
> > whose location is the address of the vtable. That variable has a
> > DW_AT_type pointing to a DW_TAG_class_type that describes the layout
> > of the vtable, and that type has a DW_AT_containing_type that points
> > to the type making use of that vtable.
> >
> > Any thoughts?
> >
> > - Kyle
> > --
> > Dwarf-discuss mailing list
> > Dwarf-discuss@lists.dwarfstd.org
> > https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

[Dwarf-discuss] Representing vtables in DWARF for downcasting

[Kyle Huey via Dwarf-discuss]

Consider the following C++ program

#include 

class Base {
public:
  virtual const char* method1() = 0;
  void method2() {
printf("%s\n", method1());
  }
};

class DerivedOne : public Base {
  virtual const char* method1() override {
return "DerivedOne";
  }
};

template
class DerivedTwo : public Base {
public:
  DerivedTwo(T t) : t(t) {}
private:
  virtual const char* method1() override {
return t;
  }
  T t;
};

template
class DerivedThree : public Base {
public:
  DerivedThree(T t) : t(t) {}
private:
  virtual const char* method1() override {
return t();
  }
  T t;
};

int main() {
  DerivedOne d1;
  DerivedTwo d2("DerivedTwo");
  DerivedThree d3([]() {
return "DerivedThree";
  });
  d1.method2();
  d2.method2();
  d3.method2();
  return 0;
}

If a debugger stops at method1, the DW_TAG_formal_parameter will tell
the debugger the type of `this` is Base. Downcasting to the derived
type is very useful for the programmer though, so both gdb and lldb
contain a feature to downcast based on the vtable pointer (the "print
object" and the "target.prefer-dynamic" settings in the respective
debuggers).

The first part of this is straightforward. The DWARF for Base will
contain a member for the vtable pointer, and that plus knowledge of
how the ABI lays out vtables allows the debugger to effectively do a
dynamic_cast to obtain a pointer to the most derived object.
>From there the vtable address is compared against the ELF symbol table
to find the mangled name of the vtable symbol.

Then things begin to get hairy, the debugger demangles the mangled
name that exists in the ELF symbol table, chops off the "vtable for "
prefix on the demangled name, and searches for the type by name in the
DWARF. If it finds the type, it adjusts the type of the value and
prints it accordingly. But this text based matching doesn't always
work. There are no mangled names for types so the debugger's
demangling has to match the compiler's output character for character.

In the example program I've provided, when using the respective
compilers, gdb can successfully downcast DerivedOne and DerivedThree
but not DerivedTwo. gdb fails because gcc emits the DW_TAG_class_type
with a DW_AT_name "DerivedTwo >" but libiberty
demangles the vtable symbol to "vtable for
DerivedTwo" and those do not match. lldb can only
successfully downcast DerivedOne. lldb appears to not handle classes
with template parameters correctly at all. And even if all of that
were fixed, libiberty and llvm disagree about how to demangle the
symbol for DerivedTwo's vtable, so the two ecosystems would not be
interoperable.

Perhaps these are merely quality of implementation issues and belong
on the respective bug trackers, however, better representations are
possible. Rustc, for example, does not rely on the ELF symbol table
and demangled string matching. It emits a global variable in the DWARF
whose location is the address of the vtable. That variable has a
DW_AT_type pointing to a DW_TAG_class_type that describes the layout
of the vtable, and that type has a DW_AT_containing_type that points
to the type making use of that vtable.

Any thoughts?

- Kyle
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

[Dwarf-discuss] Representing captured `this` in C++ lambdas

[Kyle Huey via Dwarf-discuss]

This may be more of an implementation question than a spec question,
but this seems like the place to have the discussion regardless.

C++ debuggers that support expression evaluation need to identify the
`this` pointer argument of member functions because C++ permits
unqualified access to member variables. DWARF 3 introduced the
DW_AT_object_pointer attribute for this purpose, and modern versions
of clang and gcc emit it. lldb actually uses this attribute when
present to find the this pointer argument, falling back to checking to
see if the first formal parameter to the function looks "this-like",
while gdb appears to use a name lookup at all times. Regardless, these
end up being interoperable for normal member functions.

lambdas that capture the `this` of a member function present the same
need for debuggers, as unqualified accesses within the lambda can also
access member variables of `this`. Both clang and gcc desugar lambdas
to an anonymous struct type whose members are the captured variables,
with an operator() implementation that contains the body of the
lambda. They, however, different in their representation of a captured
`this` pointer.

In clang's case the DWARF for the operator() contains a
DW_TAG_formal_parameter for a `this` parameter which is a pointer to
the anonymous struct type. That anonymous struct then contains its own
`this` member which is the captured `this`. lldb then contains code in
GetLambdaValueObject that recognizes this "double this" pattern and
deals with it appropriately.

In gcc's case the DWARF for the operator() contains a
DW_TAG_formal_parameter for a `__closure` parameter which is a pointer
to the anonymous struct type. That anonymous struct contains a
`__this` member which is the captured `this`. Additionally, gcc emits
a DW_TAG_variable inside the operator() named `this` with the
appropriate DW_AT_location that traverses the anonymous struct (as it
does for all captured variables).

In both cases the compilers emit a DW_AT_object_pointer on the
operator() pointing to the anonymous struct pointer parameter.

This results in neither debugger being able to understand the output
of the opposite compiler. gdb cannot understand what clang has emitted
because it looks at the `this` parameter (which points to the
anonymous struct) and lldb cannot understand what gcc has emitted
because it expects the "double this" pattern. This is also annoying
for third party debuggers (like the one I maintain) because we need to
recognize and explicitly support both patterns.

I haven't done any research into why the compilers chose to emit what
they do, but it seems to me[0] like things would be better if clang
copied gcc's "repeat the captured variables as locals inside
operator()" behavior (which would make gdb understand clang binaries)
and then both compilers switched their DW_AT_object_pointers to point
to the captured `this` if and only if it exists (which would make lldb
understand gcc binaries), ignoring the anonymous compiler-generated
struct entirely. Then lambdas that capture `this` would look like
member functions to debuggers and "just work" without any special
lambda-aware code.

This would require at least some clarification in the spec since the
subprogram's DW_AT_object_pointer would point to a local, not a
parameter, and would point to an object of a different type than the
DW_TAG_class_type containing the subprogram (or would exist on a
subprogram not contained in a class at all if compilers elided the
anonymous struct from DWARF entirely).

Any thoughts?

- Kyle

[0] At the risk of https://xkcd.com/927/
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss

Re: [Dwarf-discuss] Representing captured `this` in C++ lambdas

[Kyle Huey via Dwarf-discuss]

On Wed, Apr 16, 2025 at 3:15 PM Cary Coutant  wrote:
>
> Alternatively, what if DW_AT_object_pointer could also take a locdesc-class 
> with a location description? Then, in cases where 'this' is not a formal 
> parameter, it could provide a location description in the same manner as the 
> other captured locals you mentioned (presumably with an fbreg-based 
> computation).

I don't think that would work because it also needs to carry a DW_AT_type.

- Kyle

> Whichever way we want to go, it sounds like the DWARF spec should provide the 
> necessary guidance. This looks like a good start to a proposal—Kyle, would 
> you like to write one? (See https://dwarfstd.org/comment.html.)
>
> -cary
>
>
> On Wed, Apr 16, 2025 at 2:11 PM David Blaikie via Dwarf-discuss 
>  wrote:
>>
>>
>>
>> On Wed, Apr 16, 2025 at 8:59 AM Kyle Huey via Dwarf-discuss 
>>  wrote:
>>>
>>> This may be more of an implementation question than a spec question,
>>> but this seems like the place to have the discussion regardless.
>>>
>>> C++ debuggers that support expression evaluation need to identify the
>>> `this` pointer argument of member functions because C++ permits
>>> unqualified access to member variables. DWARF 3 introduced the
>>> DW_AT_object_pointer attribute for this purpose, and modern versions
>>> of clang and gcc emit it. lldb actually uses this attribute when
>>> present to find the this pointer argument, falling back to checking to
>>> see if the first formal parameter to the function looks "this-like",
>>> while gdb appears to use a name lookup at all times. Regardless, these
>>> end up being interoperable for normal member functions.
>>>
>>> lambdas that capture the `this` of a member function present the same
>>> need for debuggers, as unqualified accesses within the lambda can also
>>> access member variables of `this`. Both clang and gcc desugar lambdas
>>> to an anonymous struct type whose members are the captured variables,
>>> with an operator() implementation that contains the body of the
>>> lambda. They, however, different in their representation of a captured
>>> `this` pointer.
>>>
>>> In clang's case the DWARF for the operator() contains a
>>> DW_TAG_formal_parameter for a `this` parameter which is a pointer to
>>> the anonymous struct type. That anonymous struct then contains its own
>>> `this` member which is the captured `this`. lldb then contains code in
>>> GetLambdaValueObject that recognizes this "double this" pattern and
>>> deals with it appropriately.
>>>
>>> In gcc's case the DWARF for the operator() contains a
>>> DW_TAG_formal_parameter for a `__closure` parameter which is a pointer
>>> to the anonymous struct type. That anonymous struct contains a
>>> `__this` member which is the captured `this`. Additionally, gcc emits
>>> a DW_TAG_variable inside the operator() named `this` with the
>>> appropriate DW_AT_location that traverses the anonymous struct (as it
>>> does for all captured variables).
>>>
>>> In both cases the compilers emit a DW_AT_object_pointer on the
>>> operator() pointing to the anonymous struct pointer parameter.
>>>
>>> This results in neither debugger being able to understand the output
>>> of the opposite compiler. gdb cannot understand what clang has emitted
>>> because it looks at the `this` parameter (which points to the
>>> anonymous struct) and lldb cannot understand what gcc has emitted
>>> because it expects the "double this" pattern. This is also annoying
>>> for third party debuggers (like the one I maintain) because we need to
>>> recognize and explicitly support both patterns.
>>>
>>> I haven't done any research into why the compilers chose to emit what
>>> they do, but it seems to me[0] like things would be better if clang
>>> copied gcc's "repeat the captured variables as locals inside
>>> operator()" behavior (which would make gdb understand clang binaries)
>>> and then both compilers switched their DW_AT_object_pointers to point
>>> to the captured `this` if and only if it exists (which would make lldb
>>> understand gcc binaries), ignoring the anonymous compiler-generated
>>> struct entirely. Then lambdas that capture `this` would look like
>>> member functions to debuggers and "just work" without any special
>>> lambda-aware code.
>>>
>>> This would require at least some c

Re: [Dwarf-discuss] Representing vtables in DWARF for downcasting

[Kyle Huey via Dwarf-discuss]

On Wed, Apr 23, 2025 at 7:46 PM Cary Coutant  wrote:
>>
>> The first part of this is straightforward. The DWARF for Base will
>> contain a member for the vtable pointer, and that plus knowledge of
>> how the ABI lays out vtables allows the debugger to effectively do a
>> dynamic_cast to obtain a pointer to the most derived object.
>> From there the vtable address is compared against the ELF symbol table
>> to find the mangled name of the vtable symbol.
>
>
> This made me do a bit of research...
>
> The artificial member for the vtable pointer appears to be a DWARF extension 
> requested as far back as 2003 and implemented in 2009:
>
>https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11208

That was an interesting read. I particularly enjoyed this bit in the
original issue, written in 2002.

"This one isn't really necessary.  The ABI specifies where the vtable
 pointer will be, and GDB is quite capable of using that knowledge to
 identify the runtime type."

And yet here we are 23 years later with examples where gdb et al fail
to correctly identify the runtime type :)

> But I can't find any relevant discussion on the DWARF mailing lists, until a 
> question arose about that very member in 2022:
>
>https://dwarfstd.org/pipermail/dwarf-discuss/2022-February/002127.html
>
> It seems to me that, given the apparent need for this information in the 
> DWARF info, we should have addressed it in DWARF by now. I suspect the DWARF 
> committee's position was (or would have been) that the ABI tells you how to 
> find the vtable so it doesn't need to be explicitly recorded in the DWARF 
> info. But if both GCC and LLVM have decided it's useful enough (and there's 
> discussion about that point in the original PR that 11208 spun off from), 
> then we should discuss it. Otherwise, we risk having different toolchains 
> adopt different solutions. (GCC and LLVM appear to have avoided that through 
> careful consideration of what the other project was doing.) The argument in 
> PR 11208 is that it's /legal/ in DWARF to do this, so no new DWARF feature 
> was requested.
>
> The request in PR 11208 was for three things:
>
> > 1) I'd like to be able to locate the vtable pointer in the class
> >structure so that the debugger knows that the hole in the apparent
> >layout is not padding.
> >
> > 2) I'd like to know the type of the target of the vtable pointer, so
> >that if the user asks to see it they see something sane.
> >
> > 3) I'd like to be able to find a specific virtual functions entry in
> >the vtable, however I believe that this information will be best
> >expressed as a property of the function, not directly of the class
> >or vtable. DWARF3 has the DW_AT_vtable_elem_location attribute for
> >precisely this information. gcc should generate that too.
> >
> >Quoting the DWARF spec again :-
> >  An entry for a virtual function also has a
> >  DW_AT_vtable_elem_location attribute whose value contains a
> >  location description yielding the address of the slot for the
> >  function within the virtual function table for the enclosing
> >  class. The address of an object of the enclosing type is pushed
> >  onto the expression stack before the location description is
> >  evaluated.
>
> Point #1 is satisfied with an artificial member whose data_member_location is 
> the offset of the vtable pointer.

Right.

> I'm not clear how Point #2 was addressed.

I don't think it was.

> Point #3 was addressed via the vtable_elem_location attribute.
>
> Looking at the DWARF generated by GCC (and I'm guessing LLVM does the same), 
> I see vtable_elem_location attributes that look like this:
>
> <1b8>   DW_AT_vtable_elem_location: 2 byte block: 10 0 (DW_OP_constu: 0)
>
> This is not correct DWARF! It's supposed to be a location description, and 
> this is merely a DWARF expression that evaluates to an offset relative to the 
> vtable pointer. The description of the attribute says that address of an 
> object of the enclosing type is pushed onto the expression stack, so there 
> really ought to be a DW_OP_deref to get the vtable pointer on the stack, 
> followed by the DW_OP_constu and DW_OP_add.
>
> Now if we compare this to DW_AT_data_member_location, we see that one valid 
> form for that attribute is an integer constant providing the offset of the 
> data member. But even there, if the attribute has a location expression, it 
> should compute an actual address, not just deliver the offset.
>
> It would seem an obvious and useful extension to DWARF to allow 
> DW_AT_vtable_elem_location to take a constant class form that provides the 
> offset relative to the start of the vtable, so an acceptable form of the 
> attribute might be:
>
> <1b8>   DW_AT_vtable_elem_location: 0   # (using a constant class form)
>
> There's still the question of what do we do about the form GCC is already 
> emitting (and has been emitting since 2009)? Make it legal? Let it slide and 
> 

Re: [Dwarf-discuss] Representing captured `this` in C++ lambdas

[Kyle Huey via Dwarf-discuss]

On Wed, Apr 16, 2025 at 3:15 PM Cary Coutant  wrote:
> Whichever way we want to go, it sounds like the DWARF spec should provide the 
> necessary guidance. This looks like a good start to a proposal—Kyle, would 
> you like to write one? (See https://dwarfstd.org/comment.html.)

I will write something up later this month.

- Kyle
-- 
Dwarf-discuss mailing list
Dwarf-discuss@lists.dwarfstd.org
https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss