Re: [lldb-dev] Resolving dynamic type based on RTTI fails in case of type names inequality in DWARF and mangled symbols

[Anton Gorenkov via lldb-dev]

Tamas, Greg, thank you, I got the idea how it should work without 
accelerator tables, but I still cannot figure out how to use/update the 
existing accelerator tables. So let me walk trough it once again:
  1. It is necessary to perform lookup by mangled name (as all we 
initially have is mangled "vtable for ClassName"-symbol).
  2. All the existing apple accelerator tables (e.g. apple_types) have 
demangled and unqualified names as a key.
  3. It is not always possible to get the original demanled type name 
by the mangled one (e.g. for templates parametrized with enums the 
demangled one is Impl<(TagType)0> vs original Impl, but 
there are more complex cases).


Thus, I don't see how adding DW_AT_linkage_name to vtable member of 
class (or even to class itself) could help, as it still won't be 
possible to resolve DIE by the mangled type name. However possible 
solutions are:
  1. To generate a separate accelerator table: mangled name for vtable 
member of a class => DIE;
  2. Build index on startup iterating through the apple_types and 
gather the map mangled name => DIE;


Greg, did you mean some of these or something else?

Thanks,
Anton.

19.12.2017 19:39, Greg Clayton wrote:
I agree with Tamas. The right way to do this it to add the 
DW_AT_linkage_name to the class. Apple accelerator tables have many 
different forms, but one is a mapping of type name to exact DIE offset 
(in the __DWARF_ segment in the __apple_types section). If the mangled 
name was added to the class, then the apple accelerator tables would 
have it. So when a lookup happens with these tables around, we do a 
very quick hash lookup, and we find the exact DIE (or DIEs) we need. 
Entries for classes in the Apple accelerator tables have both the 
mangled and raw class name as entries pointing to the same DIE since 
lookups don't usually happen via mangled names. LLDB also knows how to 
pull names apart and search correctly, so if someone tries to lookup a 
type with "a::b::MyClass", we will chop that up into "MyClass" and do 
a lookup on that. We might get many many different "MyClass" results 
back (a::c::MyClass, ::MyClass, b::MyClass), but then we cull those 
down by making sure any matches have a matching decl context of 
"a::b::". For mangled names, it is easy and just a direct lookup.


The apple accelerator tables are only enabled for Darwin target, but 
there is nothing to say we couldn't enable these for other targets in 
ELF files. It would be a quick way to gauge the performance 
improvement that these accelerator tables provide for linux. Currently 
linux will completely index the DWARF, but it will load the DWARF, 
index it, and unload the DWARF so we don't hog memory for things we 
don't need loaded yet. We must manually index the DWARF because the 
DWARF accelerator tables are really not accelerator tables, they are 
random indexes of related data (names in no particular order, 
addresses in or particular order). These tables are also not complete 
so no debugger can rely on them. For example ".debug_pubtypes" is for 
"public" types only. ".debug_pubnames" is a random name table with 
only public functions (no static functions or functions in anonymous 
namespaces). So the DWARF accelerator tables can't be used by debuggers.


There is now a modified version of the Apple accelerator tables in the 
DWARF standard that can provide the same data as the Apple versions, 
but I don't believe anyone has added this support to any compilers 
yet. So for simplicity, we can try things out with the Apple 
accelerator tables and see how things go.


Another solution involves using llvm-dsymutil, a DWARF linker that is 
used on Apple platforms. It is a tool that is normally run on 
executables where the DWARF is left in the .o files and linked later 
into final DWARF files. This tool also has a "--update" option that 
take a linked dSYM file and updates the accelerator tables in case 
they change over time, or in case an older version of llvm-dsymutil 
didn't add everything that was needed to the tables due to a bug. So 
another way we can try this out is to modify the llvm-dsymutil to work 
with ELF files and have it generate and add the Apple accelerator 
tables to the ELF files. This is nice because it allows us to use 
DWARF that is generated by any compiler (no need for the compiler to 
support making the accelerator tables). This would a great way to try 
out the accelerator tables without requiring compiler changes.


The short term solution is to validate that the Apple accelerator 
tables work and do speed debugging up by a large amount. The long term 
solution is to have clang start emitting the new DWARF accelerator 
tables and modify LLDB to support and use those tables.


Let me know if there are any questions on any of this.

Greg Clayton

On Dec 19, 2017, at 5:35 AM, Tamas Berghammer via lldb-dev 
mailto:lldb-dev@lists.llvm.org>> wrote:


Hi,

I thought most compiler still emits DW_AT_MIPS_linkage_name instead 
of t

Re: [lldb-dev] Resolving dynamic type based on RTTI fails in case of type names inequality in DWARF and mangled symbols

[Anton Gorenkov via lldb-dev]

19.12.2017 23:12, Greg Clayton wrote:



On Dec 19, 2017, at 12:33 PM, Anton Gorenkov > wrote:


Tamas, Greg, thank you, I got the idea how it should work without 
accelerator tables, but I still cannot figure out how to use/update 
the existing accelerator tables. So let me walk trough it once again:
  1. It is necessary to perform lookup by mangled name (as all we 
initially have is mangled "vtable for ClassName"-symbol).
  2. All the existing apple accelerator tables (e.g. apple_types) 
have demangled and unqualified names as a key.
  3. It is not always possible to get the original demanled type name 
by the mangled one (e.g. for templates parametrized with enums the 
demangled one is Impl<(TagType)0> vs original Impl, 
but there are more complex cases).


Thus, I don't see how adding DW_AT_linkage_name to vtable member of 
class (or even to class itself) could help, as it still won't be 
possible to resolve DIE by the mangled type name. However possible 
solutions are:
  1. To generate a separate accelerator table: mangled name for 
vtable member of a class => DIE;
  2. Build index on startup iterating through the apple_types and 
gather the map mangled name => DIE;


Greg, did you mean some of these or something else?


I didn't realize that the mangled name differs in certain cases and 
that it wouldn't suffice for a lookup. Can you give an example of the 
name we try looking up versus what is actually in the symbol table?

Case 1:

enum class TagType : bool {
    Tag1
};

struct I {
    virtual ~I() = default;
};

template 
struct Impl : public I {
    private:
    int v = 123;
};

int main(int argc, const char * argv[]) {
    Impl impl;
    I& i = impl;
    return 0;
}
lldb demangles the name to Impl<(TagType)0> and it's "Impl" in 
DWARF generated by clang.

Case 2:
struct I
{
  virtual ~I(){}
};

template 
struct Impl : public I
{
int v = 123;
};

template <>
struct Impl<1+1+1> : public I  // Note the expression used for this 
specialization
{
int v = 124;
};

template 
struct TT {
  I* i = new T();
};

int main(int argc, const char * argv[]) {
TT> tt;
return 0;  // [*]
}
lldb demangles name to "Impl<3>", whereas clang generates "Impl<1+1+1>" in 
DWARF.

IIUC right now we lookup the address of the first pointer within a 
class if it is virtual and find the symbol name that this corresponds 
to, and in the failing cases you have we don't find anything in the 
DWARF that matches. Is that right?

Exactly, for the cases above and some others.


Thanks,
Anton.

19.12.2017 19:39, Greg Clayton wrote:
I agree with Tamas. The right way to do this it to add the 
DW_AT_linkage_name to the class. Apple accelerator tables have many 
different forms, but one is a mapping of type name to exact DIE 
offset (in the __DWARF_ segment in the __apple_types section). If 
the mangled name was added to the class, then the apple accelerator 
tables would have it. So when a lookup happens with these tables 
around, we do a very quick hash lookup, and we find the exact DIE 
(or DIEs) we need. Entries for classes in the Apple accelerator 
tables have both the mangled and raw class name as entries pointing 
to the same DIE since lookups don't usually happen via mangled 
names. LLDB also knows how to pull names apart and search correctly, 
so if someone tries to lookup a type with "a::b::MyClass", we will 
chop that up into "MyClass" and do a lookup on that. We might get 
many many different "MyClass" results back (a::c::MyClass, 
::MyClass, b::MyClass), but then we cull those down by making sure 
any matches have a matching decl context of "a::b::". For mangled 
names, it is easy and just a direct lookup.


The apple accelerator tables are only enabled for Darwin target, but 
there is nothing to say we couldn't enable these for other targets 
in ELF files. It would be a quick way to gauge the performance 
improvement that these accelerator tables provide for linux. 
Currently linux will completely index the DWARF, but it will load 
the DWARF, index it, and unload the DWARF so we don't hog memory for 
things we don't need loaded yet. We must manually index the DWARF 
because the DWARF accelerator tables are really not accelerator 
tables, they are random indexes of related data (names in no 
particular order, addresses in or particular order). These tables 
are also not complete so no debugger can rely on them. For example 
".debug_pubtypes" is for "public" types only. ".debug_pubnames" is a 
random name table with only public functions (no static functions or 
functions in anonymous namespaces). So the DWARF accelerator tables 
can't be used by debuggers.


There is now a modified version of the Apple accelerator tables in 
the DWARF standard that can provide the same data as the Apple 
versions, but I don't believe anyone has added this support to any 
compilers yet. So for simplicity, we can try things out with the 
Apple accelerator tables and see how