Re: [lldb-dev] LLDB + UE4 on Android

[Pavel Labath via lldb-dev]

So, from the error message it looks like lldb is getting confused by
the debug information inside the executable. I don't have any special
hints for debugging that, I think the usual process of attaching the
debugger, examining the backtrace, and seeing what data is triggering
the assert would work best.

Alternatively, if you can create a list of steps to reproduce this
bug, we may be able to look at it for you. We're interested in making
this use case work, so we'll try to fix it soon.

pl

On 10 March 2016 at 18:00, Mikhail Filimonov  wrote:
> Well, after I've added .gnu_debuglink section to stripped libUE4.so LLDB 
> managed to resolve the symbols.
> But sometimes it asserts with the following message:
> ---
> Microsoft Visual C++ Runtime Library
> ---
> Assertion failed!
> Program: ...ORPORATION\NSIGHT TEGRA\3.3.16035.4243\liblldb.dll
> File: C:\wrk\llvm_git\tools\clang\lib\AST\Recor...tBuilder.cpp
> Line: 82
> Expression: FieldOffsets.count(FD) && "Field does not have an external offset"
> For information on how your program can cause an assertion
> failure, see the Visual C++ documentation on asserts
> (Press Retry to debug the application - JIT must be enabled)
> ---
> Abort   Retry   Ignore
> ---
> Any suggestions how to debug that?
>
> Regards,
> Mikhail
>
> -Original Message-
> From: Mikhail Filimonov
> Sent: Thursday, March 10, 2016 12:33 AM
> To: lldb-dev@lists.llvm.org
> Cc: Tamas Berghammer ; Pavel Labath 
> 
> Subject: LLDB + UE4 on Android
>
> Hello, fellow developers!
>
> I’m trying to debug Unreal Engine 4 sample on Android with LLDB 3.8 build 
> from source - Win32 x86 liblldb.dll and Android ARM lldb-server : it can’t 
> match the stripped module libUE4.so running on the device with a full version 
> which is available on a host.
> Unreal Build Tool don’t add the .note.gnu.build-id or .gnu_debuglink sections 
> to ELF files – and it looks that only if the stripped ELF .so running on a 
> target does have one of these than it could be matched with a full ELF module 
> located on a host - is that true?
>
> My reasoning is based on examination of
> https://github.com/llvm-mirror/lldb/blob/release_38/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationClient.cpp#L4328
> and
> https://github.com/llvm-mirror/lldb/blob/release_38/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp#L790
>
> If so, I need to modify UBT to add the .build-id or .gnu_debuglink to a 
> stripped libUE4.so
>
> ---
> This email message is for the sole use of the intended recipient(s) and may 
> contain
> confidential information.  Any unauthorized review, use, disclosure or 
> distribution
> is prohibited.  If you are not the intended recipient, please contact the 
> sender by
> reply email and destroy all copies of the original message.
> ---
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] DWARFASTParserClang and DW_TAG_typedef for anonymous structs

[Luke Drummond via lldb-dev]

I forgot to CC the list earlier. Apologies.

 Forwarded Message 
Subject: Re: [lldb-dev] DWARFASTParserClang and DW_TAG_typedef for 
anonymous structs

Date: Fri, 11 Mar 2016 11:55:50 +
From: Luke Drummond 
To: Greg Clayton 

Hi Greg

On 10/03/16 22:26, Greg Clayton wrote:

Please file a bug for this and I will relate it to our internal apple bug that 
tracks this issue.


I made a bit of hash of it (submitted early), [but here it
is](https://llvm.org/bugs/show_bug.cgi?id=26790)

Additionally, I created a diff for a regression test
[here](http://reviews.llvm.org/D1)




On Mar 10, 2016, at 2:03 PM, Greg Clayton via lldb-dev 
 wrote:

Thanks for the example, this is indeed a new regression. It used to work (Xcode 7.2), but 
now with top of tree it doesn't. Sean Callanan recently pulled out a bunch of work around 
we used to have in the expression/JIT so that we can avoid certain issues that were 
caused by said work arounds, and those are causing problems now. I looked at the old 
expression parser and it will still making up the name _Z6myfuncP3$_0, but it would first 
try the mangled name, and if it didn't find that, then it would fall back to just looking 
for the demangled basename ("myfunc"). We removed this work around because now 
we are trying to be more correct, and that caused this regression. Sean Callanan will 
take a look at this and get a fix for it sometime soon. What is probably happening is we 
are removing the typedef sugar from the function arguments somewhere that we shouldn't be 
(like maybe in the clang::ASTImporter or our lldb_private::ClangASTImporter). We should 
be trying to lookup the mangle nam

e

"_Z6myfuncP18my_untagged_struct", but somehow when we go to lookup the 
type we lost the my_untagged_struct and are looking for an anonymous 
struct "$_0" instead.


That makes a lot of sense. My debugging efforts were originally focused
on the `lldb_private::ClangASTImporter`, and
`ClangExpressionDeclMap::FindExternalVisibleDecls`, though I didn't
quite know what the correct behaviour should be.

Thanks again for your help.

Best

Luke



Greg Clayton


On Mar 10, 2016, at 10:20 AM, Luke Drummond  wrote:

Hi Greg

First of all thanks for taking the time to help out with this.

On 10/03/16 00:18, Greg Clayton wrote:

So we ran into a problem where we had anonymous structs in modules. They have no name, so 
we had no way to say "module A, please give me a struct named... nothing in the 
namespace 'foo'". Obviously this doesn't work, so we always try to make sure a 
typedef doesn't come from a module first, by asking us to get the typedef from the DWO 
file:

type_sp = ParseTypeFromDWO(die, log);

If this fails, it just means we have the typedef in hand. If I compile your 
example I end up with:

0x000b: TAG_compile_unit [1] *
 AT_producer( "Apple LLVM version 8.0.0 (clang-800.0.5.3)" )
 AT_language( DW_LANG_C99 )
 AT_name( "main.c" )
 AT_stmt_list( 0x )
 AT_comp_dir( "/tmp" )
 AT_low_pc( 0x00010f60 )
 AT_high_pc( 0x00010fb0 )

0x002e: TAG_subprogram [2] *
 AT_low_pc( 0x00010f60 )
 AT_high_pc( 0x00010f85 )
 AT_frame_base( rbp )
 AT_name( "myfunc" )
 AT_decl_file( "/private/tmp/main.c" )
 AT_decl_line( 6 )
 AT_prototyped( 0x01 )
 AT_external( 0x01 )

0x0049: TAG_formal_parameter [3]
 AT_location( fbreg -8 )
 AT_name( "s" )
 AT_decl_file( "/private/tmp/main.c" )
 AT_decl_line( 6 )
 AT_type( {0x008c} ( my_untagged_struct* ) )

0x0057: NULL

0x0058: TAG_subprogram [4] *
 AT_low_pc( 0x00010f90 )
 AT_high_pc( 0x00010fb0 )
 AT_frame_base( rbp )
 AT_name( "main" )
 AT_decl_file( "/private/tmp/main.c" )
 AT_decl_line( 12 )
 AT_type( {0x0085} ( int ) )
 AT_external( 0x01 )

0x0076: TAG_variable [5]
 AT_location( fbreg -16 )
 AT_name( "s" )
 AT_decl_file( "/private/tmp/main.c" )
 AT_decl_line( 14 )
 AT_type( {0x0091} ( my_untagged_struct ) )

0x0084: NULL

0x0085: TAG_base_type [6]
 AT_name( "int" )
 AT_encoding( DW_ATE_signed )
 AT_byte_size( 0x04 )

0x008c: TAG_pointer_type [7]
 AT_type( {0x0091} ( my_untagged_struct ) )

0x0091: TAG_typedef [8]
 AT_type( {0x009c} ( struct  ) )
 AT_name( "my_untagged_struct" )
 AT_decl_file( "/private/tmp/main.c" )
 A

[lldb-dev] SymbolFile::FindGlobalVariables

[Zachary Turner via lldb-dev]

I'm trying to implement this function for PDB.  There are two overloads:

uint32_t
FindGlobalVariables (const ConstString &name, const CompilerDeclContext
*parent_decl_ctx, bool append, uint32_t max_matches, VariableList&
variables)

uint32_t
FindGlobalVariables(const RegularExpression& regex, bool append, uint32_t
max_matches, VariableList& variables)

I know how to implement the second overload, but not the first.  What is a
CompilerDeclContext?  Some comments in the DWARF implementation of the
function seem to imply it's related to namespaces, but there's a lot of
strange code that I don't understand.  What is the relationship between a
namespace and a symbol file?  And why does
`DeclContextMatchesThisSymbolFile` contain no code at all that accesses any
property of the symbol file?  It just checks if
decl_ctx->GetTypeSystem()->GetMinimumLanguage(nullptr)
== decl_ctx->GetTypeSystem(), which appears to have nothing to do with any
symbol file.

What user command or debugger operation results in FindGlobalVariables
getting called with this particular overload, and how does it build the
CompilerDeclContext?

On another note, why is the decl context stored as void* instead of having
an actual wrapper with an abstract interface such as ClangDeclContext /
JavaDeclContext, etc that all inherit from LanguageDeclContext, and pass
the LanguageDeclContext around instead of a void*?
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] SymbolFile::FindGlobalVariables

[Zachary Turner via lldb-dev]

Also why does the lldb_private::Variable() class take a DWARFExpression to
its constructor?  Seems like this is wrong in the face of non-DWARF debug
information.

On Fri, Mar 11, 2016 at 11:02 AM Zachary Turner  wrote:

> I'm trying to implement this function for PDB.  There are two overloads:
>
> uint32_t
> FindGlobalVariables (const ConstString &name, const CompilerDeclContext
> *parent_decl_ctx, bool append, uint32_t max_matches, VariableList&
> variables)
>
> uint32_t
> FindGlobalVariables(const RegularExpression& regex, bool append, uint32_t
> max_matches, VariableList& variables)
>
> I know how to implement the second overload, but not the first.  What is a
> CompilerDeclContext?  Some comments in the DWARF implementation of the
> function seem to imply it's related to namespaces, but there's a lot of
> strange code that I don't understand.  What is the relationship between a
> namespace and a symbol file?  And why does
> `DeclContextMatchesThisSymbolFile` contain no code at all that accesses any
> property of the symbol file?  It just checks if
> decl_ctx->GetTypeSystem()->GetMinimumLanguage(nullptr)
> == decl_ctx->GetTypeSystem(), which appears to have nothing to do with any
> symbol file.
>
> What user command or debugger operation results in FindGlobalVariables
> getting called with this particular overload, and how does it build the
> CompilerDeclContext?
>
> On another note, why is the decl context stored as void* instead of having
> an actual wrapper with an abstract interface such as ClangDeclContext /
> JavaDeclContext, etc that all inherit from LanguageDeclContext, and pass
> the LanguageDeclContext around instead of a void*?
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] SymbolFile::FindGlobalVariables

[Greg Clayton via lldb-dev]

> On Mar 11, 2016, at 11:02 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> I'm trying to implement this function for PDB.  There are two overloads:
> 
> uint32_t
> FindGlobalVariables (const ConstString &name, const CompilerDeclContext 
> *parent_decl_ctx, bool append, uint32_t max_matches, VariableList& variables)
> 
> uint32_t
> FindGlobalVariables(const RegularExpression& regex, bool append, uint32_t 
> max_matches, VariableList& variables)
> 
> I know how to implement the second overload, but not the first.  What is a 
> CompilerDeclContext?  


It is a declaration context. A variable like "foo::g_int" that is declared like:

namespace foo
{
   int g_int;
}

Has a decl context of "namespace foo". Also a variable like:


namespace foo
{
class bar
{
 struct baz
 {
  void foo();
 };
};
}

The function foo would have a decl context "struct baz". "struct baz" has a 
parent decl context "class bar". "class bar" has a parent decl context 
"namespace foo". "namespace foo" has a parent decl context of "compile unit"


> Some comments in the DWARF implementation of the function seem to imply it's 
> related to namespaces, but there's a lot of strange code that I don't 
> understand.  What is the relationship between a namespace and a symbol file?  
> And why does `DeclContextMatchesThisSymbolFile` contain no code at all that 
> accesses any property of the symbol file?  It just checks if 
> decl_ctx->GetTypeSystem()->GetMinimumLanguage(nullptr) == 
> decl_ctx->GetTypeSystem(), which appears to have nothing to do with any 
> symbol file.

When it comes down to creating types I am going to guess that you will be using 
ClangASTContext to create any types that you hand out. This already has all of 
the needed calls for you to create all of the stuff that you will need. You 
will need to take a look at DWARFASTParserClang and see how it creates types 
using the ClangASTContext.
> 
> What user command or debugger operation results in FindGlobalVariables 
> getting called with this particular overload, and how does it build the 
> CompilerDeclContext?

The SymbolFile subclasses will create decl contexts as needed. In DWARF it uses:

void
SymbolFileDWARF::ParseDeclsForContext (CompilerDeclContext decl_ctx)
{
TypeSystem *type_system = decl_ctx.GetTypeSystem();
DWARFASTParser *ast_parser = type_system->GetDWARFParser();
std::vector decl_ctx_die_list = 
ast_parser->GetDIEForDeclContext(decl_ctx);

for (DWARFDIE decl_ctx_die : decl_ctx_die_list)
for (DWARFDIE decl = decl_ctx_die.GetFirstChild(); decl; decl = 
decl.GetSibling())
ast_parser->GetDeclForUIDFromDWARF(decl);
}

> 
> On another note, why is the decl context stored as void* instead of having an 
> actual wrapper with an abstract interface such as ClangDeclContext / 
> JavaDeclContext, etc that all inherit from LanguageDeclContext, and pass the 
> LanguageDeclContext around instead of a void*?

So three classes: CompilerType, CompilerDecl and CompilerDeclContext all 
contain a "TypeSystem *" which points to a subclass of "TypeSystem". Then each 
different type system will store their native pointer to the thing that 
represents a type, decl and decl context. For ClangASTContext TypeSystem 
produced objects, CompilerType stores a QualType as an opaque pointer gotten 
from a call to "clang::QualType::getAsOpaquePtr()". CompilerDecl stores just 
the "clang::Decl *", and for CompilerDeclContext we store a "clang::DeclContext 
*". Each type system is different. 

SwiftASTContext stores "swift::Type*" in CompilerType, and it doesn't represent 
CompilerDecl or CompilerDeclContext because its expression parser doesn't need 
access to these things.

RenderScript and Go each have their own type systems and can back CompilerType, 
CompilerDecl and CompilerDeclContext as they need to backing them with whatever 
they need. 

Right now CompilerType is the important one since all variable viewing explores 
CompilerType to display the children of a struct/union/class. CompilerDecl and 
CompilerDeclContext will help with expressions and the only thing that needs 
this right now is the clang expression parser for C/C++/ObjC/ObjC++.

When you are asked to parse a type for a variable in your SymbolFilePDB, you 
will be required to make a CompilerType. I would suggest using a 
ClangASTContext as a type system to create your types. You can see how DWARF 
does this in DWARFASTParserClang. You will need to do something very similar. 
If you need to create a class for something like:

namespace A
{
class B
{
};
}

Part of correctly doing so involves you creating a "namespace A" in the 
ClangASTContext (which is a clang::DeclContext). You will specify that the 
translation unit is the decl context for "namespace A" when you create the 
namespace. When you create the "class B", you will need to specify that the 
context that is is created in is the "namespace A" from the ClangASTContext. 
These 

Re: [lldb-dev] SymbolFile::FindGlobalVariables

[Jim Ingham via lldb-dev]

lldb uses DWARF expressions internally as a convenient language to represent 
locations of values.  We had to pick some representation, and the DWARF 
expression was powerful enough for our purposes, meant we didn't have to 
reinvent something that already existed, and had the added benefit that if you 
did your DWARF then you don't have to transcode. 

Jim

> On Mar 11, 2016, at 11:32 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> Also why does the lldb_private::Variable() class take a DWARFExpression to 
> its constructor?  Seems like this is wrong in the face of non-DWARF debug 
> information.
> 
> On Fri, Mar 11, 2016 at 11:02 AM Zachary Turner  wrote:
> I'm trying to implement this function for PDB.  There are two overloads:
> 
> uint32_t
> FindGlobalVariables (const ConstString &name, const CompilerDeclContext 
> *parent_decl_ctx, bool append, uint32_t max_matches, VariableList& variables)
> 
> uint32_t
> FindGlobalVariables(const RegularExpression& regex, bool append, uint32_t 
> max_matches, VariableList& variables)
> 
> I know how to implement the second overload, but not the first.  What is a 
> CompilerDeclContext?  Some comments in the DWARF implementation of the 
> function seem to imply it's related to namespaces, but there's a lot of 
> strange code that I don't understand.  What is the relationship between a 
> namespace and a symbol file?  And why does `DeclContextMatchesThisSymbolFile` 
> contain no code at all that accesses any property of the symbol file?  It 
> just checks if decl_ctx->GetTypeSystem()->GetMinimumLanguage(nullptr) == 
> decl_ctx->GetTypeSystem(), which appears to have nothing to do with any 
> symbol file.
> 
> What user command or debugger operation results in FindGlobalVariables 
> getting called with this particular overload, and how does it build the 
> CompilerDeclContext?
> 
> On another note, why is the decl context stored as void* instead of having an 
> actual wrapper with an abstract interface such as ClangDeclContext / 
> JavaDeclContext, etc that all inherit from LanguageDeclContext, and pass the 
> LanguageDeclContext around instead of a void*?
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] SymbolFile::FindGlobalVariables

[Zachary Turner via lldb-dev]

Can we abstract this somehow?  Converting all my debug info to DWARF seems
like a non-starter, as it doesn't look like you can just do it partially,
you have to go all the way (just based on glancing at the DWARFExpression
header file)

On Fri, Mar 11, 2016 at 11:38 AM Jim Ingham  wrote:

> lldb uses DWARF expressions internally as a convenient language to
> represent locations of values.  We had to pick some representation, and the
> DWARF expression was powerful enough for our purposes, meant we didn't have
> to reinvent something that already existed, and had the added benefit that
> if you did your DWARF then you don't have to transcode.
>
> Jim
>
> > On Mar 11, 2016, at 11:32 AM, Zachary Turner via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> >
> > Also why does the lldb_private::Variable() class take a DWARFExpression
> to its constructor?  Seems like this is wrong in the face of non-DWARF
> debug information.
> >
> > On Fri, Mar 11, 2016 at 11:02 AM Zachary Turner 
> wrote:
> > I'm trying to implement this function for PDB.  There are two overloads:
> >
> > uint32_t
> > FindGlobalVariables (const ConstString &name, const CompilerDeclContext
> *parent_decl_ctx, bool append, uint32_t max_matches, VariableList&
> variables)
> >
> > uint32_t
> > FindGlobalVariables(const RegularExpression& regex, bool append,
> uint32_t max_matches, VariableList& variables)
> >
> > I know how to implement the second overload, but not the first.  What is
> a CompilerDeclContext?  Some comments in the DWARF implementation of the
> function seem to imply it's related to namespaces, but there's a lot of
> strange code that I don't understand.  What is the relationship between a
> namespace and a symbol file?  And why does
> `DeclContextMatchesThisSymbolFile` contain no code at all that accesses any
> property of the symbol file?  It just checks if
> decl_ctx->GetTypeSystem()->GetMinimumLanguage(nullptr) ==
> decl_ctx->GetTypeSystem(), which appears to have nothing to do with any
> symbol file.
> >
> > What user command or debugger operation results in FindGlobalVariables
> getting called with this particular overload, and how does it build the
> CompilerDeclContext?
> >
> > On another note, why is the decl context stored as void* instead of
> having an actual wrapper with an abstract interface such as
> ClangDeclContext / JavaDeclContext, etc that all inherit from
> LanguageDeclContext, and pass the LanguageDeclContext around instead of a
> void*?
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] SymbolFile::FindGlobalVariables

[Greg Clayton via lldb-dev]

> 
> Also why does the lldb_private::Variable() class take a DWARFExpression to 
> its constructor?  Seems like this is wrong in the face of non-DWARF debug 
> information.

They are powerful enough to handle any variable location. More powerful than 
any other format I have seen. You have two choices:

- make a new lldb_private::Location class and have DWARFExpression implement 
the pure virtuals you need
- convert PDB locations into DWARF

Personally the second sounds easier as the DWARF expressions are well 
documented and they are easy to construct. If you have the spec for the PDB 
locations and can point me at this, I can take a look to see how well things 
would map.

Variables that are in registers use a DWARF location expression like:

DW_OP_reg12

that means the value is register number 12.

DW_OP_addr(0x1) means the value is a global variable whose value lives at 
"file address 0x1 inside of the module from which is originates". We 
translate the file address into a load address if we are running and if that 
resolves to a load address, we can read the variable value.

DW_OP_fbreg32(32) means the value is 32 bytes off of register 32.

So the locations expression are often this simple: in a register, in .data at a 
file address, or on the stack. So depending on how complex locations are in 
PDB, it might be easier to just create a simple DWARF expression and be done 
with it.

Greg Clayton



___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] SymbolFile::FindGlobalVariables

[Greg Clayton via lldb-dev]

See my other email. You can abstract this, but it doesn't seem worth it unless 
PDB has some really powerful way to express variable locations?

> On Mar 11, 2016, at 11:39 AM, Zachary Turner via lldb-dev 
>  wrote:
> 
> Can we abstract this somehow?  Converting all my debug info to DWARF seems 
> like a non-starter, as it doesn't look like you can just do it partially, you 
> have to go all the way (just based on glancing at the DWARFExpression header 
> file)
> 
> On Fri, Mar 11, 2016 at 11:38 AM Jim Ingham  wrote:
> lldb uses DWARF expressions internally as a convenient language to represent 
> locations of values.  We had to pick some representation, and the DWARF 
> expression was powerful enough for our purposes, meant we didn't have to 
> reinvent something that already existed, and had the added benefit that if 
> you did your DWARF then you don't have to transcode.
> 
> Jim
> 
> > On Mar 11, 2016, at 11:32 AM, Zachary Turner via lldb-dev 
> >  wrote:
> >
> > Also why does the lldb_private::Variable() class take a DWARFExpression to 
> > its constructor?  Seems like this is wrong in the face of non-DWARF debug 
> > information.
> >
> > On Fri, Mar 11, 2016 at 11:02 AM Zachary Turner  wrote:
> > I'm trying to implement this function for PDB.  There are two overloads:
> >
> > uint32_t
> > FindGlobalVariables (const ConstString &name, const CompilerDeclContext 
> > *parent_decl_ctx, bool append, uint32_t max_matches, VariableList& 
> > variables)
> >
> > uint32_t
> > FindGlobalVariables(const RegularExpression& regex, bool append, uint32_t 
> > max_matches, VariableList& variables)
> >
> > I know how to implement the second overload, but not the first.  What is a 
> > CompilerDeclContext?  Some comments in the DWARF implementation of the 
> > function seem to imply it's related to namespaces, but there's a lot of 
> > strange code that I don't understand.  What is the relationship between a 
> > namespace and a symbol file?  And why does 
> > `DeclContextMatchesThisSymbolFile` contain no code at all that accesses any 
> > property of the symbol file?  It just checks if 
> > decl_ctx->GetTypeSystem()->GetMinimumLanguage(nullptr) == 
> > decl_ctx->GetTypeSystem(), which appears to have nothing to do with any 
> > symbol file.
> >
> > What user command or debugger operation results in FindGlobalVariables 
> > getting called with this particular overload, and how does it build the 
> > CompilerDeclContext?
> >
> > On another note, why is the decl context stored as void* instead of having 
> > an actual wrapper with an abstract interface such as ClangDeclContext / 
> > JavaDeclContext, etc that all inherit from LanguageDeclContext, and pass 
> > the LanguageDeclContext around instead of a void*?
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] SymbolFile::FindGlobalVariables

[Zachary Turner via lldb-dev]

The only "spec" is the API that allows you to access the info.  There's no
spec of the bit format.  This
 is probably all
you are actually looking for though:

The problem isn't necessarily that one is more pwoerful than the other,
it's just that PDBs can get huge (on the order of gigabytes), and
converting between formats is an unnecessary step that a) will be slow to
do the conversion, b) might not map 1 to 1 between the formats, and c) it'
already trivial (on the order of a few lines of code) to just query the PDB
for everything you need.

So we're talking about potentially thousands of lines of code to do
something that would take about 10 (as well as being more efficient) with a
proper abstraction.

On Fri, Mar 11, 2016 at 11:43 AM Greg Clayton  wrote:

> See my other email. You can abstract this, but it doesn't seem worth it
> unless PDB has some really powerful way to express variable locations?
>
> > On Mar 11, 2016, at 11:39 AM, Zachary Turner via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> >
> > Can we abstract this somehow?  Converting all my debug info to DWARF
> seems like a non-starter, as it doesn't look like you can just do it
> partially, you have to go all the way (just based on glancing at the
> DWARFExpression header file)
> >
> > On Fri, Mar 11, 2016 at 11:38 AM Jim Ingham  wrote:
> > lldb uses DWARF expressions internally as a convenient language to
> represent locations of values.  We had to pick some representation, and the
> DWARF expression was powerful enough for our purposes, meant we didn't have
> to reinvent something that already existed, and had the added benefit that
> if you did your DWARF then you don't have to transcode.
> >
> > Jim
> >
> > > On Mar 11, 2016, at 11:32 AM, Zachary Turner via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> > >
> > > Also why does the lldb_private::Variable() class take a
> DWARFExpression to its constructor?  Seems like this is wrong in the face
> of non-DWARF debug information.
> > >
> > > On Fri, Mar 11, 2016 at 11:02 AM Zachary Turner 
> wrote:
> > > I'm trying to implement this function for PDB.  There are two
> overloads:
> > >
> > > uint32_t
> > > FindGlobalVariables (const ConstString &name, const
> CompilerDeclContext *parent_decl_ctx, bool append, uint32_t max_matches,
> VariableList& variables)
> > >
> > > uint32_t
> > > FindGlobalVariables(const RegularExpression& regex, bool append,
> uint32_t max_matches, VariableList& variables)
> > >
> > > I know how to implement the second overload, but not the first.  What
> is a CompilerDeclContext?  Some comments in the DWARF implementation of the
> function seem to imply it's related to namespaces, but there's a lot of
> strange code that I don't understand.  What is the relationship between a
> namespace and a symbol file?  And why does
> `DeclContextMatchesThisSymbolFile` contain no code at all that accesses any
> property of the symbol file?  It just checks if
> decl_ctx->GetTypeSystem()->GetMinimumLanguage(nullptr) ==
> decl_ctx->GetTypeSystem(), which appears to have nothing to do with any
> symbol file.
> > >
> > > What user command or debugger operation results in FindGlobalVariables
> getting called with this particular overload, and how does it build the
> CompilerDeclContext?
> > >
> > > On another note, why is the decl context stored as void* instead of
> having an actual wrapper with an abstract interface such as
> ClangDeclContext / JavaDeclContext, etc that all inherit from
> LanguageDeclContext, and pass the LanguageDeclContext around instead of a
> void*?
> > > ___
> > > lldb-dev mailing list
> > > lldb-dev@lists.llvm.org
> > > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> >
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] SymbolFile::FindGlobalVariables

[Greg Clayton via lldb-dev]

Feel free to abstract if you need to. The page you sent me to has _very_ simple 
locations that would convert to DWARF expressions very easily. Probably less 
that a hundred lines of code.

If you need to abstract, making a lldb_private::Location class that 
DWARFExpression would implement the needed pure virtuals. Then each things that 
contains DWARFExpression would now contain a lldb_private::LocationSP which 
would be a shared pointer to a lldb_private::Location. DWARFExpression has 
grown over the years to contain a bunch of evaluate variants.

Just know that LLDB lazily parses things. We don't say "convert the entire PDB 
into the internal LLDB format now!". We say "get the line table for this one 
compile unit". Find the function for address "0x123000" and parse it. Later we 
will ask to get the function type and its args. Later, if we ever need to, we 
will lazily parse the blocks in the function. Nothing is parsed in full.

Let me know what you want to do. Abstraction is great, but comes at a cost of 
breaking things when our tests don't cover everything, so that is a worry on my 
end with any large changes...


> On Mar 11, 2016, at 11:47 AM, Zachary Turner  wrote:
> 
> The only "spec" is the API that allows you to access the info.  There's no 
> spec of the bit format.  This is probably all you are actually looking for 
> though:
> 
> The problem isn't necessarily that one is more pwoerful than the other, it's 
> just that PDBs can get huge (on the order of gigabytes), and converting 
> between formats is an unnecessary step that a) will be slow to do the 
> conversion, b) might not map 1 to 1 between the formats, and c) it' already 
> trivial (on the order of a few lines of code) to just query the PDB for 
> everything you need.
> 
> So we're talking about potentially thousands of lines of code to do something 
> that would take about 10 (as well as being more efficient) with a proper 
> abstraction.
> 
> On Fri, Mar 11, 2016 at 11:43 AM Greg Clayton  wrote:
> See my other email. You can abstract this, but it doesn't seem worth it 
> unless PDB has some really powerful way to express variable locations?
> 
> > On Mar 11, 2016, at 11:39 AM, Zachary Turner via lldb-dev 
> >  wrote:
> >
> > Can we abstract this somehow?  Converting all my debug info to DWARF seems 
> > like a non-starter, as it doesn't look like you can just do it partially, 
> > you have to go all the way (just based on glancing at the DWARFExpression 
> > header file)
> >
> > On Fri, Mar 11, 2016 at 11:38 AM Jim Ingham  wrote:
> > lldb uses DWARF expressions internally as a convenient language to 
> > represent locations of values.  We had to pick some representation, and the 
> > DWARF expression was powerful enough for our purposes, meant we didn't have 
> > to reinvent something that already existed, and had the added benefit that 
> > if you did your DWARF then you don't have to transcode.
> >
> > Jim
> >
> > > On Mar 11, 2016, at 11:32 AM, Zachary Turner via lldb-dev 
> > >  wrote:
> > >
> > > Also why does the lldb_private::Variable() class take a DWARFExpression 
> > > to its constructor?  Seems like this is wrong in the face of non-DWARF 
> > > debug information.
> > >
> > > On Fri, Mar 11, 2016 at 11:02 AM Zachary Turner  
> > > wrote:
> > > I'm trying to implement this function for PDB.  There are two overloads:
> > >
> > > uint32_t
> > > FindGlobalVariables (const ConstString &name, const CompilerDeclContext 
> > > *parent_decl_ctx, bool append, uint32_t max_matches, VariableList& 
> > > variables)
> > >
> > > uint32_t
> > > FindGlobalVariables(const RegularExpression& regex, bool append, uint32_t 
> > > max_matches, VariableList& variables)
> > >
> > > I know how to implement the second overload, but not the first.  What is 
> > > a CompilerDeclContext?  Some comments in the DWARF implementation of the 
> > > function seem to imply it's related to namespaces, but there's a lot of 
> > > strange code that I don't understand.  What is the relationship between a 
> > > namespace and a symbol file?  And why does 
> > > `DeclContextMatchesThisSymbolFile` contain no code at all that accesses 
> > > any property of the symbol file?  It just checks if 
> > > decl_ctx->GetTypeSystem()->GetMinimumLanguage(nullptr) == 
> > > decl_ctx->GetTypeSystem(), which appears to have nothing to do with any 
> > > symbol file.
> > >
> > > What user command or debugger operation results in FindGlobalVariables 
> > > getting called with this particular overload, and how does it build the 
> > > CompilerDeclContext?
> > >
> > > On another note, why is the decl context stored as void* instead of 
> > > having an actual wrapper with an abstract interface such as 
> > > ClangDeclContext / JavaDeclContext, etc that all inherit from 
> > > LanguageDeclContext, and pass the LanguageDeclContext around instead of a 
> > > void*?
> > > ___
> > > lldb-dev mailing list
> > > lldb-dev@lists.llvm.org
> > > http:

Re: [lldb-dev] SymbolFile::FindGlobalVariables

[Zachary Turner via lldb-dev]

How large of a change do you think it would be to abstract out the location
information for the variable?  As far as I can tell, our uses of this
DWARFExpression on Variables are very limited:

1. In ValueObjectVariable::UpdateValue and
ClangExpressionDeclMap::GetVariableValue, if the location is a constant
value, it refers to a a host address, we just read the value out as a
number.
2. In EntityVariable::Materialize(), we check whether it is valid.
3. In SymbolFileDWARF, we "evaluate" the expression.
4. In a few places, we check whether an input address matches the location
specified.
5. We dump the location to stdout in a few places.

Everything else could just as easily be private methods, because that's all
that public users of DWARFExpression actually use.

This seems like an easy abstraction to create.  #3 is irrelevant because
that code is in SymbolFileDWARF, it could downcast from Location to
DWARFLocation.  #1, 2, 4, and 5 could easily be implemented directly
against a PDB.

While I haven't tried to actually *do* either approach yet, I like the idea
of creating the abstraction because it provides the native / most optimized
debugging experience no matter what you're using.  For example, I can
easily imagine a scenario where I have to keep the PDB open in memory to
query some types of information, but I have to do a conversion of location
information for Variables, and the memory usage becomes unacceptable
because everything is memory twice (even though it's lazily evaluated, the
memory usage would double over time).



On Fri, Mar 11, 2016 at 11:56 AM Greg Clayton  wrote:

> Feel free to abstract if you need to. The page you sent me to has _very_
> simple locations that would convert to DWARF expressions very easily.
> Probably less that a hundred lines of code.
>
> If you need to abstract, making a lldb_private::Location class that
> DWARFExpression would implement the needed pure virtuals. Then each things
> that contains DWARFExpression would now contain a lldb_private::LocationSP
> which would be a shared pointer to a lldb_private::Location.
> DWARFExpression has grown over the years to contain a bunch of evaluate
> variants.
>
> Just know that LLDB lazily parses things. We don't say "convert the entire
> PDB into the internal LLDB format now!". We say "get the line table for
> this one compile unit". Find the function for address "0x123000" and parse
> it. Later we will ask to get the function type and its args. Later, if we
> ever need to, we will lazily parse the blocks in the function. Nothing is
> parsed in full.
>
> Let me know what you want to do. Abstraction is great, but comes at a cost
> of breaking things when our tests don't cover everything, so that is a
> worry on my end with any large changes...
>
>
> > On Mar 11, 2016, at 11:47 AM, Zachary Turner  wrote:
> >
> > The only "spec" is the API that allows you to access the info.  There's
> no spec of the bit format.  This is probably all you are actually looking
> for though:
> >
> > The problem isn't necessarily that one is more pwoerful than the other,
> it's just that PDBs can get huge (on the order of gigabytes), and
> converting between formats is an unnecessary step that a) will be slow to
> do the conversion, b) might not map 1 to 1 between the formats, and c) it'
> already trivial (on the order of a few lines of code) to just query the PDB
> for everything you need.
> >
> > So we're talking about potentially thousands of lines of code to do
> something that would take about 10 (as well as being more efficient) with a
> proper abstraction.
> >
> > On Fri, Mar 11, 2016 at 11:43 AM Greg Clayton 
> wrote:
> > See my other email. You can abstract this, but it doesn't seem worth it
> unless PDB has some really powerful way to express variable locations?
> >
> > > On Mar 11, 2016, at 11:39 AM, Zachary Turner via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> > >
> > > Can we abstract this somehow?  Converting all my debug info to DWARF
> seems like a non-starter, as it doesn't look like you can just do it
> partially, you have to go all the way (just based on glancing at the
> DWARFExpression header file)
> > >
> > > On Fri, Mar 11, 2016 at 11:38 AM Jim Ingham  wrote:
> > > lldb uses DWARF expressions internally as a convenient language to
> represent locations of values.  We had to pick some representation, and the
> DWARF expression was powerful enough for our purposes, meant we didn't have
> to reinvent something that already existed, and had the added benefit that
> if you did your DWARF then you don't have to transcode.
> > >
> > > Jim
> > >
> > > > On Mar 11, 2016, at 11:32 AM, Zachary Turner via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> > > >
> > > > Also why does the lldb_private::Variable() class take a
> DWARFExpression to its constructor?  Seems like this is wrong in the face
> of non-DWARF debug information.
> > > >
> > > > On Fri, Mar 11, 2016 at 11:02 AM Zachary Turner 
> wrote:
> > > > I'm trying to implement th

Re: [lldb-dev] SymbolFile::FindGlobalVariables

[Greg Clayton via lldb-dev]

> On Mar 11, 2016, at 1:02 PM, Zachary Turner  wrote:
> 
> How large of a change do you think it would be to abstract out the location 
> information for the variable?  As far as I can tell, our uses of this 
> DWARFExpression on Variables are very limited:
> 
> 1. In ValueObjectVariable::UpdateValue and 
> ClangExpressionDeclMap::GetVariableValue, if the location is a constant 
> value, it refers to a a host address, we just read the value out as a number.
> 2. In EntityVariable::Materialize(), we check whether it is valid.
> 3. In SymbolFileDWARF, we "evaluate" the expression.

Leave this one alone, don't abstract it since it is DWARF native.

> 4. In a few places, we check whether an input address matches the location 
> specified.
> 5. We dump the location to stdout in a few places.
> 
> Everything else could just as easily be private methods, because that's all 
> that public users of DWARFExpression actually use.

Sounds like it won't be too bad.
> 
> This seems like an easy abstraction to create.  #3 is irrelevant because that 
> code is in SymbolFileDWARF, it could downcast from Location to DWARFLocation. 
>  #1, 2, 4, and 5 could easily be implemented directly against a PDB.
> 
> While I haven't tried to actually *do* either approach yet, I like the idea 
> of creating the abstraction because it provides the native / most optimized 
> debugging experience no matter what you're using.  For example, I can easily 
> imagine a scenario where I have to keep the PDB open in memory to query some 
> types of information, but I have to do a conversion of location information 
> for Variables, and the memory usage becomes unacceptable because everything 
> is memory twice (even though it's lazily evaluated, the memory usage would 
> double over time).

You will abstract the location only and that is fine. For everything else we do 
have lldb classes that will need to be created (compile units, functions, 
blocks, variables). Types are done via the TypeSystem subclasses so you will 
need convert all types there. So feel free to abstract the DWARFExpression for 
variable locations only.

I have no problem with the abstraction if you think it is needed. I personally 
think it will be much more work, but I won't be doing it so I don't mind.

Greg

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev