[lldb-dev] [Bug 36107] New: TestGModules.test_with_gmodules fails on linux

2018-01-26 Thread via lldb-dev
https://bugs.llvm.org/show_bug.cgi?id=36107

Bug ID: 36107
   Summary: TestGModules.test_with_gmodules fails on linux
   Product: lldb
   Version: unspecified
  Hardware: PC
OS: Linux
Status: NEW
  Severity: enhancement
  Priority: P
 Component: All Bugs
  Assignee: lldb-dev@lists.llvm.org
  Reporter: lab...@google.com
CC: llvm-b...@lists.llvm.org

It started failing since r323450, but that's only because it did not actually
use modules before that.

This error message may be an indication of the problem:

error: B-1GM9AHXMM9U5Y.pcm DWARF DIE at 0x0033 (class Module) has a member
variable 0x003c (MBptr) whose type is a forward declaration, not a complete
definition.
Try compiling the source file with -fstandalone-debug


The -fstandalone-debug is a red herring because we already pass that flag.

-- 
You are receiving this mail because:
You are the assignee for the bug.___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


[lldb-dev] 2018 European LLVM Developers’ Meeting -- 2 weeks before the Paper Submission Deadline

2018-01-26 Thread p23 power via lldb-dev
*The Paper Submission Deadline is 9th February, that's only 2 weeks! *
Submit your proposal here:  https://hotcrp.llvm.org/eurollvm2018/

Registration is open too: https://www.eventbrite.co
m/e/2018-european-llvm-developers-meeting-bristol-tickets-42283244322

We look forward to reading your submissions.
-- Phillip Power



*We are looking for proposals on the following:*
* Technical Talks on LLVM Infrastructure (~30 minutes)
* Technicals Talks on related sub-projects (Clang, etc)
* Talks of uses of LLVM in academia or industry
* Talks on new projects using Clang or LLVM
* Lightning Talks (5 minutes, no questions, no discussions)
* In depth Tutorials (60 minutes)
* Posters (1 hour poster session)
* Birds of a Feather (~30 minutes)
* Panels related to LLVM or sub-projects

*Student Research Competition (SRC): *
The Student Research Competition offers students doing LLVM related
research a non-academic platform to announce and advertise their work as
well as to discuss it with other researchers, developers and users of LLVM.
Students are asked to submit a proposal for a 20 minute technical talk.
There will be a prize for the best SRC talk.

*What to submit*
For each proposal, please submit a title and short abstract (to be used on
the website) and attach/upload either an extended abstract (1 page maximum)
or slides. We ask that you do not submit a full length paper as your only
attachment as reviewer time is limited.  For the submission, you will need
to note who the speaker is. If there is more than one speaker, please
mention it in the abstract.

*FAQ*

*When will I be notified of acceptance?* - Our goal is to notify all
proposal submitters by February 22, 2018.

*Should I register if I have submitted a proposal?*  - We have 1
complimentary reserved registration for each accepted technical talk, BoF,
or student research competition talk. Accepted tutorials have been reserved
2 complimentary registrations. Panels have up to 3 reserved registrations.
There are no reserved registration spots for posters or lightning talks.
Please register any additional speakers or if you do not have a reserved
registration slot.

*What if I registered and my talk got accepted?* - We can refund your
registration fee and instructions will be sent following the notification.
If you plan to attend even if your proposal is not accepted and are worried
about the event selling out, we suggest registering before notification of
acceptance.

*What if I registered and my talk did NOT get accepted?* - We can refund
your registration fee if you no longer wish to attend if you contact the
organizer by March 2, 2018.

*What will be recorded?* - All technical talks, tutorials, SRC talks,
panels, and lightning talks will be recorded. By submitting your proposal,
you are giving us permission to record if you present at the meeting. For
SRC talks, you have the option to delay publication of the slides and video
for your talk for up to 12 months.

*I have a question, who do I contact?* - Please check the llvm-devmeeting
mailing list to see if your question has already been answered. It if has
not, and your question is generic enough that it can benefit to others,
please email llvm-devmeet...@llvm.org. Alternatively, you can ask the
organizing committee directly at eurollvm-organiz...@llvm.org.

*Are there travel grants for students?* - The LLVM Foundation sponsors
student travel to attend the European LLVM Developers' Meeting. Travel
grants cover some or all of travel related expenses. This program is open
to full time undergraduate and graduate students.  Please submit your
application by February 26th 2018. Full details on eligibility an
application. Notification of successful travel grant awards will be sent
via email by March 2 2018.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


[lldb-dev] [Bug 36109] New: libc++ queue and string tests fail with -gmodules on android

2018-01-26 Thread via lldb-dev
https://bugs.llvm.org/show_bug.cgi?id=36109

Bug ID: 36109
   Summary: libc++ queue and string tests fail with -gmodules on
android
   Product: lldb
   Version: unspecified
  Hardware: PC
OS: Linux
Status: NEW
  Severity: enhancement
  Priority: P
 Component: All Bugs
  Assignee: lldb-dev@lists.llvm.org
  Reporter: lab...@google.com
CC: llvm-b...@lists.llvm.org

The std::queue test fails with an amusing error message:
file "Android clang version 5.0.300080  (based on LLVM 5.0.300080)\Android
clang version 5.0.300080  (based on LLVM 5.0.300080)" does not exist
while the std::string just fails to print a std::wstring value.

Could be a parsing problem, or an issue with a (relatively) old clang used to
compile the tests.

-- 
You are receiving this mail because:
You are the assignee for the bug.___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Huge mangled names are causing long delays when loading symbol table symbols

2018-01-26 Thread Erik Pilkington via lldb-dev



On 2018-01-25 1:58 PM, Greg Clayton wrote:

On Jan 25, 2018, at 10:25 AM, Erik Pilkington  wrote:

Hi,
I'm not at all familiar with LLDB, but I've been doing some work on the 
demangler in libcxxabi. It's still a work in progress and I haven't yet copied 
the changes over to ItaniumDemangle, which AFAIK is what lldb uses. The 
demangler in libcxxabi now demangles the symbol you attached in 3.31 seconds, 
instead of 223.54 on my machine. I posted a RFC on my work here 
(http://lists.llvm.org/pipermail/llvm-dev/2017-June/114448.html), but basically 
the new demangler just produces an AST then traverses it to print the demangled 
name.

Great to hear the huge speedup in demangling! LLDB actually has two demanglers: 
a fast one that can demangle 99% of names, and we fall back to ItaniumDemangle 
which can do all names but is really slow. It would be fun to compare your new 
demangler with the fast one and see if we can get rid of the fast demangler now.


I think a good way of making this even faster is to have LLDB consume the AST 
the demangler produces directly. The AST is a better representation of the 
information that LLDB wants, and finishing the demangle and then fishing out 
that information from the output string is unfortunate. From the AST, it would 
be really straightforward to just individually print all the components of the 
name that LLDB wants.

This would help us to grab the important bits out of the mangled name as well. We chop up a demangled name to find the 
base name (string for std::string), containing context (std:: for std::string) and we check if we can tell if the 
function is a method (look for trailing "const" modifier on the function) versus a top level function (since 
the mangling doesn't fully specify what is a namespace and what is a class (like in "foo::bar::baz()" we 
don't know if "foo" or "bar" are classes or namespaces. So the AST would be great as long as it is 
fast.


Most of the time it takes to demangle these "symbols from hell" is during the 
printing, after the AST has been parsed, because the demangler has to flatten out all the 
potentially nested back references. Just parsing to an AST should be about proportional 
to the strlen of the mangled name. Since (AFAIK) LLDB doesn't use some sections of the 
demangled name often (such as parameters), from the AST LLDB could lazily decide not to 
even bother fully demangling some sections of the name, then if it ever needs them it 
could parse a new AST and get them from there. I think this would largely fix the issue, 
as most of the time these crazy expansions don't occur in the name itself, but in the 
parameters or return type. Even when they do appear in the name, it would be possible to 
do some simple name classification (ie, does this symbol refer to a function) or pull out 
the basename quickly without expanding anything at all.

Any thoughts? I'm really not at all familiar with LLDB, so I could have this 
all wrong!

AST sounds great. We can put this into the class we use to chop us C++ names as 
that is really our goal.

So it would be great to do a speed comparison between our fast demangler in 
LLDB (in FastDemangle.cpp/.h) and your updated libcxxabi version. If yours is 
faster, remove FastDemangle and then update the llvm::ItaniumDemangle() to use 
your new code.

ASTs would be great for the C++ name parser,

Let us know what you are thinking,


Hi Greg,

I'll almost finished with my work on the demangler, hopefully I'll be 
done within a few weeks. Once that's all finished I'll look into 
exporting the AST and comparing it to FastDemangle. I was thinking about 
adding a version of llvm::itaniumMangle() that returns a opaque handle 
to the AST and defining some functions on the LLVM side that take that 
handle and return some extra information. I'd be happy to help out with 
the LLDB side of things too, although it might be better if someone more 
experienced with LLDB did this.


I'll ping this thread when I'm finished with the demangler, then we can 
hopefully work out what a good API for LLDB would be.


Thanks,
Erik


Greg


Thanks,
Erik


On 2018-01-24 6:48 PM, Greg Clayton via lldb-dev wrote:

I have an issue where I am debugging a C++ binary that is around 250MB in size. 
It contains some mangled names that are crazy:

_ZNK3shk6detail17CallbackPublisherIZNS_5ThrowERKNSt15__exception_ptr13exception_ptrEEUlOT_E_E9SubscribeINS0_9ConcatMapINS0_18CallbackSubscriberIZNS_6GetAllIiNS1_IZZNS_9ConcatMapIZNS_6ConcatIJNS1_IZZNS_3MapIZZNS_7IfEmptyIS9_EEDaS7_ENKUlS6_E_clINS1_IZZNS_4TakeIiEESI_S7_ENKUlS6_E_clINS1_IZZNS_6FilterIZNS_9ElementAtEmEUlS7_E_EESI_S7_ENKUlS6_E_clINS1_IZZNSL_ImEESI_S7_ENKUlS6_E_clINS1_IZNS_4FromINS0_22InfiniteRangeContainerIiSI_S7_EUlS7_E_SI_S6_EUlS7_E_SI_S6_EUlS7_E_SI_S6_EUlS7_E_SI_S6_EUlS7_E_EESI_S7_ENKUlS6_E_clIS14_EESI_S6_EUlS7_E_EERNS1_IZZNSH_IS9_EESI_S7_ENKSK_IS14_EESI_S6_EUlS7_E0_ESI_DpOT_EUlS7_E_EESI_S7_ENKUlS6_E_clINS1_IZNS_5StartIJZNS_4JustIJS19_S1C_E

Re: [lldb-dev] Huge mangled names are causing long delays when loading symbol table symbols

2018-01-26 Thread Davide Italiano via lldb-dev
On Fri, Jan 26, 2018 at 8:38 AM, Erik Pilkington via lldb-dev
 wrote:
>
>
> On 2018-01-25 1:58 PM, Greg Clayton wrote:
>>>
>>> On Jan 25, 2018, at 10:25 AM, Erik Pilkington 
>>> wrote:
>>>
>>> Hi,
>>> I'm not at all familiar with LLDB, but I've been doing some work on the
>>> demangler in libcxxabi. It's still a work in progress and I haven't yet
>>> copied the changes over to ItaniumDemangle, which AFAIK is what lldb uses.
>>> The demangler in libcxxabi now demangles the symbol you attached in 3.31
>>> seconds, instead of 223.54 on my machine. I posted a RFC on my work here
>>> (http://lists.llvm.org/pipermail/llvm-dev/2017-June/114448.html), but
>>> basically the new demangler just produces an AST then traverses it to print
>>> the demangled name.
>>
>> Great to hear the huge speedup in demangling! LLDB actually has two
>> demanglers: a fast one that can demangle 99% of names, and we fall back to
>> ItaniumDemangle which can do all names but is really slow. It would be fun
>> to compare your new demangler with the fast one and see if we can get rid of
>> the fast demangler now.
>>>
>>>
>>> I think a good way of making this even faster is to have LLDB consume the
>>> AST the demangler produces directly. The AST is a better representation of
>>> the information that LLDB wants, and finishing the demangle and then fishing
>>> out that information from the output string is unfortunate. From the AST, it
>>> would be really straightforward to just individually print all the
>>> components of the name that LLDB wants.
>>
>> This would help us to grab the important bits out of the mangled name as
>> well. We chop up a demangled name to find the base name (string for
>> std::string), containing context (std:: for std::string) and we check if we
>> can tell if the function is a method (look for trailing "const" modifier on
>> the function) versus a top level function (since the mangling doesn't fully
>> specify what is a namespace and what is a class (like in "foo::bar::baz()"
>> we don't know if "foo" or "bar" are classes or namespaces. So the AST would
>> be great as long as it is fast.
>>
>>> Most of the time it takes to demangle these "symbols from hell" is during
>>> the printing, after the AST has been parsed, because the demangler has to
>>> flatten out all the potentially nested back references. Just parsing to an
>>> AST should be about proportional to the strlen of the mangled name. Since
>>> (AFAIK) LLDB doesn't use some sections of the demangled name often (such as
>>> parameters), from the AST LLDB could lazily decide not to even bother fully
>>> demangling some sections of the name, then if it ever needs them it could
>>> parse a new AST and get them from there. I think this would largely fix the
>>> issue, as most of the time these crazy expansions don't occur in the name
>>> itself, but in the parameters or return type. Even when they do appear in
>>> the name, it would be possible to do some simple name classification (ie,
>>> does this symbol refer to a function) or pull out the basename quickly
>>> without expanding anything at all.
>>>
>>> Any thoughts? I'm really not at all familiar with LLDB, so I could have
>>> this all wrong!
>>
>> AST sounds great. We can put this into the class we use to chop us C++
>> names as that is really our goal.
>>
>> So it would be great to do a speed comparison between our fast demangler
>> in LLDB (in FastDemangle.cpp/.h) and your updated libcxxabi version. If
>> yours is faster, remove FastDemangle and then update the
>> llvm::ItaniumDemangle() to use your new code.
>>
>> ASTs would be great for the C++ name parser,
>>
>> Let us know what you are thinking,
>
>
> Hi Greg,
>
> I'll almost finished with my work on the demangler, hopefully I'll be done
> within a few weeks. Once that's all finished I'll look into exporting the
> AST and comparing it to FastDemangle. I was thinking about adding a version
> of llvm::itaniumMangle() that returns a opaque handle to the AST and
> defining some functions on the LLVM side that take that handle and return
> some extra information. I'd be happy to help out with the LLDB side of
> things too, although it might be better if someone more experienced with
> LLDB did this.
>

That's great to hear. Not having 3 different demanglers scattered
between lldb and llvm will be a big win for everybody.

--
Davide
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Huge mangled names are causing long delays when loading symbol table symbols

2018-01-26 Thread Jim Ingham via lldb-dev
It's not just reduction of the number of demanglers we have to support, 
however.  

Greg and I both got excited by this proposal because we've had to maintain 
these name choppers for the tasks lldb has to do with mangled names - for 
instance matching incomplete human-typed in names - i.e. Class::method to match 
a method which is actually Namespace::Class::method.  

Having a structured representation of mangled names will be much more 
appropriate for the tasks lldb has to do with mangled names - for instance 
matching incomplete human-typed in names - i.e. Class::method to match a method 
which is actually Namespace::Class::method.  At present, we end up losing all 
the semantic information the demangler had when it parsed the mangled name, 
then trying to recreate that by hand to pick out the pieces of interest.

Greg did an experiment early on in lldb of having a node tree representation of 
mangled names, but it was too slow when you have to use it on every symbol in a 
module.  That's an important thing to remember for the debugger's use of the 
demangler.  Since we need to quickly find Namespace::Class::method when a 
somebody types Class::method we have to build up lookup tables up front for 
those pieces, and we don't always have debug information from which to grab the 
base name.  So whatever demangler we use has to survive getting passed all the 
C++ symbols in the libraries loaded by a normal program.

Another bonus of this work: we have the problem that a 700 character demangled 
name is just not useful in a backtrace.  If you have 20 frames of this one 
after another the display is really just noise...  We do some truncation of 
names, but figuring out how to truncate a name while preserving the parts that 
are actually useful to people is hard to do well if you don't understand the 
semantics of the name.  Erik Eckstein added a "display mode" to the swift 
demangler which only renders the most salient parts of the name.  The swift 
demangler does parse into a node tree so this was doable.  That made a big 
difference in the readability of backtraces in swift.  This is plumbed through 
the generic parts of lldb (Symbol::GetDisplayName & 
Mangled::GetDisplayDeangledName) but for C++ GetDisplayDemangledName just calls 
GetDemangled name.  It would be great to implement some reasonable version of 
this for C++ names as well.

Jim


> On Jan 26, 2018, at 9:01 AM, Davide Italiano via lldb-dev 
>  wrote:
> 
> On Fri, Jan 26, 2018 at 8:38 AM, Erik Pilkington via lldb-dev
>  wrote:
>> 
>> 
>> On 2018-01-25 1:58 PM, Greg Clayton wrote:
 
 On Jan 25, 2018, at 10:25 AM, Erik Pilkington 
 wrote:
 
 Hi,
 I'm not at all familiar with LLDB, but I've been doing some work on the
 demangler in libcxxabi. It's still a work in progress and I haven't yet
 copied the changes over to ItaniumDemangle, which AFAIK is what lldb uses.
 The demangler in libcxxabi now demangles the symbol you attached in 3.31
 seconds, instead of 223.54 on my machine. I posted a RFC on my work here
 (http://lists.llvm.org/pipermail/llvm-dev/2017-June/114448.html), but
 basically the new demangler just produces an AST then traverses it to print
 the demangled name.
>>> 
>>> Great to hear the huge speedup in demangling! LLDB actually has two
>>> demanglers: a fast one that can demangle 99% of names, and we fall back to
>>> ItaniumDemangle which can do all names but is really slow. It would be fun
>>> to compare your new demangler with the fast one and see if we can get rid of
>>> the fast demangler now.
 
 
 I think a good way of making this even faster is to have LLDB consume the
 AST the demangler produces directly. The AST is a better representation of
 the information that LLDB wants, and finishing the demangle and then 
 fishing
 out that information from the output string is unfortunate. From the AST, 
 it
 would be really straightforward to just individually print all the
 components of the name that LLDB wants.
>>> 
>>> This would help us to grab the important bits out of the mangled name as
>>> well. We chop up a demangled name to find the base name (string for
>>> std::string), containing context (std:: for std::string) and we check if we
>>> can tell if the function is a method (look for trailing "const" modifier on
>>> the function) versus a top level function (since the mangling doesn't fully
>>> specify what is a namespace and what is a class (like in "foo::bar::baz()"
>>> we don't know if "foo" or "bar" are classes or namespaces. So the AST would
>>> be great as long as it is fast.
>>> 
 Most of the time it takes to demangle these "symbols from hell" is during
 the printing, after the AST has been parsed, because the demangler has to
 flatten out all the potentially nested back references. Just parsing to an
 AST should be about proportional to the strlen of the mangled name. Since
 (AFAIK) LLDB doesn't use som

Re: [lldb-dev] Huge mangled names are causing long delays when loading symbol table symbols

2018-01-26 Thread Greg Clayton via lldb-dev

> On Jan 26, 2018, at 8:38 AM, Erik Pilkington  
> wrote:
> 
> 
> 
> On 2018-01-25 1:58 PM, Greg Clayton wrote:
>>> On Jan 25, 2018, at 10:25 AM, Erik Pilkington  
>>> wrote:
>>> 
>>> Hi,
>>> I'm not at all familiar with LLDB, but I've been doing some work on the 
>>> demangler in libcxxabi. It's still a work in progress and I haven't yet 
>>> copied the changes over to ItaniumDemangle, which AFAIK is what lldb uses. 
>>> The demangler in libcxxabi now demangles the symbol you attached in 3.31 
>>> seconds, instead of 223.54 on my machine. I posted a RFC on my work here 
>>> (http://lists.llvm.org/pipermail/llvm-dev/2017-June/114448.html), but 
>>> basically the new demangler just produces an AST then traverses it to print 
>>> the demangled name.
>> Great to hear the huge speedup in demangling! LLDB actually has two 
>> demanglers: a fast one that can demangle 99% of names, and we fall back to 
>> ItaniumDemangle which can do all names but is really slow. It would be fun 
>> to compare your new demangler with the fast one and see if we can get rid of 
>> the fast demangler now.
>>> 
>>> I think a good way of making this even faster is to have LLDB consume the 
>>> AST the demangler produces directly. The AST is a better representation of 
>>> the information that LLDB wants, and finishing the demangle and then 
>>> fishing out that information from the output string is unfortunate. From 
>>> the AST, it would be really straightforward to just individually print all 
>>> the components of the name that LLDB wants.
>> This would help us to grab the important bits out of the mangled name as 
>> well. We chop up a demangled name to find the base name (string for 
>> std::string), containing context (std:: for std::string) and we check if we 
>> can tell if the function is a method (look for trailing "const" modifier on 
>> the function) versus a top level function (since the mangling doesn't fully 
>> specify what is a namespace and what is a class (like in "foo::bar::baz()" 
>> we don't know if "foo" or "bar" are classes or namespaces. So the AST would 
>> be great as long as it is fast.
>> 
>>> Most of the time it takes to demangle these "symbols from hell" is during 
>>> the printing, after the AST has been parsed, because the demangler has to 
>>> flatten out all the potentially nested back references. Just parsing to an 
>>> AST should be about proportional to the strlen of the mangled name. Since 
>>> (AFAIK) LLDB doesn't use some sections of the demangled name often (such as 
>>> parameters), from the AST LLDB could lazily decide not to even bother fully 
>>> demangling some sections of the name, then if it ever needs them it could 
>>> parse a new AST and get them from there. I think this would largely fix the 
>>> issue, as most of the time these crazy expansions don't occur in the name 
>>> itself, but in the parameters or return type. Even when they do appear in 
>>> the name, it would be possible to do some simple name classification (ie, 
>>> does this symbol refer to a function) or pull out the basename quickly 
>>> without expanding anything at all.
>>> 
>>> Any thoughts? I'm really not at all familiar with LLDB, so I could have 
>>> this all wrong!
>> AST sounds great. We can put this into the class we use to chop us C++ names 
>> as that is really our goal.
>> 
>> So it would be great to do a speed comparison between our fast demangler in 
>> LLDB (in FastDemangle.cpp/.h) and your updated libcxxabi version. If yours 
>> is faster, remove FastDemangle and then update the llvm::ItaniumDemangle() 
>> to use your new code.
>> 
>> ASTs would be great for the C++ name parser,
>> 
>> Let us know what you are thinking,
> 
> Hi Greg,
> 
> I'll almost finished with my work on the demangler, hopefully I'll be done 
> within a few weeks. Once that's all finished I'll look into exporting the AST 
> and comparing it to FastDemangle. I was thinking about adding a version of 
> llvm::itaniumMangle() that returns a opaque handle to the AST and defining 
> some functions on the LLVM side that take that handle and return some extra 
> information. I'd be happy to help out with the LLDB side of things too, 
> although it might be better if someone more experienced with LLDB did this.
> 

Can't wait! The only reason we switched away from the libcxxabi demangler in 
the first place was the poor performance. GDB's demangler was 3x faster. Our 
FastDemangler made got back to the speed of the GDB demangler. But it will be 
great to get back to one fast demangler. 

It would be great if there was some way to implement the demangled name size 
cutoff in the demangler where if the detangled names goes over some max size we 
can just stop demangling. No one needs to see a 72MB string, not would anyone 
ever type in that name.

If you can get the new demangler features (AST + demangling) into 
llvm::itaniumMangle I will be happy to do the LLDB side of the work

> I'll ping this thread when I'm finished with the demangler

[lldb-dev] Rust language support question

2018-01-26 Thread Tom Tromey via lldb-dev
Hi.  I'm working on adding Rust language support to lldb.

One question that's come up is the best way to handle expression
parsing.

On the one hand, it would be convenient to reuse an existing parser --
the one that we discussed was the "syn" crate.  But, this is a Rust
program.

So then there's the question of how to ship it.  Directly using the syn
crate would mean having Rust code in-tree.  Or, perhaps the Rust parts
could be shipped as a shared library or an external executable.

Are either of these doable?  What do other language plugins do?

My original plan here was to simply make the entire language support an
external plugin.  But, from what I can tell this isn't possible -- the
necessary DWARF-related headers aren't installed.  So maybe this could
be changed?  This would provide us with the most flexibility I think.

A final idea is to do what I did for gdb, and simply write a new parser
in C++.  Doable, but for me I think a last resort.

thanks,
Tom
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Rust language support question

2018-01-26 Thread Greg Clayton via lldb-dev

> On Jan 26, 2018, at 2:54 PM, Tom Tromey via lldb-dev 
>  wrote:
> 
> Hi.  I'm working on adding Rust language support to lldb.
> 
> One question that's come up is the best way to handle expression
> parsing.
> 
> On the one hand, it would be convenient to reuse an existing parser --
> the one that we discussed was the "syn" crate.  But, this is a Rust
> program.

We have great support for multiple language in LLDB. First off you will need to 
subclass lldb_private::TypeSystem. See lldb_private::ClangASTContext for the 
most complete version. Your type system allows you to use an abstract syntax 
tree that makes sense for your language. For clang, we actually create a 
clang::ASTContext using the code from the compiler and then we translate debug 
info from DWARF into clang types. So you would probably want to make a 
"lldb_private::RustASTContext". Then, if you use DWARF as your debug info, you 
will want to make subclass DWARFASTParser with something like 
DWARFASTParserRust. This is where you will translate types from DWARF back into 
your custom AST types. TypeSystem then becomes your main type abstraction 
within LLDB. There are many virtual functions in it that you must override and 
some that you might want to override. Types in LLDB are handed out as 
"lldb_private::CompilerType". This class contains a "lldb_private::TypeSystem 
*" plus a "void *" which can point to what ever makes sense if your 
lldb_private::RustASTContext class. Similarly CompilerDecl and 
CompilerDeclContext also have a "lldb_private::TypeSystem *" plus a "void *" so 
you can hand out declarations and namespaces, etc.

When it comes to expressions, we will dig up the right lldb_private::TypeSystem 
for a given stack frame language and then we will call 
lldb_private::TypeSystem::GetUserExpression(...) and your type system can 
evaluate your expression exactly as your language should.

> 
> So then there's the question of how to ship it.  Directly using the syn
> crate would mean having Rust code in-tree.  Or, perhaps the Rust parts
> could be shipped as a shared library or an external executable.

You can make your lldb_private::RustASTContext have a custom setting in LLDB 
that specifies a path to a shared library to load. You could keep all of the 
functionality out of LLDB that way, just have lldb_private::RustASTContext shim 
over to the shared library to do the work. Or you can build it right into LLDB. 
It really depends on how sharable your compiler code is and if the code can be 
used in another program.
> 
> Are either of these doable?  What do other language plugins do?

It is. All other languages compile in their support right now, but that is 
mostly due to the lack of a stable API in clang. Clang has no library interface 
that exports all of the details that we need, so we just build it into LLDB. 
Swift does the same thing: a full compiler is included in LLDB itself. Go and 
Java and OCaml all write their own minimal debug layer that doesn't depend on 
the compiler codebase at all. 

It takes some work to build your compiler so that is can share its 
implementation with the debugger, but the investment can pay off:
- expressions in clang can use latest language features just by updating code
- no need to maintain a separate expression parser that constantly gets out of 
date with the current compiler
- no need to invent a type system, just use your native AST. This also helps 
ensure you can recreate any types from DWARF since if some info is missing in 
DWARF, you won't be able to convert it back into your AST format without losing 
something

Swift did a slight different thing that you might want to think about: if your 
compiler can serialize an AST when it builds your program, you can put that 
serialized AST into the executable, along with debug info. When you debug, you 
can deserialize the AST and hand it right back to the compiler! With Swift that 
was great because the compiler guys would change the language and add new 
features. As they did this, we wouldn't always have work to do in LLDB, because 
they would store a blob of info in the binary, and we would hand it back to 
them. The debug info didn't actually have types in it, the DWARF just had 
mangled names for the types. We take those mangled names, and hand them to the 
compiler code, and it would use the mangled names to locate the type and hand 
it back to us. So we didn't need to know anything about the type, just hand a 
string to the compiler and it would hand us back a type. Swift also has 
generics, like a "Dictionary" and even these types aren't each 
contained in the AST, but they can be re-created on the fly by the compiler 
code. This all was hidden behind the "there is a mangle type name, please give 
me a type back".

> 
> My original plan here was to simply make the entire language support an
> external plugin.  But, from what I can tell this isn't possible -- the
> necessary DWARF-related headers aren't installed.  So maybe this c

Re: [lldb-dev] Rust language support question

2018-01-26 Thread Jim Ingham via lldb-dev
I second Greg, we don't really want to make the lldb_private namespace into 
API.  We don't have any intention to stabilize this, and the interfaces expose 
a lot of clang & llvm types which are also not stable and change not 
infrequently. So in effect your plugin would end up being rev-locked to the 
version of lldb it was built with anyway.  I'm not sure you'd gain all that 
much by trying to make an external plugin.  The plugin structure on disk is 
fairly well separated language by language, so it wouldn't be hard to overlay 
your support and then build in-tree even if your code isn't in the llvm.org 
repository.  OTOH I would be happy to see Rust support in the main-line lldb, 
so I'm not encouraging this approach.

This was implicit in what Greg said but unlike gdb, lldb doesn't have a 
language neutral internal representation for types.  Rather each language can 
provide it's own internal representation for types.  The current Rust support 
in lldb amounts to telling lldb that "Rust types can be handled by clang's type 
representation."  I naively thought this would make support for Rust weak, but 
folks on Stack Overflow say it actually works pretty well for viewing variables 
(using "frame var" or lldb's ValueObject's).  Stepping and so forth apparently 
seemed to be working okay as well.  Depending on how far off this actually is, 
you might be able to reuse the Clang TypeSystem and do mutatis mutandis for the 
differences?  That would certainly be a lot simpler than inventing another type 
representation.

Note, I think the jury's still out on whether it was a great idea to have the 
swift type representation be the swift compiler's internal state.  It has 
proven more than a little fragile.  I'm not sure I would suggest that route.  I 
vaguely remember back in the day there was a -g flag in gcc that produced a 
compiler state dump that gdb was supposed to read.  But IIRC that ended up 
being more trouble than it was worth.

Anyway, to the point of using Rust code in the lldb codebase.  I don't think 
that we want to have lldb's build depend on having a Rust compiler present.  
That seems like a fairly heavy-weight requirement for all the folks who don't 
work on Rust.  But at least I wouldn't mind having some code that was 
conditionalized on the presence of the Rust compiler, and when it was absent 
the Rust TypeSystem would just return a no-op expression parser.  I don't know 
how other people feel about that, however.  Of course, this would only be worth 
the effort if your Rust compiler front-end was planning to call back to lldb 
say for name & type resolution.  If it is stand-alone then it might as well be 
some external library you call out to.  If all the parser library needed to do 
was call back to query for name resolution as it went along, though you very 
well could do that using the SB API's.  Those we do support writing external 
libraries against, our strong policy is to maintain binary compatibility 
through the SB API's.

Another little quirk of lldb that might affect your planning is that unlike 
gdb, we don't have full calling-convention emulation in lldb at present.  So we 
don't have the ability to marshal the arguments for any old randomly 
complicated function.  That means the gdb strategy of parsing the expression 
till you get to a point where you want to call a function, then just hand 
calling it off to one side, isn't currently possible.  The Go Expression parser 
just fails if there's any expression that requires a function call.

The way we get around this in lldb is that we marshal the inputs to the 
expression as well as the return type into an lldb-defined struct we cons up 
for the expression.  Then we write a wrapper function that contains the 
expression but uses the input values from the argument struct.  That way we 
only need to call a function that gets passed in a pointer to this argument 
struct, which tends to be pretty easy to do. Then we JIT and insert that 
function, create the input argument structure and call our wrapper function.  
That has some nice side-benefits like for breakpoint condition expressions we 
only need to parse & insert the expression once and then we can pretty cheaply 
call it on each breakpoint hit.  In actuality it's a little more complicated 
than this, but anyway it means a lot of work gets offloaded to the 
ExpressionParser, but we avoid having to encode the calling convention in 
detail in lldb.  

Since we already have a compiler sitting around that seemed a good "don't redo 
work" trade-off.  As a side note, it was also necessitated by the fact that 
though llvm knows about the calling convention, it didn't for a long time know 
how to express that in terms of clang types.  And there wasn't an easy way to 
cross the clang type -> llvm type barrier.  IIUC, that barrier is no longer 
present, but there still aren't useful API's to express "for an argument list 
of these Clang types, where would the arguments go.  So if 

Re: [lldb-dev] Rust language support question

2018-01-26 Thread Jim Ingham via lldb-dev
Jason points out this was gdb writing out a binary form of gdb's psymtabs to be 
a cheap accelerator table.  Anyway, having the data representation of debug 
information depend on the internal state of either the compiler or debugger is 
a fragile thing...

Jim


> On Jan 26, 2018, at 6:16 PM, Jim Ingham  wrote:
> 
> Note, I think the jury's still out on whether it was a great idea to have the 
> swift type representation be the swift compiler's internal state.  It has 
> proven more than a little fragile. I'm not sure I would suggest that route.  
> I vaguely remember back in the day there was a -g flag in gcc that produced a 
> compiler state dump that gdb was supposed to read.  But IIRC that ended up 
> being more trouble than it was worth.
> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev