Re: [lldb-dev] Unwinding call frames with separated data and return address stacks

2019-03-05 Thread Thomas Goodfellow via lldb-dev
Hi Jason

Thanks for the advice - I've been surprised overall how capable DWARF
expressions are so wouldn't have been surprised to learn that there is
also a category of pseudo-variables (not that I can think of any
others, or other circumstances where it would be useful: the usual
combined code/data stack is ubiquitous). The RestoreType suggestion is
interesting as it might be a less-intrusive change.

Cheers,
Tom

On Mon, 4 Mar 2019 at 22:05, Jason Molenda  wrote:
>
> Hi Tom, interesting problem you're working on there.
>
> I'm not sure any of the DWARF expression operators would work here.  You want 
> to have an expression that works for a given frame, saying "to find the 
> caller's pc value, look at the saved-pc stack, third entry from the bottom of 
> that stack."  But that would require generating a different DWARF expression 
> for the frame each time it shows up in a backtrace - which is unlike lldb's 
> normal design of having an UnwindPlan for a function which is computed once 
> and reused for the duration of the debug session.
>
> I supposed you could add a user-defined DW_OP which means "get the current 
> stack frame number" and then have your expression deref the emulated saved-pc 
> stack to get the value?
>
> lldb uses an intermediate representation of unwind information (UnwindPlan) 
> which will use a DWARF expression, but you could also add an entry to 
> UnwindPlan::Row::RegisterLocation::RestoreType which handled this, I suppose.
>
>
> > On Mar 4, 2019, at 2:46 AM, Thomas Goodfellow via lldb-dev 
> >  wrote:
> >
> > I'm adding LLDB support for an unconventional platform which uses two
> > stacks: one purely for return addresses and another for frame context
> > (spilled registers, local variables, etc). There is no explicit link
> > between the two stacks, i.e. the frame context doesn't include any
> > pointer or index to identify the return address: the epilog for a
> > subroutine amounts to unwinding the frame context then finally popping
> > the top return address from the return stack. It has some resemblance
> > to the Intel CET scheme of shadow stacks, but without the primary
> > stack having a copy of the return address.
> >
> > I can extend the emulation of the platform to better support LLDB. For
> > example while the real hardware platform provides no access to the
> > return address stack the emulation can expose it in the memory map,
> > provide an additional debug register for querying it, etc, which DWARF
> > expressions could then extract return addresses from. However doing
> > this seems to require knowing the frame number and I haven't found a
> > way of doing this (a pseudo-register manipulated by DWARF expressions
> > worked but needed some LLDB hacks to sneak it through the existing
> > link register handling, also seemed likely to be unstable against LLDB
> > implementation changes)
> >
> > Is there a way to access the call frame number (or a reliable proxy)
> > from a DWARF expression? Or an existing example of unwinding a shadow
> > stack?
> >
> > Thanks,
> > Tom
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [Release-testers] [8.0.0 Release] rc3 has been tagged

2019-03-05 Thread Hans Wennborg via lldb-dev
Many thanks!

I've added all binaries I've got so far to the pre-release page now
(the CDN might take a while to invalidate its cache).

On Tue, Mar 5, 2019 at 1:56 AM Brian Cain  wrote:
>
> Uploaded SLES11, Ubuntu 14.04, 18.04 binaries.
>
> 9e59c0c667cd67c61600470d2e3107576b4707c8  
> clang+llvm-8.0.0-rc3-x86_64-linux-gnu-ubuntu-14.04.tar.xz
> 0fd5690c11f045ae4d356619620778b0c17942ff  
> clang+llvm-8.0.0-rc3-x86_64-linux-gnu-ubuntu-18.04.tar.xz
> 6d1dc9b31d4f4d815b7cd18c8446404dc2a80a69  
> clang+llvm-8.0.0-rc3-x86_64-linux-sles11.3.tar.xz
>
>
>
>
> On Wed, Feb 27, 2019 at 12:51 PM Hans Wennborg via Release-testers 
>  wrote:
>>
>> Dear testers,
>>
>> 8.0.0-rc3 was just tagged from the release_80 branch at r355015.
>>
>> We're running a little behind schedule now, but I think we're also
>> close to be able to call this done.
>>
>> Please take a close look at this release candidate. Unless anything
>> bad comes up, this is probably very similar to what the final release
>> will look like.
>>
>> Testers, please run the test script, share your results, and upload binaries.
>>
>> I'll publish source tarballs and docs as soon as possible, and
>> binaries as they become available.
>>
>> Thanks,
>> Hans
>> ___
>> Release-testers mailing list
>> release-test...@lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/release-testers
>
>
>
> --
> -Brian
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


[lldb-dev] Build failure on Linux

2019-03-05 Thread Gábor Márton via lldb-dev
Hi,

On trunk I receive the following build error on Linux:
../../git/llvm/tools/lldb/include/lldb/lldb-forward.h:181:7: note:
forward declaration of 'lldb_private::ProcessInstanceInfoList'
class ProcessInstanceInfoList;
  ^
../../git/llvm/tools/lldb/source/Host/linux/Host.cpp:260:23: error:
member access into incomplete type
'lldb_private::ProcessInstanceInfoList'
  return process_infos.GetSize();

When I put back the below line into linux/Host.cpp
 #include "lldb/Target/Process.h"
the error is gone.

Based on this info, this commit could be blamed:
commit e9a520afd257bf71f7ead0d33e4734008abf690c
Author: Zachary Turner 
AuthorDate: Mon Mar 4 21:51:03 2019 +
Commit: Zachary Turner 
CommitDate: Mon Mar 4 21:51:03 2019 +

Move ProcessInfo from Host to Utility.

Could you please take a look?

Thanks,
Gabor
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] [cfe-dev] [Release-testers] [8.0.0 Release] rc3 has been tagged

2019-03-05 Thread Dan Walmsley via lldb-dev
Hans,

   Where can I find a win64 bit libclang.dll to test.

Iv just updated from 6.0 to 7.0 and tried 7.01 and found that libclang 7+ 
doesn't recognise my include arguments.

Downgrading to 6.0 fixes the issue for me.

Kind Regards

Dan

From: cfe-dev  on behalf of Hans Wennborg via 
cfe-dev 
Sent: 05 March 2019 09:28
To: Brian Cain
Cc: llvm-dev; Release-testers; clangd-...@lists.llvm.org; openmp-dev 
(openmp-...@lists.llvm.org); LLDB Dev; cfe-dev; libcxx-...@lists.llvm.org
Subject: Re: [cfe-dev] [Release-testers] [8.0.0 Release] rc3 has been tagged

Many thanks!

I've added all binaries I've got so far to the pre-release page now
(the CDN might take a while to invalidate its cache).

On Tue, Mar 5, 2019 at 1:56 AM Brian Cain  wrote:
>
> Uploaded SLES11, Ubuntu 14.04, 18.04 binaries.
>
> 9e59c0c667cd67c61600470d2e3107576b4707c8  
> clang+llvm-8.0.0-rc3-x86_64-linux-gnu-ubuntu-14.04.tar.xz
> 0fd5690c11f045ae4d356619620778b0c17942ff  
> clang+llvm-8.0.0-rc3-x86_64-linux-gnu-ubuntu-18.04.tar.xz
> 6d1dc9b31d4f4d815b7cd18c8446404dc2a80a69  
> clang+llvm-8.0.0-rc3-x86_64-linux-sles11.3.tar.xz
>
>
>
>
> On Wed, Feb 27, 2019 at 12:51 PM Hans Wennborg via Release-testers 
>  wrote:
>>
>> Dear testers,
>>
>> 8.0.0-rc3 was just tagged from the release_80 branch at r355015.
>>
>> We're running a little behind schedule now, but I think we're also
>> close to be able to call this done.
>>
>> Please take a close look at this release candidate. Unless anything
>> bad comes up, this is probably very similar to what the final release
>> will look like.
>>
>> Testers, please run the test script, share your results, and upload binaries.
>>
>> I'll publish source tarballs and docs as soon as possible, and
>> binaries as they become available.
>>
>> Thanks,
>> Hans
>> ___
>> Release-testers mailing list
>> release-test...@lists.llvm.org
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.llvm.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Frelease-testers&data=02%7C01%7C%7Ca5608d569df1498230d908d6a14d31d2%7C84df9e7fe9f640afb435%7C1%7C0%7C636873750257381088&sdata=MBPbkBr6r8F0jMSZn2ZdsAIGMqUDNBhj2rFkrdAK58g%3D&reserved=0
>
>
>
> --
> -Brian
___
cfe-dev mailing list
cfe-...@lists.llvm.org
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.llvm.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fcfe-dev&data=02%7C01%7C%7Ca5608d569df1498230d908d6a14d31d2%7C84df9e7fe9f640afb435%7C1%7C0%7C636873750257381088&sdata=JZF%2F2NmiEsUKHzL%2Bb5x1VQQeq53qcGIGYmy4zAHHCyI%3D&reserved=0
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Unwinding call frames with separated data and return address stacks

2019-03-05 Thread Pavel Labath via lldb-dev

On 04/03/2019 11:46, Thomas Goodfellow via lldb-dev wrote:

I'm adding LLDB support for an unconventional platform which uses two
stacks: one purely for return addresses and another for frame context
(spilled registers, local variables, etc). There is no explicit link
between the two stacks, i.e. the frame context doesn't include any
pointer or index to identify the return address: the epilog for a
subroutine amounts to unwinding the frame context then finally popping
the top return address from the return stack. It has some resemblance
to the Intel CET scheme of shadow stacks, but without the primary
stack having a copy of the return address.

I can extend the emulation of the platform to better support LLDB. For
example while the real hardware platform provides no access to the
return address stack the emulation can expose it in the memory map,
provide an additional debug register for querying it, etc, which DWARF
expressions could then extract return addresses from. However doing
this seems to require knowing the frame number and I haven't found a
way of doing this (a pseudo-register manipulated by DWARF expressions
worked but needed some LLDB hacks to sneak it through the existing
link register handling, also seemed likely to be unstable against LLDB
implementation changes)

Is there a way to access the call frame number (or a reliable proxy)
from a DWARF expression? Or an existing example of unwinding a shadow
stack?



I'm not sure I fully understood your setup, but it seems to me that this 
could be easily fixed if, in addition to the "fake" memory map, you 
could provide a fake "stack pointer" register which points to it.


Then, it should be possible to express the unwind info in regular 
debug_frame syntax:

previous_IP := [ fake_SP ]
previous_fake_SP := fake_SP +/- sizeof(IP)

regards,
pavel
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Remote debugging a docker process

2019-03-05 Thread Jan Kratochvil via lldb-dev
On Sun, 03 Mar 2019 04:17:42 +0100, Mason Kramer via lldb-dev wrote:
> Then, I restricted the acceptable range of
> gdbserver ports to just 5001, using the flags suggested in the email.
> 
> lldb-server-4.0 platform --verbose --listen "*:5000" --min-gdbserver-port 
> 5001 --max-gdbserver-port 5001

  
There should be 5002 as --max-gdbserver-port is in reality "max+1" value and
not "max" value.

The options --min-gdbserver-port and --max-gdbserver-port get ignored this way.

Submitted:
Sanity check --max-gdbserver-port
https://reviews.llvm.org/D58962

Replied with a working Docker example:
Remote LLDB debugging - Docker container
https://stackoverflow.com/a/55004346/2995591


# Add an option 'allow-all-hosts' to permit lldb debugging inside a Docker 
container
> [2] https://reviews.llvm.org/D42845

I also haven't found a need for this new proposed option.


Jan
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Build failure on Linux

2019-03-05 Thread Davide Italiano via lldb-dev
I think this might have been fixed, try to pull.

On Tue, Mar 5, 2019 at 3:10 AM Gábor Márton via lldb-dev
 wrote:
>
> Hi,
>
> On trunk I receive the following build error on Linux:
> ../../git/llvm/tools/lldb/include/lldb/lldb-forward.h:181:7: note:
> forward declaration of 'lldb_private::ProcessInstanceInfoList'
> class ProcessInstanceInfoList;
>   ^
> ../../git/llvm/tools/lldb/source/Host/linux/Host.cpp:260:23: error:
> member access into incomplete type
> 'lldb_private::ProcessInstanceInfoList'
>   return process_infos.GetSize();
>
> When I put back the below line into linux/Host.cpp
>  #include "lldb/Target/Process.h"
> the error is gone.
>
> Based on this info, this commit could be blamed:
> commit e9a520afd257bf71f7ead0d33e4734008abf690c
> Author: Zachary Turner 
> AuthorDate: Mon Mar 4 21:51:03 2019 +
> Commit: Zachary Turner 
> CommitDate: Mon Mar 4 21:51:03 2019 +
>
> Move ProcessInfo from Host to Utility.
>
> Could you please take a look?
>
> Thanks,
> Gabor
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


[lldb-dev] new tool (core2yaml) + a new top-level library (Formats)

2019-03-05 Thread Pavel Labath via lldb-dev

Hello all,

I have just posted a large-ish patch series for review (D58971, D58973, 
D58975, D58976), and I want to use this opportunity to draw more 
attention to it and highlight various bikeshedding 
opportunities^H^H^Htopics for discussion :).


The new tool is called core2yaml, and it's goal is to fill the gap in 
the testing story for core files. As you might know, at present, the 
only way to test core file parsing code (*) is to check in an opaque 
binary blob and have the debugger open that. This presents a couple of 
challenges:

- it's really hard to review what is inside the core file
- one has to jump through various hoops to create a "small" core file
This tools fixes both issues by enabling one to check in text files, 
with human-readable content. The yaml files can also be easily edited to 
prune out the content which is not relevant for the test. While that's 
not my goal at present, I am hoping that this will one day enable us to 
write self-contained tests for the unwinder, as the core file can be 
used to synthesize (or capture&reduce) interesting unwinder scenarios.


Since I also needed to find a home for the new code I was writing, I 
thought this would be good opportunity to create a new library for 
various stuff. The goals I was trying to solve are:
- make the yaml code a library. The reason for that is that we have a 
number of unittests using checked in binaries, and I thought it would be 
nice to be able to convert those to use yaml representation as well.
- make the existing minidump parsing code more easily accessible. The 
parsing code currently lives in source/Plugins/Process/minidump, and is 
impossible to use it without pulling in the rest of lldb (which the tool 
doesn't need).
The solution I came up with here is a new "Formats" library. I chose a 
fairly generic name, because I realized that we have code for 
(de)serializing a bunch of small formats, which don't really have a good 
place to live in. Currently I needed a parser for linux /proc/PID/maps 
files and minidump files, but I am hoping that a generic name would 
enable us to one day move the gdb-remote protocol code there (which is 
also currently buried in some plugin code, which makes it hard to depend 
on from lldb-server), as well as the future debug-info-server, if it 
ever comes into existence.


Discussion topic #1: The library name and scope.
There are lost of other ways this could be organized. One of the names I 
considered was "BinaryFormat" for symmetry with llvm, but then I chose 
to drop the "Binary" part as it seemed to me we have plenty of 
non-binary formats as well. As for it's dependencies I currently have it 
depending on Utility and nothing else (as far as lldb libraries go). I 
can imagine using some Host code might be useful there too, but I would 
like to avoid any other lldb dependencies right now. Another question is 
whether this should be a single library or a bunch of smaller ones. I 
chose a single library now because the things I initially plan to put 
there are fairly small (/proc/pid/maps parser is 200 LOC), but I can see 
how we may want to create sub-libraries for things that grow big (the 
debug-info server code might turn out to be one of those) or that have 
some additional dependencies.


Discussion topic #2: tool name and scope
A case could be made to integrate this functionality into the llvm 
yaml2obj utilities. Here I chose not to do that because the minidump 
format is not at all implemented in llvm, and I do not see a use case 
for it to be implemented/moved there. A stronger case could be made to 
put the elf core code there, since llvm already supports reading elf 
files. While originally being in favour of that, I eventually adopted 
the view that doing this in lldb would be better because:

- it would bring more symmetry with minidumps
- it would enable us to do fine-grained yamlization for things that we 
care about (e.g., registers), which is something that would probably be 
uninteresting to the rest of llvm.


Discussion topic #3: Use of .def files in lldb. In one of the patches a 
create a .def textual header to be used for avoiding repetitive code 
when dealing various constants. This is fairly common practice in llvm, 
but would be a first in lldb.


Discussion topic #4: Overlap with "process plugin dump". This tool has 
some overlap with the given command for minidump files, which also 
provides a textual description of minidump files. In case we are ok with 
tweaking the interface of that command slightly (and ok with some yaml 
artefacts in it's output), it should be possible to reimplement that 
command on top of the yaml serialization library.


Discussion topic #5: Anything else I haven't thought of.

regards,
pavel

(*) This is not entirely true for MachO core files, where yaml2obj is 
already able to convert the core files into text form. However, it is 
definitely true for ELF and minidump core files, and even the MachO yaml 
for isn't that well s

Re: [lldb-dev] Build failure on Linux

2019-03-05 Thread Gábor Márton via lldb-dev
Yes, it works. Thanks!

On Tue, Mar 5, 2019 at 4:54 PM Davide Italiano  wrote:
>
> I think this might have been fixed, try to pull.
>
> On Tue, Mar 5, 2019 at 3:10 AM Gábor Márton via lldb-dev
>  wrote:
> >
> > Hi,
> >
> > On trunk I receive the following build error on Linux:
> > ../../git/llvm/tools/lldb/include/lldb/lldb-forward.h:181:7: note:
> > forward declaration of 'lldb_private::ProcessInstanceInfoList'
> > class ProcessInstanceInfoList;
> >   ^
> > ../../git/llvm/tools/lldb/source/Host/linux/Host.cpp:260:23: error:
> > member access into incomplete type
> > 'lldb_private::ProcessInstanceInfoList'
> >   return process_infos.GetSize();
> >
> > When I put back the below line into linux/Host.cpp
> >  #include "lldb/Target/Process.h"
> > the error is gone.
> >
> > Based on this info, this commit could be blamed:
> > commit e9a520afd257bf71f7ead0d33e4734008abf690c
> > Author: Zachary Turner 
> > AuthorDate: Mon Mar 4 21:51:03 2019 +
> > Commit: Zachary Turner 
> > CommitDate: Mon Mar 4 21:51:03 2019 +
> >
> > Move ProcessInfo from Host to Utility.
> >
> > Could you please take a look?
> >
> > Thanks,
> > Gabor
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] Unwinding call frames with separated data and return address stacks

2019-03-05 Thread Jason Molenda via lldb-dev
Yeah, if you don't need to find a way to express this in DWARF, then adding a 
type to RestoreType would be very simple.  lldb maps all the different unwind 
sources (debug_frame, eh_frame, arm index, compact unwind, assembly instruction 
scanning) into its internal intermediate representation (UnwindPlan) - so if 
you had an assembly-scanning unwind implementation for your target, you could 
add the appropriate RestoreType's.  There are also architectural default unwind 
plans that are provided by the ABI plugin, both a default one (usually 
appropriate for frames up the stack) and an unwind plan that is valid at the 
first instruction of a function.  These are good starting points for a new 
port, where you won't step through the prologue/epilogue correctly, but once 
you're in the middle of a function they can do a correct unwind on most 
architectures.

J

> On Mar 5, 2019, at 12:09 AM, Thomas Goodfellow  
> wrote:
> 
> Hi Jason
> 
> Thanks for the advice - I've been surprised overall how capable DWARF
> expressions are so wouldn't have been surprised to learn that there is
> also a category of pseudo-variables (not that I can think of any
> others, or other circumstances where it would be useful: the usual
> combined code/data stack is ubiquitous). The RestoreType suggestion is
> interesting as it might be a less-intrusive change.
> 
> Cheers,
> Tom
> 
> On Mon, 4 Mar 2019 at 22:05, Jason Molenda  wrote:
>> 
>> Hi Tom, interesting problem you're working on there.
>> 
>> I'm not sure any of the DWARF expression operators would work here.  You 
>> want to have an expression that works for a given frame, saying "to find the 
>> caller's pc value, look at the saved-pc stack, third entry from the bottom 
>> of that stack."  But that would require generating a different DWARF 
>> expression for the frame each time it shows up in a backtrace - which is 
>> unlike lldb's normal design of having an UnwindPlan for a function which is 
>> computed once and reused for the duration of the debug session.
>> 
>> I supposed you could add a user-defined DW_OP which means "get the current 
>> stack frame number" and then have your expression deref the emulated 
>> saved-pc stack to get the value?
>> 
>> lldb uses an intermediate representation of unwind information (UnwindPlan) 
>> which will use a DWARF expression, but you could also add an entry to 
>> UnwindPlan::Row::RegisterLocation::RestoreType which handled this, I suppose.
>> 
>> 
>>> On Mar 4, 2019, at 2:46 AM, Thomas Goodfellow via lldb-dev 
>>>  wrote:
>>> 
>>> I'm adding LLDB support for an unconventional platform which uses two
>>> stacks: one purely for return addresses and another for frame context
>>> (spilled registers, local variables, etc). There is no explicit link
>>> between the two stacks, i.e. the frame context doesn't include any
>>> pointer or index to identify the return address: the epilog for a
>>> subroutine amounts to unwinding the frame context then finally popping
>>> the top return address from the return stack. It has some resemblance
>>> to the Intel CET scheme of shadow stacks, but without the primary
>>> stack having a copy of the return address.
>>> 
>>> I can extend the emulation of the platform to better support LLDB. For
>>> example while the real hardware platform provides no access to the
>>> return address stack the emulation can expose it in the memory map,
>>> provide an additional debug register for querying it, etc, which DWARF
>>> expressions could then extract return addresses from. However doing
>>> this seems to require knowing the frame number and I haven't found a
>>> way of doing this (a pseudo-register manipulated by DWARF expressions
>>> worked but needed some LLDB hacks to sneak it through the existing
>>> link register handling, also seemed likely to be unstable against LLDB
>>> implementation changes)
>>> 
>>> Is there a way to access the call frame number (or a reliable proxy)
>>> from a DWARF expression? Or an existing example of unwinding a shadow
>>> stack?
>>> 
>>> Thanks,
>>> Tom
>>> ___
>>> lldb-dev mailing list
>>> lldb-dev@lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev


Re: [lldb-dev] new tool (core2yaml) + a new top-level library (Formats)

2019-03-05 Thread Jonas Devlieghere via lldb-dev
Hi Pavel,

On Tue, Mar 5, 2019 at 8:31 AM Pavel Labath via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> Hello all,
>
> I have just posted a large-ish patch series for review (D58971, D58973,
> D58975, D58976), and I want to use this opportunity to draw more
> attention to it and highlight various bikeshedding
> opportunities^H^H^Htopics for discussion :).
>
> The new tool is called core2yaml, and it's goal is to fill the gap in
> the testing story for core files. As you might know, at present, the
> only way to test core file parsing code (*) is to check in an opaque
> binary blob and have the debugger open that. This presents a couple of
> challenges:
> - it's really hard to review what is inside the core file
> - one has to jump through various hoops to create a "small" core file
> This tools fixes both issues by enabling one to check in text files,
> with human-readable content. The yaml files can also be easily edited to
> prune out the content which is not relevant for the test. While that's
> not my goal at present, I am hoping that this will one day enable us to
> write self-contained tests for the unwinder, as the core file can be
> used to synthesize (or capture&reduce) interesting unwinder scenarios.
>
> Since I also needed to find a home for the new code I was writing, I
> thought this would be good opportunity to create a new library for
> various stuff. The goals I was trying to solve are:
> - make the yaml code a library. The reason for that is that we have a
> number of unittests using checked in binaries, and I thought it would be
> nice to be able to convert those to use yaml representation as well.
> - make the existing minidump parsing code more easily accessible. The
> parsing code currently lives in source/Plugins/Process/minidump, and is
> impossible to use it without pulling in the rest of lldb (which the tool
> doesn't need).
> The solution I came up with here is a new "Formats" library. I chose a
> fairly generic name, because I realized that we have code for
> (de)serializing a bunch of small formats, which don't really have a good
> place to live in. Currently I needed a parser for linux /proc/PID/maps
> files and minidump files, but I am hoping that a generic name would
> enable us to one day move the gdb-remote protocol code there (which is
> also currently buried in some plugin code, which makes it hard to depend
> on from lldb-server), as well as the future debug-info-server, if it
> ever comes into existence.
>
> Discussion topic #1: The library name and scope.
> There are lost of other ways this could be organized. One of the names I
> considered was "BinaryFormat" for symmetry with llvm, but then I chose
> to drop the "Binary" part as it seemed to me we have plenty of
> non-binary formats as well. As for it's dependencies I currently have it
> depending on Utility and nothing else (as far as lldb libraries go). I
> can imagine using some Host code might be useful there too, but I would
> like to avoid any other lldb dependencies right now. Another question is
> whether this should be a single library or a bunch of smaller ones. I
> chose a single library now because the things I initially plan to put
> there are fairly small (/proc/pid/maps parser is 200 LOC), but I can see
> how we may want to create sub-libraries for things that grow big (the
> debug-info server code might turn out to be one of those) or that have
> some additional dependencies.
>

I don't have strong opinions here, nor do I have a better suggestion for
the name.


> Discussion topic #2: tool name and scope
> A case could be made to integrate this functionality into the llvm
> yaml2obj utilities. Here I chose not to do that because the minidump
> format is not at all implemented in llvm, and I do not see a use case
> for it to be implemented/moved there. A stronger case could be made to
> put the elf core code there, since llvm already supports reading elf
> files. While originally being in favour of that, I eventually adopted
> the view that doing this in lldb would be better because:
> - it would bring more symmetry with minidumps
> - it would enable us to do fine-grained yamlization for things that we
> care about (e.g., registers), which is something that would probably be
> uninteresting to the rest of llvm.
>

I don't know much about the minidump format or code, but it sounds
reasonable for me to have support for it in yaml2obj, which would be a
sufficient motivation to have the code live there. As you mention in your
footnote, MachO core files are already supported, and it sounds like ELF
could reuse a bunch of existing code as well. So having everything in LLVM
would give you even more symmetry. I also doubt anyone would mind having
more fine grained yamlization, even if you cannot use it to reduce a test
it's nicer to see structure than a binary blob (imho). Anyway, that's just
my take, I guess this is more of a question for the LLVM mailing list.


> Discussion topic #3: Use of .def files in l

Re: [lldb-dev] new tool (core2yaml) + a new top-level library (Formats)

2019-03-05 Thread Zachary Turner via lldb-dev
On Tue, Mar 5, 2019 at 1:47 PM Jonas Devlieghere via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

>
> I don't know much about the minidump format or code, but it sounds
> reasonable for me to have support for it in yaml2obj, which would be a
> sufficient motivation to have the code live there. As you mention in your
> footnote, MachO core files are already supported, and it sounds like ELF
> could reuse a bunch of existing code as well. So having everything in LLVM
> would give you even more symmetry. I also doubt anyone would mind having
> more fine grained yamlization, even if you cannot use it to reduce a test
> it's nicer to see structure than a binary blob (imho). Anyway, that's just
> my take, I guess this is more of a question for the LLVM mailing list.
>

A lot of obj2yaml output is just "Section Name" / "Section Contents" and
then a long hex string consisting of the contents.  Since a core file is an
ELF file, this would already be supported for obj2yaml today (in theory),
but I also agree that specific knowledge of breaking it down into finer
grained fields and subfields, and actually parsing the core, is probably
not useful for anything else in LLVM.



>
>
>> Discussion topic #3: Use of .def files in lldb. In one of the patches a
>> create a .def textual header to be used for avoiding repetitive code
>> when dealing various constants. This is fairly common practice in llvm,
>> but would be a first in lldb.
>>
>
> I think this is a good idea. Although not exactly the same, we already got
> our feet wet with a tablegen file in the driver.
>
+1
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev