[lldb-dev] Inquiry about SBFileSpec

[Ravitheja Addepally via lldb-dev]

Hello,
The SBFileSpec class of APIs can be used to read files on the local
file system right ? I want to know if it can be used for reading files on
the remote file system ? or do we need to use some other API ?

BR,
A Ravi Theja
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] UnicodeDecodeError for serialize SBValue description

[Tamas Berghammer via lldb-dev]

LLDB supports adding data formatters without modifying the source code and
I would strongly prefer to go that way as we don't want each user of LLDB
to start adding data formatters to their own custom types. We have a pretty
detailed (but possible a bit outdated) description about how they work and
how you can add a new one here: http://lldb.llvm.org/varformats.html

Enrico: Is there any reason you suggested the data formatters written
inside LLDB over the python based ones?

On Thu, Apr 7, 2016 at 3:31 AM Jeffrey Tan via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> Thanks Enrico. This is very detailed! I will take a look.
> Btw: originally, I was hoping that data formatter can be added without
> changing the source code. Like giving a xml/json format file telling lldb
> the memory layout/structure of the data structure, lldb can parse the
> xml/json and deduce the formatting. This is approach used by data
> visualizer in VS debugger:
> https://msdn.microsoft.com/en-us/library/jj620914.aspx
> This will make adding data formatter more extensible/flexible. Any reason
> we did not take this approach?
>
> Jeffrey
>
> On Wed, Apr 6, 2016 at 11:49 AM, Enrico Granata 
> wrote:
>
>>
>> On Apr 5, 2016, at 2:42 PM, Jeffrey Tan  wrote:
>>
>> Hi Enrico,
>>
>> Any suggestion/example how to add a data formatter for our own STL
>> string? From the output below I can see we are using our own "
>> *fbstring_core*" which I assume I need to write a type summary for this
>> type:
>>
>> frame variable corpus -T
>> (const string &const) corpus = error: summary string parsing error: {
>>   (std::*fbstring_core*) store_ = {
>> (std::*fbstring_core*::(anonymous union))  = {
>>   (char [24]) small_ = "www"
>>   (std::fbstring_core::MediumLarge) ml_ = {
>> (char *) data_ = 0x0077
>> "H\x89U\xa8H\x89M\xa0L\x89E\x98H\x8bE\xa8H\x89��_U��D\x88e�H\x8bE\xa0H\x89��]U��H\x89�H\x8dE�H\x89�H\x89���
>> ��L\x8dm�H\x8bE\x98H\x89��IU��\x88]�L\x8be\xb0L\x89��
>> (std::size_t) size_ = 0
>> (std::size_t) capacity_ = 1441151880758558720
>>   }
>> }
>>   }
>> }
>>
>>
>> Admittedly, this is going to be a little vague since I haven’t really
>> seen your code and I am only working off of one sample
>>
>> There’s going to be two parts to getting this to work:
>>
>> *Part 1 - Formatting fbstring_core*
>>
>> At a glance, an fbstring_core can be backed by two representations.
>> A “small” representation (a char array), and a “medium/large"
>> representation (a char* + a size)
>> I assume that the way you tell one from the other is
>>
>> if (size == 0) small
>> else medium-large
>>
>> If my assumption is not correct, you’ll need to discover what the correct
>> discriminator logic is - the class has to know, and so do you :-)
>>
>> Armed with that knowledge, look in lldb
>> source/Plugins/Language/CPlusPlus/Formatters/LibCxx.cpp
>> There’s a bunch of code that deals with formatting llvm’s libc++
>> std::string - which follows a very similar logic to your class
>>
>> ExtractLibcxxStringInfo() is the function that handles discovering which
>> layout the string uses - where the data lives - and how much data there is
>>
>> Once you have told yourself how much data there is (the size) and where
>> it lives (array or pointer), LibcxxStringSummaryProvider() has the easy
>> task - it sets up a StringPrinter, tells it how much data to print, where
>> to get it from, and then delegates the StringPrinter to do the grunt work
>> StringPrinter is a nifty little tool - it can handle generating summaries
>> for different kinds of strings (UTF8? UTF16? we got it - is a \0 a
>> terminator? what quote character would you like? …) - you point it at some
>> data, set up a few options, and it will generate a printable representation
>> for you - if your string type is doing anything out of the ordinary, let’s
>> talk - I am definitely open to extending StringPrinter to handle even more
>> magic
>>
>> *Part 2 - Teaching std::string that it can be backed by an fbstring_core*
>>
>> At the end of part 1, you’ll probably end up with a
>> FBStringCoreSummaryProvider() - now you need to teach LLDB about it
>> The obvious thing you could do would be to go in CPlusPlusLanguage
>> ::GetFormatters() add a LoadFBStringFormatter(g_category) to it - and
>> then imitate - say - LoadLibCxxFormatters()
>>
>> AddCXXSummary(cpp_category_sp, lldb_private::formatters::
>> FBStringCoreSummaryProvider, “fbstringcore summary provider", ConstString
>> (“std::fbstring_core<.+>"), stl_summary_flags, true);
>>
>> That will work - but what you would see is:
>>
>> (const string &const) corpus = error: summary string parsing error: {
>>   (std::*fbstring_core*) store_ = “www"
>>
>>
>> You wanna do
>>
>> (lldb) log enable lldb formatters
>> (lldb) frame variable -T corpus
>>
>> It will list one or more typenames - the most specific one is the one you
>> like (e.g. for libc++ we get std::__1::string - this is how we tell
>> ourselves this is the std::s

Re: [lldb-dev] UnicodeDecodeError for serialize SBValue description

[Jim Ingham via lldb-dev]

I don't think Enrico was suggesting that we maintain a bunch of third party 
data formatters in the lldb source base.  He was giving C++ examples (using the 
lldb_private API's) because the STL formatters are in C++, so that's what he 
had on hand to demonstrate the kinds of algorithms you would use to dig into 
these complex structures.  For the most part the lldb_private API's used in 
Enrico's examples are mirrored in the SB API's pretty directly, so this isn't a 
terrible source for examples.

Note, it used to be possible to write C++ based data formatters, build them in 
a shared library and load them with the "plugin load" command.  These have the 
advantage of working on systems that don't support Python.  Not sure what the 
state of that is these days, however.  But even if you were going to write C++ 
formatters you'd be better off using the SB API's not the lldb_private API's 
since then your plugins would have a longer useful life-cycle.

Jim


> On Apr 7, 2016, at 2:45 AM, Tamas Berghammer via lldb-dev 
>  wrote:
> 
> LLDB supports adding data formatters without modifying the source code and I 
> would strongly prefer to go that way as we don't want each user of LLDB to 
> start adding data formatters to their own custom types. We have a pretty 
> detailed (but possible a bit outdated) description about how they work and 
> how you can add a new one here: http://lldb.llvm.org/varformats.html
> 
> Enrico: Is there any reason you suggested the data formatters written inside 
> LLDB over the python based ones?
> 
> On Thu, Apr 7, 2016 at 3:31 AM Jeffrey Tan via lldb-dev 
>  wrote:
> Thanks Enrico. This is very detailed! I will take a look. 
> Btw: originally, I was hoping that data formatter can be added without 
> changing the source code. Like giving a xml/json format file telling lldb the 
> memory layout/structure of the data structure, lldb can parse the xml/json 
> and deduce the formatting. This is approach used by data visualizer in VS 
> debugger: https://msdn.microsoft.com/en-us/library/jj620914.aspx
> This will make adding data formatter more extensible/flexible. Any reason we 
> did not take this approach? 
> 
> Jeffrey
> 
> On Wed, Apr 6, 2016 at 11:49 AM, Enrico Granata  wrote:
> 
>> On Apr 5, 2016, at 2:42 PM, Jeffrey Tan  wrote:
>> 
>> Hi Enrico,
>> 
>> Any suggestion/example how to add a data formatter for our own STL string? 
>> From the output below I can see we are using our own "fbstring_core" which I 
>> assume I need to write a type summary for this type:
>> 
>> frame variable corpus -T
>> (const string &const) corpus = error: summary string parsing error: {
>>   (std::fbstring_core) store_ = {
>> (std::fbstring_core::(anonymous union))  = {
>>   (char [24]) small_ = "www"
>>   (std::fbstring_core::MediumLarge) ml_ = {
>> (char *) data_ = 0x0077 
>> "H\x89U\xa8H\x89M\xa0L\x89E\x98H\x8bE\xa8H\x89��_U��D\x88e�H\x8bE\xa0H\x89��]U��H\x89�H\x8dE�H\x89�H\x89���
>>  ��L\x8dm�H\x8bE\x98H\x89��IU��\x88]�L\x8be\xb0L\x89��
>> (std::size_t) size_ = 0
>> (std::size_t) capacity_ = 1441151880758558720
>>   }
>> }
>>   }
>> }
>> 
> 
> Admittedly, this is going to be a little vague since I haven’t really seen 
> your code and I am only working off of one sample
> 
> There’s going to be two parts to getting this to work:
> 
> Part 1 - Formatting fbstring_core
> 
> At a glance, an fbstring_core can be backed by two representations. A 
> “small” representation (a char array), and a “medium/large" representation (a 
> char* + a size)
> I assume that the way you tell one from the other is
> 
> if (size == 0) small
> else medium-large
> 
> If my assumption is not correct, you’ll need to discover what the correct 
> discriminator logic is - the class has to know, and so do you :-)
> 
> Armed with that knowledge, look in lldb 
> source/Plugins/Language/CPlusPlus/Formatters/LibCxx.cpp
> There’s a bunch of code that deals with formatting llvm’s libc++ std::string 
> - which follows a very similar logic to your class
> 
> ExtractLibcxxStringInfo() is the function that handles discovering which 
> layout the string uses - where the data lives - and how much data there is
> 
> Once you have told yourself how much data there is (the size) and where it 
> lives (array or pointer), LibcxxStringSummaryProvider() has the easy task - 
> it sets up a StringPrinter, tells it how much data to print, where to get it 
> from, and then delegates the StringPrinter to do the grunt work
> StringPrinter is a nifty little tool - it can handle generating summaries for 
> different kinds of strings (UTF8? UTF16? we got it - is a \0 a terminator? 
> what quote character would you like? …) - you point it at some data, set up a 
> few options, and it will generate a printable representation for you - if 
> your string type is doing anything out of the ordinary, let’s talk - I am 
> definitely open to extending StringPrinter to handle even more magic
> 
> Part 2 - Teachin

Re: [lldb-dev] Inquiry about SBFileSpec

[Greg Clayton via lldb-dev]

The SBPlatform that you can get from your SBTarget is the way to do this 
correctly:

SBPlatform platform = target.GetPlatform();

SBFileSpec src("/some/remote/path/to/remote/file/file.txt", false);
SBFileSpec dst("/tmp/file.txt");
SBError error = platform.Get (src, dst);

Then you can plat with "dst" locally on your machine. This of course means that 
you have to be connected to the remote platform. The "lldb-server" we build can 
be used as a platform that can be started and allows the local LLDB to attach 
to the remote platform. 

So on the remote machine you would start your "lldb-server" in platform mode:

remote.foo.com% lldb-server platform other.foo.com:1234 [options]


Then you can attach your platform to this:

% lldb
(lldb) platform select remote-linux
(lldb) platform connect remote.foo.com:1234

now you are connected and can run things remotely and get files from the 
platform:

(lldb) platform settings --working-dir "/tmp"
(lldb) file a.out
(lldb) run

Since we are connected to a platform, the main executable of any target will be 
sent over to the remote machine in the platform's working directory and then 
run from there.

And if you are connected, you can get the platform from the current target and 
transfer files.



> On Apr 7, 2016, at 2:34 AM, Ravitheja Addepally via lldb-dev 
>  wrote:
> 
> Hello,
> The SBFileSpec class of APIs can be used to read files on the local 
> file system right ? I want to know if it can be used for reading files on the 
> remote file system ? or do we need to use some other API ?
> 
> BR,
> A Ravi Theja
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] UnicodeDecodeError for serialize SBValue description

[Enrico Granata via lldb-dev]

> On Apr 7, 2016, at 9:51 AM, Jim Ingham  wrote:
> 
> I don't think Enrico was suggesting that we maintain a bunch of third party 
> data formatters in the lldb source base.  

That depends - if this std::string implementation is part of a publicly 
available STL implementation, it might make sense for us to “know about it” out 
of the box in the same way we know about libstdc++ and libc++
If it is an internal-only string class, then, yes, I would definitely not 
suggest putting this inside the LLDB core

> He was giving C++ examples (using the lldb_private API's) because the STL 
> formatters are in C++, so that's what he had on hand to demonstrate the kinds 
> of algorithms you would use to dig into these complex structures.  For the 
> most part the lldb_private API's used in Enrico's examples are mirrored in 
> the SB API's pretty directly, so this isn't a terrible source for examples.
> 
> Note, it used to be possible to write C++ based data formatters, build them 
> in a shared library and load them with the "plugin load" command.  These have 
> the advantage of working on systems that don't support Python.  Not sure what 
> the state of that is these days, however.

It might or might not work. If it didn’t work and somebody wanted to fix that, 
I suspect we would gladly accept their patches.

>  But even if you were going to write C++ formatters you'd be better off using 
> the SB API's not the lldb_private API's since then your plugins would have a 
> longer useful life-cycle.
> 
> Jim
> 
> 
>> On Apr 7, 2016, at 2:45 AM, Tamas Berghammer via lldb-dev 
>>  wrote:
>> 
>> LLDB supports adding data formatters without modifying the source code and I 
>> would strongly prefer to go that way as we don't want each user of LLDB to 
>> start adding data formatters to their own custom types. We have a pretty 
>> detailed (but possible a bit outdated) description about how they work and 
>> how you can add a new one here: http://lldb.llvm.org/varformats.html
>> 
>> Enrico: Is there any reason you suggested the data formatters written inside 
>> LLDB over the python based ones?
>> 
>> On Thu, Apr 7, 2016 at 3:31 AM Jeffrey Tan via lldb-dev 
>>  wrote:
>> Thanks Enrico. This is very detailed! I will take a look. 
>> Btw: originally, I was hoping that data formatter can be added without 
>> changing the source code. Like giving a xml/json format file telling lldb 
>> the memory layout/structure of the data structure, lldb can parse the 
>> xml/json and deduce the formatting. This is approach used by data visualizer 
>> in VS debugger: https://msdn.microsoft.com/en-us/library/jj620914.aspx
>> This will make adding data formatter more extensible/flexible. Any reason we 
>> did not take this approach? 
>> 
>> Jeffrey
>> 
>> On Wed, Apr 6, 2016 at 11:49 AM, Enrico Granata  wrote:
>> 
>>> On Apr 5, 2016, at 2:42 PM, Jeffrey Tan  wrote:
>>> 
>>> Hi Enrico,
>>> 
>>> Any suggestion/example how to add a data formatter for our own STL string? 
>>> From the output below I can see we are using our own "fbstring_core" which 
>>> I assume I need to write a type summary for this type:
>>> 
>>> frame variable corpus -T
>>> (const string &const) corpus = error: summary string parsing error: {
>>>  (std::fbstring_core) store_ = {
>>>(std::fbstring_core::(anonymous union))  = {
>>>  (char [24]) small_ = "www"
>>>  (std::fbstring_core::MediumLarge) ml_ = {
>>>(char *) data_ = 0x0077 
>>> "H\x89U\xa8H\x89M\xa0L\x89E\x98H\x8bE\xa8H\x89��_U��D\x88e�H\x8bE\xa0H\x89��]U��H\x89�H\x8dE�H\x89�H\x89���
>>>  ��L\x8dm�H\x8bE\x98H\x89��IU��\x88]�L\x8be\xb0L\x89��
>>>(std::size_t) size_ = 0
>>>(std::size_t) capacity_ = 1441151880758558720
>>>  }
>>>}
>>>  }
>>> }
>>> 
>> 
>> Admittedly, this is going to be a little vague since I haven’t really seen 
>> your code and I am only working off of one sample
>> 
>> There’s going to be two parts to getting this to work:
>> 
>> Part 1 - Formatting fbstring_core
>> 
>> At a glance, an fbstring_core can be backed by two representations. A 
>> “small” representation (a char array), and a “medium/large" representation 
>> (a char* + a size)
>> I assume that the way you tell one from the other is
>> 
>> if (size == 0) small
>> else medium-large
>> 
>> If my assumption is not correct, you’ll need to discover what the correct 
>> discriminator logic is - the class has to know, and so do you :-)
>> 
>> Armed with that knowledge, look in lldb 
>> source/Plugins/Language/CPlusPlus/Formatters/LibCxx.cpp
>> There’s a bunch of code that deals with formatting llvm’s libc++ std::string 
>> - which follows a very similar logic to your class
>> 
>> ExtractLibcxxStringInfo() is the function that handles discovering which 
>> layout the string uses - where the data lives - and how much data there is
>> 
>> Once you have told yourself how much data there is (the size) and where it 
>> lives (array or pointer), LibcxxStringSummaryProvider() has the easy task - 
>> it 

Re: [lldb-dev] UnicodeDecodeError for serialize SBValue description

[Enrico Granata via lldb-dev]

> On Apr 6, 2016, at 7:31 PM, Jeffrey Tan  wrote:
> 
> Thanks Enrico. This is very detailed! I will take a look. 
> Btw: originally, I was hoping that data formatter can be added without 
> changing the source code. Like giving a xml/json format file telling lldb the 
> memory layout/structure of the data structure, lldb can parse the xml/json 
> and deduce the formatting. This is approach used by data visualizer in VS 
> debugger: https://msdn.microsoft.com/en-us/library/jj620914.aspx 
> 
> This will make adding data formatter more extensible/flexible. Any reason we 
> did not take this approach? 
> 

The way I understand the Natvis system, it allows one to provide a bunch of 
expressions that describe how the debugger would go about retrieving the 
interesting data bits
This has the bonus of being really easy, since you’re writing code in the same 
language/context of the types you’re formatting
On the other hand it has a few drawbacks, in terms of performance as well as 
safety (imagine trying to run code on an object when said object is in an 
incoherent state)
The LLDB approach, on the other hand, is that you should try to not run code 
when providing these data formatters. In order to do that, we vend an API that 
can do things such as retrieve child values, read memory, cast values, …, all 
without code execution
Once you have this kind of API that is not expressed in your source language, 
you might just as well describe it in a scripting language. Hence were born the 
Python data formatters.
In order for us to gain even more performance for native system types that we 
know we’re gonna run into all the time, we then switched a bunch of the 
“mission critical” formatters from Python to C++
The Python extension points are still available, as Jim pointed out, and you 
are more than welcome to use those instead of modifying the debugger core

> Jeffrey
> 
> On Wed, Apr 6, 2016 at 11:49 AM, Enrico Granata  > wrote:
> 
>> On Apr 5, 2016, at 2:42 PM, Jeffrey Tan > > wrote:
>> 
>> Hi Enrico,
>> 
>> Any suggestion/example how to add a data formatter for our own STL string? 
>> From the output below I can see we are using our own "fbstring_core" which I 
>> assume I need to write a type summary for this type:
>> 
>> frame variable corpus -T
>> (const string &const) corpus = error: summary string parsing error: {
>>   (std::fbstring_core) store_ = {
>> (std::fbstring_core::(anonymous union))  = {
>>   (char [24]) small_ = "www"
>>   (std::fbstring_core::MediumLarge) ml_ = {
>> (char *) data_ = 0x0077 
>> "H\x89U\xa8H\x89M\xa0L\x89E\x98H\x8bE\xa8H\x89��_U��D\x88e�H\x8bE\xa0H\x89��]U��H\x89�H\x8dE�H\x89�H\x89���
>>  ��L\x8dm�H\x8bE\x98H\x89��IU��\x88]�L\x8be\xb0L\x89��
>> (std::size_t) size_ = 0
>> (std::size_t) capacity_ = 1441151880758558720
>>   }
>> }
>>   }
>> }
>> 
> 
> Admittedly, this is going to be a little vague since I haven’t really seen 
> your code and I am only working off of one sample
> 
> There’s going to be two parts to getting this to work:
> 
> Part 1 - Formatting fbstring_core
> 
> At a glance, an fbstring_core can be backed by two representations. A 
> “small” representation (a char array), and a “medium/large" representation (a 
> char* + a size)
> I assume that the way you tell one from the other is
> 
> if (size == 0) small
> else medium-large
> 
> If my assumption is not correct, you’ll need to discover what the correct 
> discriminator logic is - the class has to know, and so do you :-)
> 
> Armed with that knowledge, look in lldb 
> source/Plugins/Language/CPlusPlus/Formatters/LibCxx.cpp
> There’s a bunch of code that deals with formatting llvm’s libc++ std::string 
> - which follows a very similar logic to your class
> 
> ExtractLibcxxStringInfo() is the function that handles discovering which 
> layout the string uses - where the data lives - and how much data there is
> 
> Once you have told yourself how much data there is (the size) and where it 
> lives (array or pointer), LibcxxStringSummaryProvider() has the easy task - 
> it sets up a StringPrinter, tells it how much data to print, where to get it 
> from, and then delegates the StringPrinter to do the grunt work
> StringPrinter is a nifty little tool - it can handle generating summaries for 
> different kinds of strings (UTF8? UTF16? we got it - is a \0 a terminator? 
> what quote character would you like? …) - you point it at some data, set up a 
> few options, and it will generate a printable representation for you - if 
> your string type is doing anything out of the ordinary, let’s talk - I am 
> definitely open to extending StringPrinter to handle even more magic
> 
> Part 2 - Teaching std::string that it can be backed by an fbstring_core
> 
> At the end of part 1, you’ll probably end up with a 
> FBStringCoreSummaryProvider() - now you need to teac