LLDB supports adding data formatters without modifying the source code and I would strongly prefer to go that way as we don't want each user of LLDB to start adding data formatters to their own custom types. We have a pretty detailed (but possible a bit outdated) description about how they work and how you can add a new one here: http://lldb.llvm.org/varformats.html
Enrico: Is there any reason you suggested the data formatters written inside LLDB over the python based ones? On Thu, Apr 7, 2016 at 3:31 AM Jeffrey Tan via lldb-dev < lldb-dev@lists.llvm.org> wrote: > Thanks Enrico. This is very detailed! I will take a look. > Btw: originally, I was hoping that data formatter can be added without > changing the source code. Like giving a xml/json format file telling lldb > the memory layout/structure of the data structure, lldb can parse the > xml/json and deduce the formatting. This is approach used by data > visualizer in VS debugger: > https://msdn.microsoft.com/en-us/library/jj620914.aspx > This will make adding data formatter more extensible/flexible. Any reason > we did not take this approach? > > Jeffrey > > On Wed, Apr 6, 2016 at 11:49 AM, Enrico Granata <egran...@apple.com> > wrote: > >> >> On Apr 5, 2016, at 2:42 PM, Jeffrey Tan <jeffrey.fu...@gmail.com> wrote: >> >> Hi Enrico, >> >> Any suggestion/example how to add a data formatter for our own STL >> string? From the output below I can see we are using our own " >> *fbstring_core*" which I assume I need to write a type summary for this >> type: >> >> frame variable corpus -T >> (const string &const) corpus = error: summary string parsing error: { >> (std::*fbstring_core*<char>) store_ = { >> (std::*fbstring_core*<char>::(anonymous union)) = { >> (char [24]) small_ = "www" >> (std::fbstring_core<char>::MediumLarge) ml_ = { >> (char *) data_ = 0x0000000000777777 >> "H\x89U\xa8H\x89M\xa0L\x89E\x98H\x8bE\xa8H\x89��_U��D\x88e�H\x8bE\xa0H\x89��]U��H\x89�H\x8dE�H\x89�H\x89��� >> ��L\x8dm�H\x8bE\x98H\x89��IU��\x88]�L\x8be\xb0L\x89�� >> (std::size_t) size_ = 0 >> (std::size_t) capacity_ = 1441151880758558720 >> } >> } >> } >> } >> >> >> Admittedly, this is going to be a little vague since I haven’t really >> seen your code and I am only working off of one sample >> >> There’s going to be two parts to getting this to work: >> >> *Part 1 - Formatting fbstring_core* >> >> At a glance, an fbstring_core<char> can be backed by two representations. >> A “small” representation (a char array), and a “medium/large" >> representation (a char* + a size) >> I assume that the way you tell one from the other is >> >> if (size == 0) small >> else medium-large >> >> If my assumption is not correct, you’ll need to discover what the correct >> discriminator logic is - the class has to know, and so do you :-) >> >> Armed with that knowledge, look in lldb >> source/Plugins/Language/CPlusPlus/Formatters/LibCxx.cpp >> There’s a bunch of code that deals with formatting llvm’s libc++ >> std::string - which follows a very similar logic to your class >> >> ExtractLibcxxStringInfo() is the function that handles discovering which >> layout the string uses - where the data lives - and how much data there is >> >> Once you have told yourself how much data there is (the size) and where >> it lives (array or pointer), LibcxxStringSummaryProvider() has the easy >> task - it sets up a StringPrinter, tells it how much data to print, where >> to get it from, and then delegates the StringPrinter to do the grunt work >> StringPrinter is a nifty little tool - it can handle generating summaries >> for different kinds of strings (UTF8? UTF16? we got it - is a \0 a >> terminator? what quote character would you like? …) - you point it at some >> data, set up a few options, and it will generate a printable representation >> for you - if your string type is doing anything out of the ordinary, let’s >> talk - I am definitely open to extending StringPrinter to handle even more >> magic >> >> *Part 2 - Teaching std::string that it can be backed by an fbstring_core* >> >> At the end of part 1, you’ll probably end up with a >> FBStringCoreSummaryProvider() - now you need to teach LLDB about it >> The obvious thing you could do would be to go in CPlusPlusLanguage >> ::GetFormatters() add a LoadFBStringFormatter(g_category) to it - and >> then imitate - say - LoadLibCxxFormatters() >> >> AddCXXSummary(cpp_category_sp, lldb_private::formatters:: >> FBStringCoreSummaryProvider, “fbstringcore summary provider", ConstString >> (“std::fbstring_core<.+>"), stl_summary_flags, true); >> >> That will work - but what you would see is: >> >> (const string &const) corpus = error: summary string parsing error: { >> (std::*fbstring_core*<char>) store_ = “www" >> >> >> You wanna do >> >> (lldb) log enable lldb formatters >> (lldb) frame variable -T corpus >> >> It will list one or more typenames - the most specific one is the one you >> like (e.g. for libc++ we get std::__1::string - this is how we tell >> ourselves this is the std::string from libc++) >> Once you find that typename, you’ll make a new formatter - >> FBStringSummaryProvider() - and register that formatter with that very >> specific typename >> >> All that FBStringSummaryProvider() has to do is get the “store_” member >> (ValueObject::GetChildMemberWithName() is your friend) - and pass it down >> to FBStringCoreSummaryProvider() >> >> >> I understand this may seem a little convoluted and arcane at first - but >> feel free to ask more questions, and I’ll try to help out! >> >> Thanks. >> Jeffrey >> >> On Mon, Mar 28, 2016 at 11:38 AM, Enrico Granata <egran...@apple.com> >> wrote: >> >>> This is kind of orthogonal to your problem, but the reason why you are >>> not seeing the kind of simplified printing Greg is suggesting, is because >>> your std::string doesn’t look like any of the kinds we recognize >>> >>> Specifically, LLDB data formatters work by matching against type names, >>> and once they recognize a typename, then they try to inspect the variable >>> in order to grab a summary >>> In your example, your std::string exposes a layout that we are not >>> handling - hence we bail out of the formatter and we fall back to the raw >>> view >>> >>> If you want pretty printing to work, you’ll need to write a data >>> formatter >>> >>> There are a few avenues. The obvious easy one is to extend the existing >>> std::string formatter to recognize your type’s internal layout. >>> If one were signing up for more infrastructure work, they could decide >>> to try and detect shared library loads and load formatters that match with >>> whatever libraries are being loaded. >>> >>> On Mar 28, 2016, at 9:47 AM, Greg Clayton via lldb-dev < >>> lldb-dev@lists.llvm.org> wrote: >>> >>> So you need to be prepared to escape any text that can have special >>> characters. A "std::string" or any container can contain special >>> characters. If you are encoding stuff into JSON, you will either need to >>> escape any special characters, or hex encode the string into ASCII hex >>> bytes. >>> >>> In debuggers we often get bogus data because variables are not >>> initialized, but the compiler tells us that a variable is valid in address >>> range [0x1000-0x2000), but it actually is [0x1200-0x2000). If we read a >>> variable in this case, a std::string might contain bogus data and the bytes >>> might not make sense. So you always have to be prepared for bad data. >>> >>> If we look at: >>> >>> store_ = { >>> = { >>> small_ = "www" >>> ml_ = (data_ = >>> >>> "��UH\x89�H�}�H\x8bE�]ÐUH\x89�H��H\x89}�H\x8bE�H\x89��~\xb4��\x90��UH\x89�SH\x83�H\x89}�H�u�H�E�H���\x9e���H\x8b\x18H\x8bE�H���O\xb4��H\x89ƿ\b", >>> size_ = 0, capacity_ = 1441151880758558720) >>> } >>> } >>> } >>> >>> We can see the "size_" is zero, and capacity_ is 1441151880758558720 >>> (which is 0x1400000000000000). "data_" seems to be some random pointer. >>> >>> On MacOSX, we have a special formatting code that displays std::string >>> in CPlusPlusLanguage.cpp that gets installed in the LoadLibCxxFormatters() >>> or LoadLibStdcppFormatters() functions with code like: >>> >>> lldb::TypeSummaryImplSP std_string_summary_sp(new >>> CXXFunctionSummaryFormat(stl_summary_flags, >>> lldb_private::formatters::LibcxxStringSummaryProvider, "std::string summary >>> provider")); >>> >>> cpp_category_sp->GetTypeSummariesContainer()->Add(ConstString("std::__1::string"), >>> std_string_summary_sp); >>> >>> Special flags are set on std::string to say "don't show children of this >>> and just show a summary" So if a std::string contained "hello". So for the >>> following code: >>> >>> std::string h ("hello"); >>> >>> You should just see: >>> >>> (lldb) fr var h >>> (std::__1::string) h = "hello" >>> >>> If you take a look at the normal value in the raw we see: >>> >>> (lldb) fr var --raw h >>> (std::__1::string) h = { >>> __r_ = { >>> std::__1::__libcpp_compressed_pair_imp<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> >::__rep, >>> std::__1::allocator<char>, 2> = { >>> __first_ = { >>> = { >>> __l = { >>> __cap_ = 122511465736202 >>> __size_ = 0 >>> __data_ = 0x0000000000000000 >>> } >>> __s = { >>> = { >>> __size_ = '\n' >>> __lx = '\n' >>> } >>> __data_ = { >>> [0] = 'h' >>> [1] = 'e' >>> [2] = 'l' >>> [3] = 'l' >>> [4] = 'o' >>> [5] = '\0' >>> [6] = '\0' >>> [7] = '\0' >>> [8] = '\0' >>> [9] = '\0' >>> [10] = '\0' >>> [11] = '\0' >>> [12] = '\0' >>> [13] = '\0' >>> [14] = '\0' >>> [15] = '\0' >>> [16] = '\0' >>> [17] = '\0' >>> [18] = '\0' >>> [19] = '\0' >>> [20] = '\0' >>> [21] = '\0' >>> [22] = '\0' >>> } >>> } >>> __r = { >>> __words = { >>> [0] = 122511465736202 >>> [1] = 0 >>> [2] = 0 >>> } >>> } >>> } >>> } >>> } >>> } >>> } >>> >>> So the main question is why are our "std::string" formatters not kicking >>> in for you. That comes down to a typename match, or the format of the >>> string isn't what the formatter is expecting. >>> >>> But again, since you std::string can contain anything, you will need to >>> escape any and all text that is encoded into JSON to ensure it doesn't >>> contain anything JSON can't deal with. >>> >>> On Mar 27, 2016, at 9:20 PM, Jeffrey Tan via lldb-dev < >>> lldb-dev@lists.llvm.org> wrote: >>> >>> Thanks Siva. All the DW_TAG_member related errors seems to go away after >>> patching with your fix. The current problem is handling the decoding. >>> >>> Here is the correct decoding from gdb whic might be useful: >>> (gdb) p corpus >>> $3 = (const std::string &) @0x7fd133cfb888: { >>> static npos = 18446744073709551615, store_ = { >>> static kIsLittleEndian = <optimized out>, >>> static kIsBigEndian = <optimized out>, { >>> small_ = "www", '\000' <repeats 20 times>, "\024", ml_ = { >>> data_ = 0x777777 <std::_Any_data::_M_access<void >>> folly::fibers::Baton::waitFiber<folly::fibers::FirstArgOf<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1}, >>> void>::type::value_type >>> folly::fibers::await<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1}>(folly::fibers::FirstArgOf&&)::{lambda()#1}>(folly::fibers::FiberManager&, >>> folly::fibers::FirstArgOf<folly::fibers::FirstArgOf<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1}, >>> void>::type::value_type >>> folly::fibers::await<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBase<facebook::servicerouter::ThriftDispatcher>::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1}>(folly::fibers::FirstArgOf&&)::{lambda()#1}, >>> void>::type::value_type)::{lambda(folly::fibers::Fiber&)#1}*>() const+25> >>> "\311\303UH\211\345H\211}\370H\213E\370]ÐUH\211\345H\203\354\020H\211}\370H\213E\370H\211\307\350~\264\312\377\220\311\303UH\211\345SH\203\354\030H\211}\350H\211u\340H\213E\340H\211\307\350\236\377\377\377H\213\030H\213E\350H\211\307\350O\264\312\377H\211ƿ\b", >>> size_ = 0, >>> capacity_ = 1441151880758558720}}}} >>> >>> Utf-16 does not seem to decode it, while 'latin-1' does: >>> >>> '\xc9'.decode('utf-16') >>> >>> Traceback (most recent call last): >>> File "<stdin>", line 1, in <module> >>> File >>> "/mnt/gvfs/third-party2/python/55c1fd79d91c77c95932db31a4769919611c12bb/2.7.8/centos6-native/da39a3e/lib/python2.7/encodings/utf_16.py", >>> line 16, in decode >>> return codecs.utf_16_decode(input, errors, True) >>> UnicodeDecodeError: 'utf16' codec can't decode byte 0xc9 in position 0: >>> truncated data >>> >>> '\xc9'.decode('latin-1') >>> >>> u'\xc9' >>> >>> Instead of guessing what kind of decoding I should use, I would use >>> 'ensure_ascii=False' to prevent the crash for now. >>> >>> I tried to reproduce this crash, but it seems that the crash might be >>> related with some internal stl implementation we are using. I will see if I >>> can narrow down to a small repro later. >>> >>> Thanks >>> Jeffrey >>> >>> On Sun, Mar 27, 2016 at 2:49 PM, Siva Chandra <sivachan...@gmail.com> >>> wrote: >>> On Sat, Mar 26, 2016 at 11:58 PM, Jeffrey Tan <jeffrey.fu...@gmail.com> >>> wrote: >>> >>> Btw: after patching with Siva's fix http://reviews.llvm.org/D18008, the >>> first field 'small_' is fixed, however the second field 'ml_' still emits >>> garbage: >>> >>> (lldb) fr v corpus >>> (const string &const) corpus = error: summary string parsing error: { >>> store_ = { >>> = { >>> small_ = "www" >>> ml_ = (data_ = >>> >>> "��UH\x89�H�}�H\x8bE�]ÐUH\x89�H��H\x89}�H\x8bE�H\x89��~\xb4��\x90��UH\x89�SH\x83�H\x89}�H�u�H�E�H���\x9e���H\x8b\x18H\x8bE�H���O\xb4��H\x89ƿ\b", >>> size_ = 0, capacity_ = 1441151880758558720) >>> } >>> } >>> } >>> >>> >>> Do you still see the DW_TAG_member related error? >>> >>> A wild (and really wild at that) guess: Is it utf16 data that is being >>> decoded as utf8? >>> >>> As David Blaikie mentioned on the other thread, it would really help >>> if you provide us with a minimal example to repro this. Atleast, repro >>> instructions. >>> >>> _______________________________________________ >>> lldb-dev mailing list >>> lldb-dev@lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >>> >>> >>> _______________________________________________ >>> lldb-dev mailing list >>> lldb-dev@lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >>> >>> >>> >>> Thanks, >>> *- Enrico* >>> 📩 egranata@.com ☎️ 27683 >>> >>> >> >> >> Thanks, >> *- Enrico* >> 📩 egranata@.com ☎️ 27683 >> >> > _______________________________________________ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >
_______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
