Re: [lldb-dev] Listing memory regions in lldb

[Howard Hellyer via lldb-dev]

I have experimented with the GetMemoryRegionInfo approach on the internal 
APIs already, it has some positives and negatives.

- GetMemoryRegionInfo is unimplemented for Linux and Mac core dumps. 
That's not necessarily a bad thing as it could be implemented the "right" 
way. (As Jim said GetMemoryRegionInfo would have to return the right thing 
for an unmapped region.) Interestingly when I've worked on Windows core 
dumps before I've seen that MiniDump, with the right flags, will 
deliberately insert a region in the middle of the memory ranges to 
represent the unmapped space, on 64 bit it's quite a large section.

- Using GetMemoryRegionInfo to iterate might be quite expensive if there 
are many small memory regions.

- One reason I hadn't looked at just exposing an SBGetMemoryRegionInfo is 
that it wouldn't match a lot of the SB API's at the moment (for example 
for Threads and Sections) which tend work by having GetNumXs() and 
GetXAtIndex() calls. Iterating with SBGetMemoryRegionInfo would be 
non-obvious to the user of the API since it doesn't match the convention. 
It would need to be documented in the SBGetMemoryRegionInfo API.

- I've found the only way to reliably filter out inaccessible memory is to 
do a test read and check the error return code. I'm pretty sure I've seen 
some that are readable via a memory read but marked read=false, 
write=false, execute=false. (I can't remember the exact case now off the 
top of my head, but I think it might have been allocated but uncommitted 
memory or similar.)

- Using GetMemoryRegionInfo over a remote connection on Linux and Mac 
worked well but seemed to coalesce some of the memory regions together. It 
also only allows for read/write/exec attributes to be passed. That's a 
shame as a live process can often tell you more about what the region is 
for. The remote command memory map looks like it sends back XML so it 
might be possible to insert custom properties in there to give more 
information but I'm not sure how safe it is to do that, I don't know the 
project quite well enough to understand all the use cases for the 
protocol.

- Extended infomation /proc/pid/maps marking a region as [stack] would be 
lost. All you would get is read/write/exec info. (That said supporting 
everything every platform can provide might be a bit much.)

- LLDB's ReadMemory implementations seem to return 0's for missing memory 
that should be accessible. It might be nice to include whether the memory 
is backed by real data or not in the API. (For example Linux core files 
can be missing huge pages depending on the setting of 
/proc/PID/coredump_filter or files can simply be truncated.)

I could implement the GetMemoryRegionInfo iteration mechanism pretty 
quickly and it would actually fit my purposes as far as providing all the 
addresses you can sensibly access. 

I'm quite keen to provide a patch but don't want to provide one that is at 
odds with how the rest of lldb works or provides data that's only useful 
to me so I'm quite keen to get a bit of feedback on what the preferred 
approach would be. It could be that providing both SBGetMemoryRegionInfo 
and the SBGetNumMemoryRegions/SBGetMemoryRegionAtIndex pair is the right 
solution.

Would a patch also need to provide a command to dump this information as 
it can be awkward to have data that's only accessible via the API?


Howard Hellyer
IBM Runtime Technologies, IBM Systems




Greg Clayton  wrote on 12/05/2016 19:09:49:

> From: Greg Clayton 
> To: Howard Hellyer/UK/IBM@IBMGB
> Cc: lldb-dev@lists.llvm.org
> Date: 12/05/2016 19:10
> Subject: Re: [lldb-dev] Listing memory regions in lldb
> 
> We have a way internally with:
> 
> virtual Error
> lldb_private::Process::GetMemoryRegionInfo (lldb::addr_t 
> load_addr, MemoryRegionInfo &range_info);
> 
> This isn't expose via the public API in lldb::SBProcess. If you 
> want, you can expose this. We would need to expose a 
> SBMemoryRegionInfo and the call could be:
> 
> namespace lldb
> {
> class SBProcess
> {
> SBError GetMemoryRegionInfo (lldb::addr_t load_addr, 
> SBMemoryRegionInfo &range_info);
> };
> }
> 
> then you would call this API with address zero and it would return a
> SBMemoryRegionInfo with an address range and if that memory is read/
> write/execute. On MacOSX we always have a page zero at address 0 for
> 64 bit apps so it would respond with:
> 
> [0x0 - 0x1) read=false, write=false, execute=false
> 
> Then you call the function again with the end address of the previous 
range. 
> 
> I would love to see this functionality exported through our public 
> API. Let me know if you are up for making a patch. If you are, you 
> might want to quickly read the following web page to see the rules 
> that we apply to anything going into our public API:
> 
> http://lldb.llvm.org/SB-api-coding-rules.html
> 
> 
> Greg
> 
> > On May 12, 2016, at 6:20 AM, Howard Hellyer via lldb-dev  d...@lists.llvm.org> wrote:
> > 
> > I'm worki

Re: [lldb-dev] Listing memory regions in lldb

[Adrian McCarthy via lldb-dev]

>
> Interestingly when I've worked on Windows core dumps before I've seen that
> MiniDump, with the right flags, will deliberately insert a region in the
> middle of the memory ranges to represent the unmapped space, on 64 bit it's
> quite a large section.


Are you saying that's a bug in the minidump itself or in LLDB's handling of
it?  If the latter, please send me the details (e.g., which flags trigger
this behavior), and I'll see what I can do about it.

Adrian.

On Fri, May 13, 2016 at 3:35 AM, Howard Hellyer via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> I have experimented with the GetMemoryRegionInfo approach on the internal
> APIs already, it has some positives and negatives.
>
> - GetMemoryRegionInfo is unimplemented for Linux and Mac core dumps.
> That's not necessarily a bad thing as it could be implemented the "right"
> way. (As Jim said GetMemoryRegionInfo would have to return the right thing
> for an unmapped region.) Interestingly when I've worked on Windows core
> dumps before I've seen that MiniDump, with the right flags, will
> deliberately insert a region in the middle of the memory ranges to
> represent the unmapped space, on 64 bit it's quite a large section.
>
> - Using GetMemoryRegionInfo to iterate might be quite expensive if there
> are many small memory regions.
>
> - One reason I hadn't looked at just exposing an SBGetMemoryRegionInfo is
> that it wouldn't match a lot of the SB API's at the moment (for example for
> Threads and Sections) which tend work by having GetNumXs() and
> GetXAtIndex() calls. Iterating with SBGetMemoryRegionInfo would be
> non-obvious to the user of the API since it doesn't match the convention.
> It would need to be documented in the SBGetMemoryRegionInfo API.
>
> - I've found the only way to reliably filter out inaccessible memory is to
> do a test read and check the error return code. I'm pretty sure I've seen
> some that are readable via a memory read but marked read=false,
> write=false, execute=false. (I can't remember the exact case now off the
> top of my head, but I think it might have been allocated but uncommitted
> memory or similar.)
>
> - Using GetMemoryRegionInfo over a remote connection on Linux and Mac
> worked well but seemed to coalesce some of the memory regions together. It
> also only allows for read/write/exec attributes to be passed. That's a
> shame as a live process can often tell you more about what the region is
> for. The remote command memory map looks like it sends back XML so it might
> be possible to insert custom properties in there to give more information
> but I'm not sure how safe it is to do that, I don't know the project quite
> well enough to understand all the use cases for the protocol.
>
> - Extended infomation /proc/pid/maps marking a region as [stack] would be
> lost. All you would get is read/write/exec info. (That said supporting
> everything every platform can provide might be a bit much.)
>
> - LLDB's ReadMemory implementations seem to return 0's for missing memory
> that should be accessible. It might be nice to include whether the memory
> is backed by real data or not in the API. (For example Linux core files can
> be missing huge pages depending on the setting of /proc/PID/coredump_filter
> or files can simply be truncated.)
>
> I could implement the GetMemoryRegionInfo iteration mechanism pretty
> quickly and it would actually fit my purposes as far as providing all the
> addresses you can sensibly access.
>
> I'm quite keen to provide a patch but don't want to provide one that is at
> odds with how the rest of lldb works or provides data that's only useful to
> me so I'm quite keen to get a bit of feedback on what the preferred
> approach would be. It could be that providing both SBGetMemoryRegionInfo
> and the SBGetNumMemoryRegions/SBGetMemoryRegionAtIndex pair is the right
> solution.
>
> Would a patch also need to provide a command to dump this information as
> it can be awkward to have data that's only accessible via the API?
>
> *Howard Hellyer*
> IBM Runtime Technologies, IBM Systems
>
>
>
>
> Greg Clayton  wrote on 12/05/2016 19:09:49:
>
> > From: Greg Clayton 
> > To: Howard Hellyer/UK/IBM@IBMGB
> > Cc: lldb-dev@lists.llvm.org
> > Date: 12/05/2016 19:10
> > Subject: Re: [lldb-dev] Listing memory regions in lldb
> >
> > We have a way internally with:
> >
> > virtual Error
> > lldb_private::Process::GetMemoryRegionInfo (lldb::addr_t
> > load_addr, MemoryRegionInfo &range_info);
> >
> > This isn't expose via the public API in lldb::SBProcess. If you
> > want, you can expose this. We would need to expose a
> > SBMemoryRegionInfo and the call could be:
> >
> > namespace lldb
> > {
> > class SBProcess
> > {
> > SBError GetMemoryRegionInfo (lldb::addr_t load_addr,
> > SBMemoryRegionInfo &range_info);
> > };
> > }
> >
> > then you would call this API with address zero and it would return a
> > SBMemoryRegionInfo with an address range and if that memory is read/
> > wri

Re: [lldb-dev] Listing memory regions in lldb

[Zachary Turner via lldb-dev]

On Intel processors, the best way to do this is probably going to be to
walk the page directory (see Intel processor manuals). Assuming someone
implements this command in lldb, I hope it can be done in such a way as to
allow different implementations when one os/architecture has a better way
of doing it.

On Thu, May 12, 2016 at 11:09 AM Jim Ingham via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> You should be able to enumerate the memory that is occupied by loaded
> executables, by getting the list of loaded Modules from the target, and
> iterate through the all the Sections.  The Sections know their loaded
> locations.  I assume all the mapped ones will return a valid load address
> from GetLoadBaseAddress, so you can distinguish the loaded and unloaded
> ones.  So you shouldn't need "readelf --segments" or "otool -l", lldb
> should know this.
>
> You can scan the currently active stacks, but we don't currently know the
> allocated stack extents, just what is being used.  It would be interesting
> to know the actual stack extents, so you could search in old stacks and to
> know if you are close to exhausting the stack of a thread.
>
> We don't have either a generic API, or internal implementations, for
> getting all the mapped memory regions (or shared pages) of processes.  That
> would be quite useful.  IIRC ptr_refs does this by injecting some code into
> the target program that enumerates the regions.  Greg would know more about
> this.  Most systems provide some API to get at this that works
> cross-process, but that doesn't help debugging remotely.  So we either need
> to teach debugserver & lldb-server to do this, or use appropriate code
> injection.  The gdb-remote protocol has a query for the "memory map" of the
> process, though this is more tailored to identify things like memory mapped
> registers.  Still it might be possible to use this as well.
>
> It would be nice to be able to separately query heap, executable & stack
> memory as well.  Though a properly annotated memory map would give you a
> way to do this, so that could be layered on top.
>
> Jim
>
> > On May 12, 2016, at 6:20 AM, Howard Hellyer via lldb-dev <
> lldb-dev@lists.llvm.org> wrote:
> >
> > I'm working on a plugin for lldb and need to scan the memory of a
> crashed process. Using the API to read chunks of memory and scan (via
> SBProcess::Read*) for what I'm looking for is easy but I haven't been able
> to find a way to find which address ranges are accessible. The
> SBProcess::Read* calls will return an error on an invalid address but it's
> not an efficient way to scan a 64 bit address space.
> >
> > This seems like it blocks simple tasks like scanning memory for blocks
> allocated with a header and footer to track down memory leaks, which is
> crude but traditional, and ought to be pretty quick to script via the
> Python API.
> >
> > At the moment I've resorted to running a python script prior to
> launching my plugin that takes the output of "readelf --segments",
> /proc//maps or "otool -l" but this isn't ideal. On the assumption that
> I'm not missing something huge I've looked at whether it is possible to
> extend LLDB to provide this functionality and it seems possible, there are
> checks protecting calls to read memory that use the data that would need to
> be exposed. I'm working on a prototype implementation which I'd like to
> deliver back at some stage but before I go too far does this sound like a
> good idea?
> > Howard Hellyer
> > IBM Runtime Technologies, IBM Systems
> >
> >
> > ___
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] How to use command regex in this situation?

[Karl Peng via lldb-dev]

Actually I posted a question 

 in stackoverflow?

Can any one  give me some suggestion?

Thanks___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] No C++ SB API for Watchpoint callbacks?

[Daniel Noland via lldb-dev]

Please correct me if I have missed something critical here, but the C++
SB API does not seem to provide a way to set callbacks to run when a
watchpoint is hit (while SBBreakpoint and the private Watchpoint class
does offer this functionality).

* Is this omission by design?
* Is there another way to get similar functionality?
* Would it be a good idea for me to contribute code which adds this
functionality?

I read through some of the source code, and I think it would be quite
possible to add callback functionality in a manor nearly identical to
the SBBreakpoint implementation.

I have briefly started putting together the needed changes, but I
thought I would check here to make sure I am solving a real problem in a
good way before I go too far.

Thanks,
\author{Daniel Noland}



signature.asc
Description: OpenPGP digital signature
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] Standalone build broken after r269332

[Eugene Zelenko via lldb-dev]

Hi!

I tried to build standalone LLDB after r269332 change in
LLDBStandalone.cmake and encountered next problems:

It implies that LLVM and Clang were built separately, but they could
be built together, so next changes should be made as in previous
version:

-  
include("${LLVM_OBJ_ROOT}/lib${LLVM_LIBDIR_SUFFIX}/cmake/clang/ClangConfig.cmake")
+  if (EXISTS 
"${LLVM_OBJ_ROOT}/lib${LLVM_LIBDIR_SUFFIX}/cmake/clang/ClangConfig.cmake")
+
include("${LLVM_OBJ_ROOT}/lib${LLVM_LIBDIR_SUFFIX}/cmake/clang/ClangConfig.cmake")
+  endif()

Include path for generated LLVM headers were not added, so I made next tweak:

-  include_directories("${LLVM_BINARY_DIR}/include" "${LLVM_MAIN_INCLUDE_DIR}")
+  include_directories("${LLVM_OBJ_ROOT}/include"
"${LLVM_BINARY_DIR}/include" "${LLVM_MAIN_INCLUDE_DIR}")

But biggest problem remains is how to specify Clang source and build
directories, since llvm-config doesn't tells about them.

Previously I set next variables, but situation may be different when
LLVM and Clang are built separately:

-DLLDB_PATH_TO_CLANG_BUILD=${LLVMBuildDir}/tools/clang
-DLLDB_PATH_TO_CLANG_SOURCE=${LLVMSourceDir}/tools/clang
-DLLDB_PATH_TO_LLVM_BUILD=${LLVMBuildDir}
-DLLDB_PATH_TO_LLVM_SOURCE=${LLVMSourceDir}

Eugene.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Standalone build broken after r269332

[Kamil Rytarowski via lldb-dev]

On 14.05.2016 02:46, Eugene Zelenko via lldb-dev wrote:
> But biggest problem remains is how to specify Clang source and build
> directories, since llvm-config doesn't tells about them.

The requirement of source-code of Clang+LLVM and build dirs of Clang and
LLVM is no-go in a package-manager use-case. This is only needed for the
import of regex implementation pulled out of LLDB sources.

The proper fix for it is to stop using regex implementation from out of
the LLDB sources.

Temporarily, I'm building LLDB on NetBSD with a local patch [1] adding
these regex libraries without any further issues in the standalone
build. In order to work on proper regex usage in LLDB, I need to
increase the number of passed tests on my platform -- to catch any
regressions easier.

[1] https://github.com/NetBSD/pkgsrc-wip/tree/master/lldb-git/patches
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] LLDB targeting NetBSD-8.0

[Kamil Rytarowski via lldb-dev]

I'm moving my efforts of porting LLDB from NetBSD-7.0 to NetBSD-8.0.

I decided that 7.0 isn't worth more effort since libstdc++ is already
too old (no ) and it misses support for debug registers and
useful ptrace(2) interfaces for a full-process instrumentation.

I'm going to upgrade the LLVM+Clang+LLDB buildbot to NetBSD HEAD (pre 8.0).
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev