date:20201110

Re: [lldb-dev] [RFC] Segmented Address Space Support in LLDB

2020-11-10 Thread Zdenek Prikryl via lldb-dev


Hi all,

Just for the record, we have successfully implemented the wrapping of 
addr_t into a class to support multiple address spaces. The info about 
address space is stored in the ELF file, so we get the info from ELF 
parser and then pass it to the rest of the system. CLI/MI interface has 
been extended as well, so user can select with address space he wants 
for memory printing. Similarly, we patched expression evaluation, 
disassembler, etc.


If the address wrap is part of the upstream version, it will be awesome 
:-)...


Best regards.

On 10/20/20 9:30 PM, Ted Woodward via lldb-dev wrote:

I agree with Pavel about the larger picture - we need to know the driver behind 
address spaces before we can discuss a workable solution.

I've dealt with 2 use cases - Harvard architecture cores, and low level 
hardware debugging.

A Harvard architecture core has separate instruction and data memories. These often use 
the same addresses, so to distinguish between them you need address spaces. The Motorola 
DSP56300 had 1 program and 2 data memories, called p, x and y. p:100, x:100 and y:100 
were all separate memories, so "address 100" isn't enough to get what the user 
needed to see.

For low level hardware debugging (often using JTAG), many devices let you access memories in ways like 
"virtual using the TLB", or "virtual == physical, through the core", or "physical, 
through the SoC, not cached". Memory spaces, done right, can give the user the flexibility to pick how 
to view memory.


Are these the use cases you were envisioning, Jonas?


-Original Message-
From: lldb-dev  On Behalf Of Pavel Labath
via lldb-dev
Sent: Tuesday, October 20, 2020 12:51 PM
To: Jonas Devlieghere ; LLDB 
Subject: [EXT] Re: [lldb-dev] [RFC] Segmented Address Space Support in
LLDB

There's a lot of things that are unclear to me about this proposal. The
mechanics of representing an segmented address are one thing, but I I think
that the really interesting part will be the interaction with the rest of lldb. 
Like
- What's going to be the source of this address space information? Is it going
to be statically baked into lldb (a function of the target architecture?), or
dynamically retrieved from the target or platform we're debugging? How
would that work?
- How is this going to interact with Object/SymbolFile classes? Are you
expecting to use existing object and symbol formats for address space
information, or some custom ones? AFAIK, none of the existing formats
actually support encoding address space information (though that hasn't
stopped people from trying).

Without understanding the bigger picture it's hard for me to say whether the
proposed large scale refactoring is a good idea. Nonetheless, I am doubtful of
the viability of that approach. Some of my reasons for that are:
- not all addr_ts represent an actual address -- sometimes that is a difference
between two addresses, which still uses addr_t, as that's guaranteed to fit.
- relatedly to that, there is a difference (I'd expect) between the operations
supported by the two types. addr_t supports all integral operations (though I
hope we don't use all of them), but I wouldn't expect to be able to do the
same with a SegmentedAddress. For one, I'd expect it wouldn't be possible
to add two SegmentedAddresses together (which is possible for addr_t).
OTOH, adding a SegmentedAddress and an addr_t would probably be fine?
Would subtracting two SegmentedAddresses should result in an addr_t? But
only if they have matching address spaces (and assert otherwise)?
- I'd also be worried about over-generalizing specialized code which can
afford to work with plain addresses, and where the added address space
would be a nuisance (or a source of bugs). E.g. ELF has no notion of address
space, so I don't think I'd find it helpful to replace all plain integer 
calculations
in elf parsing code with something more complex.
(I'm aware that some people are using elf to encode address space
information, but this is a pretty nonstandard extension, and it'd take more
than type substitution to support anything like that.)
- large scale refactorings are very much not the norm in llvm



On 19/10/2020 23:56, Jonas Devlieghere via lldb-dev wrote:

We want to support segmented address spaces in LLDB. Currently, all of
LLDB’s external API, command line interface, and internals assume that
an address in memory can be addressed unambiguously as an addr_t (aka
uint64_t). To support a segmented address space we’d need to extend
addr_t with a discriminator (an aspace_t) to uniquely identify a
location in memory. This RFC outlines what would need to change and
how we propose to do that.

### Addresses in LLDB

Currently, LLDB has two ways of representing an address:

   - Address object. Mostly represents addresses as Section+offset for
a binary image loaded in the Target. An Address in this form can
persist across executions, e.g. an address breakpoint in a binary
image that loads at a different address e

Re: [lldb-dev] [RFC] Segmented Address Space Support in LLDB

2020-11-10 Thread Jonas Devlieghere via lldb-dev


> On Nov 10, 2020, at 12:58 PM, Zdenek Prikryl  wrote:
> 
> Hi all,
> 
> Just for the record, we have successfully implemented the wrapping of addr_t 
> into a class to support multiple address spaces. The info about address space 
> is stored in the ELF file, so we get the info from ELF parser and then pass 
> it to the rest of the system. CLI/MI interface has been extended as well, so 
> user can select with address space he wants for memory printing. Similarly, 
> we patched expression evaluation, disassembler, etc.

That's really interesting, I'm excited to hear that this is feasible and has 
been done before. Is this code available publicly and/or is this something 
you'd be willing to upstream (with our help)? 

> 
> If the address wrap is part of the upstream version, it will be awesome :-)...
> 
> Best regards.
> 
> On 10/20/20 9:30 PM, Ted Woodward via lldb-dev wrote:
>> I agree with Pavel about the larger picture - we need to know the driver 
>> behind address spaces before we can discuss a workable solution.
>> 
>> I've dealt with 2 use cases - Harvard architecture cores, and low level 
>> hardware debugging.
>> 
>> A Harvard architecture core has separate instruction and data memories. 
>> These often use the same addresses, so to distinguish between them you need 
>> address spaces. The Motorola DSP56300 had 1 program and 2 data memories, 
>> called p, x and y. p:100, x:100 and y:100 were all separate memories, so 
>> "address 100" isn't enough to get what the user needed to see.
>> 
>> For low level hardware debugging (often using JTAG), many devices let you 
>> access memories in ways like "virtual using the TLB", or "virtual == 
>> physical, through the core", or "physical, through the SoC, not cached". 
>> Memory spaces, done right, can give the user the flexibility to pick how to 
>> view memory.
>> 
>> 
>> Are these the use cases you were envisioning, Jonas?
>> 
>>> -Original Message-
>>> From: lldb-dev  On Behalf Of Pavel Labath
>>> via lldb-dev
>>> Sent: Tuesday, October 20, 2020 12:51 PM
>>> To: Jonas Devlieghere ; LLDB >> d...@lists.llvm.org>
>>> Subject: [EXT] Re: [lldb-dev] [RFC] Segmented Address Space Support in
>>> LLDB
>>> 
>>> There's a lot of things that are unclear to me about this proposal. The
>>> mechanics of representing an segmented address are one thing, but I I think
>>> that the really interesting part will be the interaction with the rest of 
>>> lldb. Like
>>> - What's going to be the source of this address space information? Is it 
>>> going
>>> to be statically baked into lldb (a function of the target architecture?), 
>>> or
>>> dynamically retrieved from the target or platform we're debugging? How
>>> would that work?
>>> - How is this going to interact with Object/SymbolFile classes? Are you
>>> expecting to use existing object and symbol formats for address space
>>> information, or some custom ones? AFAIK, none of the existing formats
>>> actually support encoding address space information (though that hasn't
>>> stopped people from trying).
>>> 
>>> Without understanding the bigger picture it's hard for me to say whether the
>>> proposed large scale refactoring is a good idea. Nonetheless, I am doubtful 
>>> of
>>> the viability of that approach. Some of my reasons for that are:
>>> - not all addr_ts represent an actual address -- sometimes that is a 
>>> difference
>>> between two addresses, which still uses addr_t, as that's guaranteed to fit.
>>> - relatedly to that, there is a difference (I'd expect) between the 
>>> operations
>>> supported by the two types. addr_t supports all integral operations (though 
>>> I
>>> hope we don't use all of them), but I wouldn't expect to be able to do the
>>> same with a SegmentedAddress. For one, I'd expect it wouldn't be possible
>>> to add two SegmentedAddresses together (which is possible for addr_t).
>>> OTOH, adding a SegmentedAddress and an addr_t would probably be fine?
>>> Would subtracting two SegmentedAddresses should result in an addr_t? But
>>> only if they have matching address spaces (and assert otherwise)?
>>> - I'd also be worried about over-generalizing specialized code which can
>>> afford to work with plain addresses, and where the added address space
>>> would be a nuisance (or a source of bugs). E.g. ELF has no notion of address
>>> space, so I don't think I'd find it helpful to replace all plain integer 
>>> calculations
>>> in elf parsing code with something more complex.
>>> (I'm aware that some people are using elf to encode address space
>>> information, but this is a pretty nonstandard extension, and it'd take more
>>> than type substitution to support anything like that.)
>>> - large scale refactorings are very much not the norm in llvm
>>> 
>>> 
>>> 
>>> On 19/10/2020 23:56, Jonas Devlieghere via lldb-dev wrote:
 We want to support segmented address spaces in LLDB. Currently, all of
 LLDB’s external API, command line interface, and internals assume that
 an address in

Re: [lldb-dev] [RFC] Segmented Address Space Support in LLDB

2020-11-10 Thread Zdenek Prikryl via lldb-dev



On 11/10/20 10:01 PM, Jonas Devlieghere wrote:

On Nov 10, 2020, at 12:58 PM, Zdenek Prikryl  wrote:

Hi all,

Just for the record, we have successfully implemented the wrapping of addr_t 
into a class to support multiple address spaces. The info about address space 
is stored in the ELF file, so we get the info from ELF parser and then pass it 
to the rest of the system. CLI/MI interface has been extended as well, so user 
can select with address space he wants for memory printing. Similarly, we 
patched expression evaluation, disassembler, etc.

That's really interesting, I'm excited to hear that this is feasible and has 
been done before. Is this code available publicly and/or is this something 
you'd be willing to upstream (with our help)?


The code is not available at public repository, but we're more than 
happy to work on/help with the new proposal and share our findings or 
code snippets.


--
Zdenek Prikryl
CTO
T +420 541 141 475
Codasip.com

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Need help with failing LLDB tests on Windows

2020-11-10 Thread Tatyana Krasnukha via lldb-dev

On Windows one should run debug version of Python (python_d.exe) to load debug 
version of liblldb.dll. I hope this will help you.

From: lldb-dev  On Behalf Of Adrian McCarthy 
via lldb-dev
Sent: Tuesday, November 10, 2020 4:00 AM
To: Ted Woodward 
Cc: LLDB 
Subject: Re: [lldb-dev] Need help with failing LLDB tests on Windows

Thanks for all the info and pointers.  That's helping me zero in on the problem.

This category of the failures appears to all be dotest.py tests, so it makes 
sense that it's the second import statement (per Pavel's explanation).

The module is not being found because it's actually named _lldb_d.pyd.  
Apparently the `_d` suffix is because I'm building debug.  That seems 
consistent with Stella's experience.

However, I've been building debug since before these problems arose.  (In fact, 
I've been working on fixes for a small number of tests that only fail in debug, 
because of an assertion that detects the problem.)

Ted's got me thinking that it was working due to a symlink that somehow got 
blown away and/or isn't being recreated by the build.  If I recall correctly, 
the symlinks on Windows are created using ln.exe, which may come from GnuWin32 
or from git/usr/bin.  In my case, it's git/usr/bin.  There seem to have been 
many git updates in the past couple months, so perhaps one of those updates 
tweaked ln.exe.  That could have been the trigger for me.  Folks who didn't 
take the git update or who are configured to prefer GnuWin32 tools might not 
have been affected.

I'll let you know what I eventually find.

On Wed, Nov 4, 2020 at 12:05 PM Ted Woodward 
mailto:tedw...@quicinc.com>> wrote:
To expand a bit on what Pavel has written, the lldb module should be in 
\lib\site-packages\lldb . In that directory is a file, _lldb.pyd, that 
should be a copy of \bin\liblldb.dll .

Do both files exist? Is _lldb.pyd a copy of liblldb.dll?
See function create_relative_symlink in llvm-project/lldb/CMakeLists.txt for 
the copy (on non-unix hosts) procedure.

Did you recently change your version of swig? LLDB requires swig 2, but, as you 
pointed out last year, there are issues with some versions of swig. We use 
4.0.1 on Windows.

> -Original Message-
> From: lldb-dev 
> mailto:lldb-dev-boun...@lists.llvm.org>> On 
> Behalf Of Pavel Labath
> via lldb-dev
> Sent: Wednesday, November 4, 2020 2:49 AM
> To: Adrian McCarthy mailto:amcca...@google.com>>; LLDB 
>  d...@lists.llvm.org>
> Subject: [EXT] Re: [lldb-dev] Need help with failing LLDB tests on Windows
>
> On 04/11/2020 01:53, Adrian McCarthy via lldb-dev wrote:
> > For the past couple weeks, I've been trying to figure out why
> > approximately 900+ LLDB tests have been failing for me on my local
> > Windows builds.  Bisect turned up nothing--the "good" version that was
> > working for me no longer works.  Since nobody else seems to be seeing
> > these failures, I suspect it's something environmental.
> >
> > There are three categories of errors.  I'm currently focused on
> > failures that look like this:
> >
> > FAIL: lldb-api :: lang/objc/unicode-string/TestUnicodeString.py (732
> > of 2180)
> >  TEST 'lldb-api ::
> > lang/objc/unicode-string/TestUnicodeString.py' FAILED
> > 
> > Script:
> > --
> > C:/Program Files/Python38/python.exe
> > D:/src/llvm/llvm-project/lldb\test\API\dotest.py -S nm -u CXXFLAGS
> > -u CFLAGS --enable-crash-dialog --env
> > LLVM_LIBS_DIR=D:/src/llvm/build/ninja_dbg/./lib --arch x86_64
> > --build-dir D:/src/llvm/build/ninja_dbg/lldb-test-build.noindex -s
> > D:/src/llvm/build/ninja_dbg/lldb-test-traces --lldb-module-cache-dir
> > D:/src/llvm/build/ninja_dbg/lldb-test-build.noindex/module-cache-
> lldb\lldb-api
> > --clang-module-cache-dir
> > D:/src/llvm/build/ninja_dbg/lldb-test-build.noindex/module-cache-
> clang\lldb-api
> > --executable D:/src/llvm/build/ninja_dbg/./bin/lldb.exe --compiler
> > D:/src/llvm/build/ninja_dbg/bin/clang.exe --dsymutil
> > D:/src/llvm/build/ninja_dbg/./bin/dsymutil.exe --filecheck
> > D:/src/llvm/build/ninja_dbg/./bin/FileCheck.exe --yaml2obj
> > D:/src/llvm/build/ninja_dbg/./bin/yaml2obj.exe --lldb-libs-dir
> > D:/src/llvm/build/ninja_dbg/./lib
> > D:\src\llvm\llvm-project\lldb\test\API\lang\objc\unicode-string -p
> > TestUnicodeString.py
> > --
> > Exit Code: 1
> >
> > Command Output (stdout):
> > --
> > lldb version 12.0.0 
> > (https://github.com/llvm/llvm-project.git
> > 
> > >
> >  revision
> > 0fdcd1ae1c988fa19d0c97e9e8678

[lldb-dev] [Bug 48143] New: lldb-vscode tests failing on macOS after `check-lldb`

2020-11-10 Thread via lldb-dev

https://bugs.llvm.org/show_bug.cgi?id=48143

Bug ID: 48143
   Summary: lldb-vscode tests failing on macOS after `check-lldb`
   Product: lldb
   Version: unspecified
  Hardware: PC
OS: All
Status: NEW
  Severity: enhancement
  Priority: P
 Component: All Bugs
  Assignee: lldb-dev@lists.llvm.org
  Reporter: v...@apple.com
CC: jdevliegh...@apple.com, llvm-b...@lists.llvm.org

Several lldb-vscode tests appear to fail 100% of the time on macOS 10.15.6.

I see two failure modes. The first is:

```
  File "../lldb/packages/Python/lldbsuite/test/tools/lldb-vscode/vscode.py",
line 306, in send_recv
raise ValueError(desc)
ValueError: no response for "initialize"
```

And the second is:

```
  File "../lldb/packages/Python/lldbsuite/test/tools/lldb-vscode/vscode.py",
line 573, in request_disconnect
return self.send_recv(command_dict)
  File "../lldb/packages/Python/lldbsuite/test/tools/lldb-vscode/vscode.py",
line 306, in send_recv
raise ValueError(desc)
ValueError: no response for "disconnect"
```

After the test run, I see some ~20 or so lldb-vscode processes idling on my
machine. The specific tests that are failing are:

```
Failed Tests (13):
  lldb-api :: tools/lldb-vscode/attach/TestVSCode_attach.py
  lldb-api ::
tools/lldb-vscode/breakpoint-events/TestVSCode_breakpointEvents.py
  lldb-api :: tools/lldb-vscode/breakpoint/TestVSCode_setBreakpoints.py
  lldb-api ::
tools/lldb-vscode/breakpoint/TestVSCode_setExceptionBreakpoints.py
  lldb-api :: tools/lldb-vscode/breakpoint/TestVSCode_setFunctionBreakpoints.py
  lldb-api :: tools/lldb-vscode/console/TestVSCode_console.py
  lldb-api :: tools/lldb-vscode/coreFile/TestVSCode_coreFile.py
  lldb-api :: tools/lldb-vscode/launch/TestVSCode_launch.py
  lldb-api :: tools/lldb-vscode/module/TestVSCode_module.py
  lldb-api :: tools/lldb-vscode/runInTerminal/TestVSCode_runInTerminal.py
  lldb-api :: tools/lldb-vscode/stackTrace/TestVSCode_stackTrace.py
  lldb-api :: tools/lldb-vscode/step/TestVSCode_step.py
  lldb-api :: tools/lldb-vscode/variables/TestVSCode_variables.py
```

-- 
You are receiving this mail because:
You are the assignee for the bug.___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [RFC] Segmented Address Space Support in LLDB

Re: [lldb-dev] [RFC] Segmented Address Space Support in LLDB

Re: [lldb-dev] [RFC] Segmented Address Space Support in LLDB

Re: [lldb-dev] Need help with failing LLDB tests on Windows

[lldb-dev] [Bug 48143] New: lldb-vscode tests failing on macOS after `check-lldb`

5 matches

Site Navigation

Mail list logo

Footer information