https://github.com/DmT021 closed
https://github.com/llvm/llvm-project/pull/110439
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
DmT021 wrote:
I'm closing this in favor https://github.com/llvm/llvm-project/pull/110646
https://github.com/llvm/llvm-project/pull/110439
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-com
jasonmolenda wrote:
Sounds good, I agree. I marked
https://github.com/llvm/llvm-project/pull/110646 as approved, please merge at
your convenience. Thanks again for spotting this opportunity for
multithreading and seeing it through!
https://github.com/llvm/llvm-project/pull/110439
__
DmT021 wrote:
> I think that's where the real perf difference you were measuring kicked in
> (you showed a 10.5 second versus 13.4 second time difference for preload
> versus parallel in the beginning), do I have that right?
That's right.
> Do you prefer the non-preload approach?
No, I thin
jasonmolenda wrote:
Thanks for all the work you've done on this, and updating the setting. I
looked over the implementations, and they all look like reasonable changes to
me - I did laugh a little when I realized that 2/3rds of all the changes were
related to adding the setting :) that's alwa
DmT021 wrote:
> I was playing with the performance in a couple of different scenarios. For
> some reason that I haven't looked into, we're getting less parallelism when
> many of the binaries are in the shared cache in lldb. Maybe there is locking
> around the code which finds the binary in ll
jimingham wrote:
> > it should definitely be enabled by default when we're at the point to merge
> > it, and the setting should only be a safety mechanism if this turns out to
> > cause a problem for a configuration we weren't able to test.
>
> That's fine with me either.
>
> > I would even p
DmT021 wrote:
> I was chatting with Jim Ingham and he was a little bummed that we're looking
> at doing this in a single DynamicLoader plugin, instead of having the
> DynamicLoader plugin create a list of ModuleSpec's and having a central
> method in ModuleList or something, create Modules for
DmT021 wrote:
> it should definitely be enabled by default when we're at the point to merge
> it, and the setting should only be a safety mechanism if this turns out to
> cause a problem for a configuration we weren't able to test.
That's fine with me either.
> I would even put it explicitly
jasonmolenda wrote:
For what it's worth, this thread pool for parallel processing has been used in
another part of lldb - it's used on ELF systems when processing DWARF, when we
need to scan the debug info in the individual .o files iirc. So we've had some
living-on time with the thread pool
jasonmolenda wrote:
> > Was the setting intended for testing purposes only, or did you intend to
> > include that in a final PR?
>
> The latter. IMO the risks involved by parallelization are a bit too high to
> do it without a flag. I'm even thinking about having it opt-in rather than
> opt-o
DmT021 wrote:
> Was the setting intended for testing purposes only, or did you intend to
> include that in a final PR?
The latter. IMO the risks involved by parallelization are a bit too high to do
it without a flag. I'm even thinking about having it opt-in rather than opt-out
for some time.
jasonmolenda wrote:
I built the swiftlang `rebranch` sources clean, with the parallel patch, with
the parallel plus preload patch. I ran them against the same Slack binary with
934 binary images, 10 of them outside the shared cache. I built them something
like
```
swift/utils/build-script --
jasonmolenda wrote:
It is a bit interesting that the user and system time are so much higher --
looks like around 40% higher? -- than the single-threaded approach, I wonder if
that's thread creation/teardown overhead, or if it's costlier to acquire locks
and it's adding up. Wallclock is the r
jasonmolenda wrote:
I know your benchmarking numbers are against the `main` swiftlang branch which
is using stable/20230725 - it's a bit older than llvm-project main as you'd
guess from the date. They're bringing up the `rebranch` branch right now to
become `main` at some point (I haven't fol
DmT021 wrote:
Anyway, my main goal is iOS apps running in the simulator. And for them, the
speed-up is much more noticeable (at least for big apps). Let me know if you'd
like me to measure something else.
https://github.com/llvm/llvm-project/pull/110439
DmT021 wrote:
Nice. Also there's no significant difference between `build.release` and
`release.parallel-with-preload` with sequential load which is good.
I wonder why your parallel runs are slightly less performant than mine relative
to the sequential ones. I mean it's almost certainly the con
jasonmolenda wrote:
So yes, I'm seeing a 30% improvement with the multithreaded creation of
Modules, for a macOS app mostly in the shared cache.
https://github.com/llvm/llvm-project/pull/110439
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
jasonmolenda wrote:
For completeness sakes, here's what I see for parallel+preload PR 110646 with
llvm-project main (no swift demangling) attaching to Slack (nearly all
libraries in the shared cache)
```
% time build.release.parallel-with-preload//bin/lldb -x -b -o 'pro att -n
Slack' -o 'det
jasonmolenda wrote:
Ah wait, I see where the difference is coming in -- it's my own fault. I
misread tcsh's time output and only looked at USER seconds. Looking at the
wallclock times, we're 50% faster with the multithreading enabled for this PR's
change. My apologies!
https://github.com/l
jasonmolenda wrote:
Hmmm, interesting. I built my optimized builds with
```
cmake ../llvm -G Ninja -DLLDB_ENABLE_SWIFT_SUPPORT=0
-DPython3_EXECUTABLE=/usr/bin/python3 -DCMAKE_BUILD_TYPE=RelWithDebInfo
-DLLVM_ENABLE_ASSERTIONS=1 -DLLVM_ENABLE_PROJECTS='llvm;lldb;clang'
'-DLLVM_ENABLE_RUNTIMES=
DmT021 wrote:
@jasonmolenda I tried to reproduce your results but got drastically different
numbers for parallel runs. Here's the setup:
- Using this (the main) repository of llvm-project.
- `cmake -G Ninja -DLLVM_ENABLE_PROJECTS="clang;lldb;lld"
-DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_ENABLE_
DmT021 wrote:
@jasonmolenda If you still have a build with the patch can you please compare
the unmodified version with the patched one with `enable-parallel-image-load`
disabled? There shouldn't be a noticeable difference. If you don't it's ok, I
haven't had time to inspect it yet but I'll tr
bulbazord wrote:
Love to see this kind of work done.
FWIW I worked on improving the performance of `ObjectFileMachO::ParseSymtab` in
https://github.com/llvm/llvm-project/pull/106791.
That change to reverted and I haven't had the time to figure out what happened
yet though...
https://github.com
jasonmolenda wrote:
> > Built RelWithDebInfo, unmodified lldb took 4.5 seconds. Parallel took 6.6
> > seconds. Parallel with preload took 6.7 seconds.
> > Built Debug, unmodified lldb took 27.6 seconds. Parallel took 35.5 seconds.
> > Parallel plus preload took 35.6 seconds.
>
> Oh wow. 4.5 s
jasonmolenda wrote:
Ah, yes Simulator debugging is a bit different than macOS process debugging.
With macOS process debugging -- with lldb and the inferior process running on
the same computer -- lldb is using the same shared cache as the inferior
process, so to read the libraries it reads th
DmT021 wrote:
> my main concern was that all of the Modules would be trying to add strings to
> the constant string pool and lock contention could become a bottleneck.
Yeah it sure would be a bottleneck. I didn't measure it precisely but I think I
saw something about 30-50% of the time is spen
jasonmolenda wrote:
nb I used
`plugin.experimental.dynamic-loader.darwin.enable-parallel-image-load` as the
settings name so I could use the same command for all three types of lldb's -
the unmodified lldb would give an error on an unrecognized setting normally and
I didn't want to see that.
jasonmolenda wrote:
This is an interesting idea, thanks for looking into it. When I spoke with
Augusto about it, my main concern was that all of the Modules would be trying
to add strings to the constant string pool and lock contention could become a
bottleneck.
I built github main unmodifie
DmT021 wrote:
@augusto2112 Take a look when you have time
This is just one possible approach to parallelizing the initial image loading.
The other solution I'm checking now is to parallelize loading somewhere
earlier, perhaps in `DynamicLoaderMacOS::DoInitialImageFetch`. The difference
is that
https://github.com/DmT021 edited
https://github.com/llvm/llvm-project/pull/110439
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
https://github.com/DmT021 created
https://github.com/llvm/llvm-project/pull/110439
When `plugin.dynamic-loader.darwin.enable-parallel-image-load` is enabled
`DynamicLoaderDarwin::AddModulesUsingImageInfos` will load images in parallel
using the thread pool.
>From d6183d6b0e1c755d6adf3f5246374
32 matches
Mail list logo
