[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

This revision was automatically updated to reflect the committed changes. Closed by commit rG0f7e8631547a: [LinkerWrapper] Perform device linking steps in parallel (authored by jhuber6). Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D136701/new/ http

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

jhuber6 added a comment. ping Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D136701/new/ https://reviews.llvm.org/D136701 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

jhuber6 updated this revision to Diff 471998. jhuber6 added a comment. Sorting on offload kind as well. Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D136701/new/ https://reviews.llvm.org/D136701 Files: clang/test/Driver/linker-wrapper.c clang/

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

jhuber6 added inline comments. Comment at: clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp:1211-1212 +llvm::sort(Input, [](OffloadingImage &A, OffloadingImage &B) { + return A.StringData["triple"].compare(B.StringData["triple"]) == 1 || + A.StringData

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

yaxunl added inline comments. Comment at: clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp:1211-1212 +llvm::sort(Input, [](OffloadingImage &A, OffloadingImage &B) { + return A.StringData["triple"].compare(B.StringData["triple"]) == 1 || + A.StringData[

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

jhuber6 updated this revision to Diff 471994. jhuber6 added a comment. Ping and fix test. Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D136701/new/ https://reviews.llvm.org/D136701 Files: clang/test/Driver/linker-wrapper.c clang/tools/clang-li

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

jhuber6 updated this revision to Diff 470867. jhuber6 added a comment. Herald added a subscriber: mgrang. Adding a sort so the entires appear in a deterministic order. The sort is simply a lexigraphic comparison. Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

jhuber6 updated this revision to Diff 470859. jhuber6 added a comment. Make the default number of threads one, let users use `-Wl,--wrapper-jobs=N` to use parallelism. Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D136701/new/ https://reviews.llvm.

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

yaxunl added a comment. In general, I think parallelizing the linking workload for multiple GPU's in the linker wrapper is a useful feature. I am not sure whether the workload to be parallelized includes the LLVM passes and codegen, which is usually the bottleneck. Parallelizing this workload w

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

jhuber6 added a comment. In D136701#3883416 , @tra wrote: > In D136701#3883300 , @jhuber6 wrote: > >> However, as an opt-in feature it would be very helpful in some cases. > > I'm OK with the explicit opt-in. Mig

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

tra added a comment. In D136701#3883300 , @jhuber6 wrote: > However, as an opt-in feature it would be very helpful in some cases. I'm OK with the explicit opt-in. > Like consider someone creating a static library that supports every GPU > architecture

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

jhuber6 added a comment. In D136701#3883218 , @tra wrote: > I would argue that parallel compilation and linking may need to be disabled > by default. I believe similar patches were discussed in the past regarding > sub-compilations, but they are relevan

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

tra added a comment. @yaxunl - Found it: D69582 Repository: rG LLVM Github Monorepo CHANGES SINCE LAST ACTION https://reviews.llvm.org/D136701/new/ https://reviews.llvm.org/D136701 ___ cfe-commits mailing list c

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

tra added a comment. I would argue that parallel compilation and linking may need to be disabled by default. I believe similar patches were discussed in the past regarding sub-compilations, but they are relevant for parallel linking, too. Google search shows D52193

[PATCH] D136701: [LinkerWrapper] Perform device linking steps in parallel

jhuber6 created this revision. jhuber6 added reviewers: jdoerfert, tianshilei1992, tra, yaxunl, JonChesterfield, ronlieb. Herald added a project: All. jhuber6 requested review of this revision. Herald added a project: clang. Herald added a subscriber: cfe-commits. This patch changes the device li