[clang] [clang] Expand invalid PCM diagnostic (PR #69489)

2023-10-18 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari created 
https://github.com/llvm/llvm-project/pull/69489

Summary:

When a PCM file is loaded, it can go wrong in various ways. The current 
diagnostic only produces the name of the malformed PCM, not why it is 
malformed. Expand the diagnostic to display what went wrong!

There is only one call site for this diagnostic, and it already passes the 
error message:

https://github.com/llvm/llvm-project/blob/main/clang/lib/Serialization/ASTReader.cpp#L4763-L4764

Test Plan:

The modified LIT test.

>From 3fa795fae47d00ece00e9414a268610d3c4a6bf3 Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Fri, 22 Sep 2023 12:03:23 -0700
Subject: [PATCH] Expand invalid PCM diagnostic

Summary:

When a PCM file is loaded, it can go wrong in various ways. The current
diagnostic only produces the name of the malformed PCM, not why
it is malformed. Expand the diagnostic to display what went wrong!

There is only one call site for this diagnostic, and it
already passes the error message.

Test Plan:

The modified LIT test.
---
 clang/include/clang/Basic/DiagnosticSerializationKinds.td | 2 +-
 clang/test/Modules/explicit-build.cpp | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/clang/include/clang/Basic/DiagnosticSerializationKinds.td 
b/clang/include/clang/Basic/DiagnosticSerializationKinds.td
index a1ae23a62802104..1c049792d16eda4 100644
--- a/clang/include/clang/Basic/DiagnosticSerializationKinds.td
+++ b/clang/include/clang/Basic/DiagnosticSerializationKinds.td
@@ -62,7 +62,7 @@ def err_ast_file_out_of_date : Error<
   "%select{PCH|module|AST}0 file '%1' is out of date and "
   "needs to be rebuilt%select{|: %3}2">, DefaultFatal;
 def err_ast_file_invalid : Error<
-  "file '%1' is not a valid precompiled %select{PCH|module|AST}0 file">, 
DefaultFatal;
+  "file '%1' is not a valid precompiled %select{PCH|module|AST}0 file: '%2'">, 
DefaultFatal;
 def note_module_file_imported_by : Note<
   "imported by %select{|module '%2' in }1'%0'">;
 def err_module_file_not_module : Error<
diff --git a/clang/test/Modules/explicit-build.cpp 
b/clang/test/Modules/explicit-build.cpp
index 16eb604708c9d83..42444369dc92c5f 100644
--- a/clang/test/Modules/explicit-build.cpp
+++ b/clang/test/Modules/explicit-build.cpp
@@ -161,7 +161,7 @@
 // RUN:-fmodule-file=%t/not.pcm \
 // RUN:%s 2>&1 | FileCheck --check-prefix=CHECK-BAD-FILE %s
 //
-// CHECK-BAD-FILE: fatal error: file '{{.*}}not.pcm' is not a valid 
precompiled module file
+// CHECK-BAD-FILE: fatal error: file '{{.*}}not.pcm' is not a valid 
precompiled module file: 'file too small to contain AST file magic'
 
 // RUN: not %clang_cc1 -x c++ -std=c++11 -fmodules -fimplicit-module-maps 
-fmodules-cache-path=%t -Rmodule-build -fno-modules-error-recovery \
 // RUN:-fmodule-file=%t/nonexistent.pcm \

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [clang] Expand invalid PCM diagnostic (PR #69489)

2023-10-18 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/69489

>From 3fa795fae47d00ece00e9414a268610d3c4a6bf3 Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Fri, 22 Sep 2023 12:03:23 -0700
Subject: [PATCH 1/2] Expand invalid PCM diagnostic

Summary:

When a PCM file is loaded, it can go wrong in various ways. The current
diagnostic only produces the name of the malformed PCM, not why
it is malformed. Expand the diagnostic to display what went wrong!

There is only one call site for this diagnostic, and it
already passes the error message.

Test Plan:

The modified LIT test.
---
 clang/include/clang/Basic/DiagnosticSerializationKinds.td | 2 +-
 clang/test/Modules/explicit-build.cpp | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/clang/include/clang/Basic/DiagnosticSerializationKinds.td 
b/clang/include/clang/Basic/DiagnosticSerializationKinds.td
index a1ae23a62802104..1c049792d16eda4 100644
--- a/clang/include/clang/Basic/DiagnosticSerializationKinds.td
+++ b/clang/include/clang/Basic/DiagnosticSerializationKinds.td
@@ -62,7 +62,7 @@ def err_ast_file_out_of_date : Error<
   "%select{PCH|module|AST}0 file '%1' is out of date and "
   "needs to be rebuilt%select{|: %3}2">, DefaultFatal;
 def err_ast_file_invalid : Error<
-  "file '%1' is not a valid precompiled %select{PCH|module|AST}0 file">, 
DefaultFatal;
+  "file '%1' is not a valid precompiled %select{PCH|module|AST}0 file: '%2'">, 
DefaultFatal;
 def note_module_file_imported_by : Note<
   "imported by %select{|module '%2' in }1'%0'">;
 def err_module_file_not_module : Error<
diff --git a/clang/test/Modules/explicit-build.cpp 
b/clang/test/Modules/explicit-build.cpp
index 16eb604708c9d83..42444369dc92c5f 100644
--- a/clang/test/Modules/explicit-build.cpp
+++ b/clang/test/Modules/explicit-build.cpp
@@ -161,7 +161,7 @@
 // RUN:-fmodule-file=%t/not.pcm \
 // RUN:%s 2>&1 | FileCheck --check-prefix=CHECK-BAD-FILE %s
 //
-// CHECK-BAD-FILE: fatal error: file '{{.*}}not.pcm' is not a valid 
precompiled module file
+// CHECK-BAD-FILE: fatal error: file '{{.*}}not.pcm' is not a valid 
precompiled module file: 'file too small to contain AST file magic'
 
 // RUN: not %clang_cc1 -x c++ -std=c++11 -fmodules -fimplicit-module-maps 
-fmodules-cache-path=%t -Rmodule-build -fno-modules-error-recovery \
 // RUN:-fmodule-file=%t/nonexistent.pcm \

>From e2bb320ff4f64188c9b248b769efed51ab7bac0f Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Wed, 18 Oct 2023 14:36:56 -0700
Subject: [PATCH 2/2] Address PR Feedback

---
 clang/include/clang/Basic/DiagnosticSerializationKinds.td | 2 +-
 clang/test/Modules/explicit-build.cpp | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/clang/include/clang/Basic/DiagnosticSerializationKinds.td 
b/clang/include/clang/Basic/DiagnosticSerializationKinds.td
index 1c049792d16eda4..3cb2cd32cf6d09f 100644
--- a/clang/include/clang/Basic/DiagnosticSerializationKinds.td
+++ b/clang/include/clang/Basic/DiagnosticSerializationKinds.td
@@ -62,7 +62,7 @@ def err_ast_file_out_of_date : Error<
   "%select{PCH|module|AST}0 file '%1' is out of date and "
   "needs to be rebuilt%select{|: %3}2">, DefaultFatal;
 def err_ast_file_invalid : Error<
-  "file '%1' is not a valid precompiled %select{PCH|module|AST}0 file: '%2'">, 
DefaultFatal;
+  "file '%1' is not a valid precompiled %select{PCH|module|AST}0 file: %2">, 
DefaultFatal;
 def note_module_file_imported_by : Note<
   "imported by %select{|module '%2' in }1'%0'">;
 def err_module_file_not_module : Error<
diff --git a/clang/test/Modules/explicit-build.cpp 
b/clang/test/Modules/explicit-build.cpp
index 42444369dc92c5f..7fca0082ea28add 100644
--- a/clang/test/Modules/explicit-build.cpp
+++ b/clang/test/Modules/explicit-build.cpp
@@ -161,7 +161,7 @@
 // RUN:-fmodule-file=%t/not.pcm \
 // RUN:%s 2>&1 | FileCheck --check-prefix=CHECK-BAD-FILE %s
 //
-// CHECK-BAD-FILE: fatal error: file '{{.*}}not.pcm' is not a valid 
precompiled module file: 'file too small to contain AST file magic'
+// CHECK-BAD-FILE: fatal error: file '{{.*}}not.pcm' is not a valid 
precompiled module file: file too small to contain AST file magic
 
 // RUN: not %clang_cc1 -x c++ -std=c++11 -fmodules -fimplicit-module-maps 
-fmodules-cache-path=%t -Rmodule-build -fno-modules-error-recovery \
 // RUN:-fmodule-file=%t/nonexistent.pcm \

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Add option to dump IR to files instead of stderr (PR #66412)

2023-09-27 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/66412

>From da5da863d20cd8bef88066bba931c068042833cf Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Thu, 7 Sep 2023 12:34:15 -0700
Subject: [PATCH 1/3] Add option to dump IR to files intstead of stderr

This patch adds a flag to LLVM such that the output
generated by the `-print-(before|after|all)`
family of flags is written to files in a directory rather
than to stderr.

This new flag is `-ir-dump-directory` and is used to
specify where to write the files. No other flags are
added, it just modifies the behavior of the print flags.
---
 llvm/include/llvm/IR/PrintPasses.h|   4 +
 .../llvm/Passes/StandardInstrumentations.h|  20 ++-
 llvm/lib/IR/PrintPasses.cpp   |   9 +
 llvm/lib/Passes/StandardInstrumentations.cpp  | 167 +++---
 llvm/test/Other/dump-before-after.ll  |  57 ++
 5 files changed, 231 insertions(+), 26 deletions(-)
 create mode 100644 llvm/test/Other/dump-before-after.ll

diff --git a/llvm/include/llvm/IR/PrintPasses.h 
b/llvm/include/llvm/IR/PrintPasses.h
index 95b97e76c867cb2..c4baadfa3975531 100644
--- a/llvm/include/llvm/IR/PrintPasses.h
+++ b/llvm/include/llvm/IR/PrintPasses.h
@@ -55,6 +55,10 @@ bool forcePrintModuleIR();
 bool isPassInPrintList(StringRef PassName);
 bool isFilterPassesEmpty();
 
+// Returns a non-empty string if printing before/after passes is to be
+// dumped into files in the returned directory instead of written to stderr.
+std::string irDumpDirectory();
+
 // Returns true if we should print the function.
 bool isFunctionInPrintList(StringRef FunctionName);
 
diff --git a/llvm/include/llvm/Passes/StandardInstrumentations.h 
b/llvm/include/llvm/Passes/StandardInstrumentations.h
index 331130c6b22d990..fd512635356e5dd 100644
--- a/llvm/include/llvm/Passes/StandardInstrumentations.h
+++ b/llvm/include/llvm/Passes/StandardInstrumentations.h
@@ -46,6 +46,16 @@ class PrintIRInstrumentation {
   void registerCallbacks(PassInstrumentationCallbacks &PIC);
 
 private:
+  enum SuffixType {
+before,
+after,
+invalidated,
+  };
+
+  using PrintModuleDesc = std::tuple /* DumpFilename */>;
+
   void printBeforePass(StringRef PassID, Any IR);
   void printAfterPass(StringRef PassID, Any IR);
   void printAfterPassInvalidated(StringRef PassID);
@@ -55,11 +65,15 @@ class PrintIRInstrumentation {
   bool shouldPrintPassNumbers();
   bool shouldPrintAtPassNumber();
 
-  using PrintModuleDesc = std::tuple;
-
-  void pushModuleDesc(StringRef PassID, Any IR);
+  void pushModuleDesc(StringRef PassID, Any IR,
+  SmallString<128> DumpIRFilename);
   PrintModuleDesc popModuleDesc(StringRef PassID);
 
+  SmallString<128> fetchDumpFilename(StringRef PassId, Any IR);
+  StringRef getFileSuffix(SuffixType);
+
+  static constexpr std::array FileSuffixes = {"-before.ll", "-after.ll",
+  "-invalidated.ll"};
   PassInstrumentationCallbacks *PIC;
   /// Stack of Module description, enough to print the module after a given
   /// pass.
diff --git a/llvm/lib/IR/PrintPasses.cpp b/llvm/lib/IR/PrintPasses.cpp
index e2ef20bb81ba7d7..406af4a0a5e004e 100644
--- a/llvm/lib/IR/PrintPasses.cpp
+++ b/llvm/lib/IR/PrintPasses.cpp
@@ -103,6 +103,13 @@ static cl::list
 "options"),
cl::CommaSeparated, cl::Hidden);
 
+static cl::opt IRDumpDirectory(
+"ir-dump-directory",
+llvm::cl::desc("If specified, IR printed using the "
+   "-print-[before|after]{-all} options will be dumped into "
+   "files in this directory rather than written to stderr"),
+cl::init(""), cl::Hidden, cl::value_desc("filename"));
+
 /// This is a helper to determine whether to print IR before or
 /// after a pass.
 
@@ -139,6 +146,8 @@ std::vector llvm::printAfterPasses() {
   return std::vector(PrintAfter);
 }
 
+std::string llvm::irDumpDirectory() { return IRDumpDirectory; }
+
 bool llvm::forcePrintModuleIR() { return PrintModuleScope; }
 
 bool llvm::isPassInPrintList(StringRef PassName) {
diff --git a/llvm/lib/Passes/StandardInstrumentations.cpp 
b/llvm/lib/Passes/StandardInstrumentations.cpp
index 6244c0a5a949ba1..64bf306438d99d2 100644
--- a/llvm/lib/Passes/StandardInstrumentations.cpp
+++ b/llvm/lib/Passes/StandardInstrumentations.cpp
@@ -14,6 +14,7 @@
 
 #include "llvm/Passes/StandardInstrumentations.h"
 #include "llvm/ADT/Any.h"
+#include "llvm/ADT/Hashing.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Analysis/CallGraphSCCPass.h"
 #include "llvm/Analysis/LazyCallGraph.h"
@@ -33,6 +34,7 @@
 #include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/GraphWriter.h"
 #include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
 #include "llvm/Support/Program.h"
 #include "llvm/Support/Regex.h"
 #include "llvm/Support/Signals.h"
@@ -684,9 +686,64 @@ PrintIRInstrumentation::~PrintIRInstrumentation() {
   assert(ModuleDescStac

[clang-tools-extra] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-07 Thread Nuri Amari via cfe-commits

NuriAmari wrote:

> What about an option that just redirects the PrintIRInstrumentation output to 
> files instead of stderr? The files can be something like `$N-$modulename.ll` 
> where `$N` is an int that increments.

Yeah that's more or less what I'm working on now. My plan is:

- Add a flag that provides a directory to write files to
- If that flag is passed, write files of the structure 
`$N-$modulename-$passname-(before|after).ll` into that directory instead of 
writing to stderr
- If you pass both `-print-after=foo` and `-print-before=foo` the dump files 
for a particular run of Foo on a particular unit of IR should match except for 
the before / after suffix.

This should be relatively easy to implement in the old pass manager as well.

https://github.com/llvm/llvm-project/pull/65179
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-07 Thread Nuri Amari via cfe-commits

NuriAmari wrote:

> What about an option that just redirects the PrintIRInstrumentation output to 
> files instead of stderr? The files can be something like `$N-$modulename.ll` 
> where `$N` is an int that increments.

Yeah that's more or less what I'm working on now. My plan is:

- Add a flag that provides a directory to write files to
- If that flag is passed, write files of the structure 
`$N-$modulename-$passname-(before|after).ll` into that directory instead of 
writing to stderr
- If you pass both `-print-after=foo` and `-print-before=foo` the dump files 
for a particular run of Foo on a particular unit of IR should match except for 
the before / after suffix.

This should be relatively easy to implement in the old pass manager as well.

https://github.com/llvm/llvm-project/pull/65179
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-03 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/65179:

>From 5d395c85b84e5a554df4b092014d38123e666c6c Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Tue, 29 Aug 2023 10:10:57 -0700
Subject: [PATCH 1/2] Add flags to dump IR to a file before and after LLVM
 passes

Summary:

LLVM offers -print-after and -print-before flags that allow you
to print IR to stderr before and after any pass you want. This can
be useful for debugging LLVM optimization issue, but is far too
noisy for large builds.

This patch adds analogous options -dump-after and -dump-before that
dump the IR to appropriately named files. In addition, it also
introduces flags -dump-after-all, -dump-before-all and -ir-dump-directory
to control where the files are written to.

Test Plan:

Included LIT tests:
```
ninja check-llvm
```
---
 llvm/include/llvm/IR/PrintPasses.h|  21 +++
 .../llvm/Passes/StandardInstrumentations.h|  57 ++
 llvm/lib/IR/PrintPasses.cpp   |  54 ++
 llvm/lib/Passes/PassBuilder.cpp   |   3 +-
 llvm/lib/Passes/StandardInstrumentations.cpp  | 178 ++
 .../Other/dump-before-after-file-contents |  76 
 llvm/test/Other/dump-before-after-filenames   |  71 +++
 .../Other/dump-before-after-multiple-modules  |  36 
 llvm/test/Other/lit.local.cfg |   1 +
 9 files changed, 496 insertions(+), 1 deletion(-)
 create mode 100644 llvm/test/Other/dump-before-after-file-contents
 create mode 100644 llvm/test/Other/dump-before-after-filenames
 create mode 100644 llvm/test/Other/dump-before-after-multiple-modules
 create mode 100644 llvm/test/Other/lit.local.cfg

diff --git a/llvm/include/llvm/IR/PrintPasses.h 
b/llvm/include/llvm/IR/PrintPasses.h
index 95b97e76c867cb2..b2a9017521c6a91 100644
--- a/llvm/include/llvm/IR/PrintPasses.h
+++ b/llvm/include/llvm/IR/PrintPasses.h
@@ -48,6 +48,27 @@ bool shouldPrintAfterAll();
 std::vector printBeforePasses();
 std::vector printAfterPasses();
 
+// Returns true if dumping IR to a file before/after some pass is enabled
+// wether all passes or a specific pass.
+bool shouldDumpBeforeSomePass();
+bool shouldDumpAfterSomePass();
+
+// Returns true if we should dump IR to a file before/after a specific pass. 
The
+// argument should be the pass ID, e.g. "instcombine"
+bool shouldDumpBeforePass(StringRef PassID);
+bool shouldDumpAfterPass(StringRef PassID);
+
+// Returns true if we should dump IR to a file before/after all passes.
+bool shouldDumpBeforeAll();
+bool shouldDumpAfterAll();
+
+// The list of passes to dump IR to a file before/after, if we only want
+// to print before/after specific passes.
+std::vector dumpBeforePasses();
+std::vector dumpAfterPasses();
+
+StringRef irInstrumentationDumpDirectory();
+
 // Returns true if we should always print the entire module.
 bool forcePrintModuleIR();
 
diff --git a/llvm/include/llvm/Passes/StandardInstrumentations.h 
b/llvm/include/llvm/Passes/StandardInstrumentations.h
index 331130c6b22d990..4068a68aa956fd1 100644
--- a/llvm/include/llvm/Passes/StandardInstrumentations.h
+++ b/llvm/include/llvm/Passes/StandardInstrumentations.h
@@ -69,6 +69,62 @@ class PrintIRInstrumentation {
   unsigned CurrentPassNumber = 0;
 };
 
+class DumpIRInstrumentation {
+public:
+  void registerCallbacks(PassInstrumentationCallbacks &PIC);
+
+private:
+  void dumpBeforePass(StringRef PassID, Any IR);
+  void dumpAfterPass(StringRef PassID, Any IR);
+
+  bool shouldDumpBeforePass(StringRef PassID);
+  bool shouldDumpAfterPass(StringRef PassID);
+
+  PassInstrumentationCallbacks *PIC;
+
+  // The module currently being processed in the pipeline.
+  Module const *CurrentModule = nullptr;
+
+  void pushPass(StringRef PassID, Any IR);
+  void popPass(StringRef PassID);
+
+  SmallString<16> InstrumentationDumpDirectory;
+  StringRef fetchInstrumentationDumpDirectory();
+
+  SmallString<16> fetchCurrentInstrumentationDumpFile(StringRef Suffix);
+
+  // A table to store how many times a given pass has run at the current 
"nested
+  // level"
+  using PassRunsFrequencyTableT = DenseMap;
+  // A stack each frame of which (aside from the very first) represents a pass
+  // being run on some unit of IR. The larger, the stack grows, the smaller the
+  // unit of IR. For example, we would first push a module pass, then for each
+  // function pass in that module pass, we would push a frame and so on. This
+  // information is used to craft the output path for this logging.
+  //
+  // Each frame contains a map to track how many times a given subpass runs. 
For
+  // example, to keep track of how many times a function pass Foo runs within a
+  // module pass Bar. The first frame of the stack represents the module being
+  // processed rather than any particular pass. This is to create a frequency
+  // table to track module level pass run counts without having to special case
+  // that logic.
+  //
+  // When a change in the module being processed is detecte

[clang-tools-extra] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-03 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/65179:

>From 5d395c85b84e5a554df4b092014d38123e666c6c Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Tue, 29 Aug 2023 10:10:57 -0700
Subject: [PATCH 1/2] Add flags to dump IR to a file before and after LLVM
 passes

Summary:

LLVM offers -print-after and -print-before flags that allow you
to print IR to stderr before and after any pass you want. This can
be useful for debugging LLVM optimization issue, but is far too
noisy for large builds.

This patch adds analogous options -dump-after and -dump-before that
dump the IR to appropriately named files. In addition, it also
introduces flags -dump-after-all, -dump-before-all and -ir-dump-directory
to control where the files are written to.

Test Plan:

Included LIT tests:
```
ninja check-llvm
```
---
 llvm/include/llvm/IR/PrintPasses.h|  21 +++
 .../llvm/Passes/StandardInstrumentations.h|  57 ++
 llvm/lib/IR/PrintPasses.cpp   |  54 ++
 llvm/lib/Passes/PassBuilder.cpp   |   3 +-
 llvm/lib/Passes/StandardInstrumentations.cpp  | 178 ++
 .../Other/dump-before-after-file-contents |  76 
 llvm/test/Other/dump-before-after-filenames   |  71 +++
 .../Other/dump-before-after-multiple-modules  |  36 
 llvm/test/Other/lit.local.cfg |   1 +
 9 files changed, 496 insertions(+), 1 deletion(-)
 create mode 100644 llvm/test/Other/dump-before-after-file-contents
 create mode 100644 llvm/test/Other/dump-before-after-filenames
 create mode 100644 llvm/test/Other/dump-before-after-multiple-modules
 create mode 100644 llvm/test/Other/lit.local.cfg

diff --git a/llvm/include/llvm/IR/PrintPasses.h 
b/llvm/include/llvm/IR/PrintPasses.h
index 95b97e76c867cb2..b2a9017521c6a91 100644
--- a/llvm/include/llvm/IR/PrintPasses.h
+++ b/llvm/include/llvm/IR/PrintPasses.h
@@ -48,6 +48,27 @@ bool shouldPrintAfterAll();
 std::vector printBeforePasses();
 std::vector printAfterPasses();
 
+// Returns true if dumping IR to a file before/after some pass is enabled
+// wether all passes or a specific pass.
+bool shouldDumpBeforeSomePass();
+bool shouldDumpAfterSomePass();
+
+// Returns true if we should dump IR to a file before/after a specific pass. 
The
+// argument should be the pass ID, e.g. "instcombine"
+bool shouldDumpBeforePass(StringRef PassID);
+bool shouldDumpAfterPass(StringRef PassID);
+
+// Returns true if we should dump IR to a file before/after all passes.
+bool shouldDumpBeforeAll();
+bool shouldDumpAfterAll();
+
+// The list of passes to dump IR to a file before/after, if we only want
+// to print before/after specific passes.
+std::vector dumpBeforePasses();
+std::vector dumpAfterPasses();
+
+StringRef irInstrumentationDumpDirectory();
+
 // Returns true if we should always print the entire module.
 bool forcePrintModuleIR();
 
diff --git a/llvm/include/llvm/Passes/StandardInstrumentations.h 
b/llvm/include/llvm/Passes/StandardInstrumentations.h
index 331130c6b22d990..4068a68aa956fd1 100644
--- a/llvm/include/llvm/Passes/StandardInstrumentations.h
+++ b/llvm/include/llvm/Passes/StandardInstrumentations.h
@@ -69,6 +69,62 @@ class PrintIRInstrumentation {
   unsigned CurrentPassNumber = 0;
 };
 
+class DumpIRInstrumentation {
+public:
+  void registerCallbacks(PassInstrumentationCallbacks &PIC);
+
+private:
+  void dumpBeforePass(StringRef PassID, Any IR);
+  void dumpAfterPass(StringRef PassID, Any IR);
+
+  bool shouldDumpBeforePass(StringRef PassID);
+  bool shouldDumpAfterPass(StringRef PassID);
+
+  PassInstrumentationCallbacks *PIC;
+
+  // The module currently being processed in the pipeline.
+  Module const *CurrentModule = nullptr;
+
+  void pushPass(StringRef PassID, Any IR);
+  void popPass(StringRef PassID);
+
+  SmallString<16> InstrumentationDumpDirectory;
+  StringRef fetchInstrumentationDumpDirectory();
+
+  SmallString<16> fetchCurrentInstrumentationDumpFile(StringRef Suffix);
+
+  // A table to store how many times a given pass has run at the current 
"nested
+  // level"
+  using PassRunsFrequencyTableT = DenseMap;
+  // A stack each frame of which (aside from the very first) represents a pass
+  // being run on some unit of IR. The larger, the stack grows, the smaller the
+  // unit of IR. For example, we would first push a module pass, then for each
+  // function pass in that module pass, we would push a frame and so on. This
+  // information is used to craft the output path for this logging.
+  //
+  // Each frame contains a map to track how many times a given subpass runs. 
For
+  // example, to keep track of how many times a function pass Foo runs within a
+  // module pass Bar. The first frame of the stack represents the module being
+  // processed rather than any particular pass. This is to create a frequency
+  // table to track module level pass run counts without having to special case
+  // that logic.
+  //
+  // When a change in the module being processed is detecte

[clang-tools-extra] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-03 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/65179:

>From 5d395c85b84e5a554df4b092014d38123e666c6c Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Tue, 29 Aug 2023 10:10:57 -0700
Subject: [PATCH 1/3] Add flags to dump IR to a file before and after LLVM
 passes

Summary:

LLVM offers -print-after and -print-before flags that allow you
to print IR to stderr before and after any pass you want. This can
be useful for debugging LLVM optimization issue, but is far too
noisy for large builds.

This patch adds analogous options -dump-after and -dump-before that
dump the IR to appropriately named files. In addition, it also
introduces flags -dump-after-all, -dump-before-all and -ir-dump-directory
to control where the files are written to.

Test Plan:

Included LIT tests:
```
ninja check-llvm
```
---
 llvm/include/llvm/IR/PrintPasses.h|  21 +++
 .../llvm/Passes/StandardInstrumentations.h|  57 ++
 llvm/lib/IR/PrintPasses.cpp   |  54 ++
 llvm/lib/Passes/PassBuilder.cpp   |   3 +-
 llvm/lib/Passes/StandardInstrumentations.cpp  | 178 ++
 .../Other/dump-before-after-file-contents |  76 
 llvm/test/Other/dump-before-after-filenames   |  71 +++
 .../Other/dump-before-after-multiple-modules  |  36 
 llvm/test/Other/lit.local.cfg |   1 +
 9 files changed, 496 insertions(+), 1 deletion(-)
 create mode 100644 llvm/test/Other/dump-before-after-file-contents
 create mode 100644 llvm/test/Other/dump-before-after-filenames
 create mode 100644 llvm/test/Other/dump-before-after-multiple-modules
 create mode 100644 llvm/test/Other/lit.local.cfg

diff --git a/llvm/include/llvm/IR/PrintPasses.h 
b/llvm/include/llvm/IR/PrintPasses.h
index 95b97e76c867cb2..b2a9017521c6a91 100644
--- a/llvm/include/llvm/IR/PrintPasses.h
+++ b/llvm/include/llvm/IR/PrintPasses.h
@@ -48,6 +48,27 @@ bool shouldPrintAfterAll();
 std::vector printBeforePasses();
 std::vector printAfterPasses();
 
+// Returns true if dumping IR to a file before/after some pass is enabled
+// wether all passes or a specific pass.
+bool shouldDumpBeforeSomePass();
+bool shouldDumpAfterSomePass();
+
+// Returns true if we should dump IR to a file before/after a specific pass. 
The
+// argument should be the pass ID, e.g. "instcombine"
+bool shouldDumpBeforePass(StringRef PassID);
+bool shouldDumpAfterPass(StringRef PassID);
+
+// Returns true if we should dump IR to a file before/after all passes.
+bool shouldDumpBeforeAll();
+bool shouldDumpAfterAll();
+
+// The list of passes to dump IR to a file before/after, if we only want
+// to print before/after specific passes.
+std::vector dumpBeforePasses();
+std::vector dumpAfterPasses();
+
+StringRef irInstrumentationDumpDirectory();
+
 // Returns true if we should always print the entire module.
 bool forcePrintModuleIR();
 
diff --git a/llvm/include/llvm/Passes/StandardInstrumentations.h 
b/llvm/include/llvm/Passes/StandardInstrumentations.h
index 331130c6b22d990..4068a68aa956fd1 100644
--- a/llvm/include/llvm/Passes/StandardInstrumentations.h
+++ b/llvm/include/llvm/Passes/StandardInstrumentations.h
@@ -69,6 +69,62 @@ class PrintIRInstrumentation {
   unsigned CurrentPassNumber = 0;
 };
 
+class DumpIRInstrumentation {
+public:
+  void registerCallbacks(PassInstrumentationCallbacks &PIC);
+
+private:
+  void dumpBeforePass(StringRef PassID, Any IR);
+  void dumpAfterPass(StringRef PassID, Any IR);
+
+  bool shouldDumpBeforePass(StringRef PassID);
+  bool shouldDumpAfterPass(StringRef PassID);
+
+  PassInstrumentationCallbacks *PIC;
+
+  // The module currently being processed in the pipeline.
+  Module const *CurrentModule = nullptr;
+
+  void pushPass(StringRef PassID, Any IR);
+  void popPass(StringRef PassID);
+
+  SmallString<16> InstrumentationDumpDirectory;
+  StringRef fetchInstrumentationDumpDirectory();
+
+  SmallString<16> fetchCurrentInstrumentationDumpFile(StringRef Suffix);
+
+  // A table to store how many times a given pass has run at the current 
"nested
+  // level"
+  using PassRunsFrequencyTableT = DenseMap;
+  // A stack each frame of which (aside from the very first) represents a pass
+  // being run on some unit of IR. The larger, the stack grows, the smaller the
+  // unit of IR. For example, we would first push a module pass, then for each
+  // function pass in that module pass, we would push a frame and so on. This
+  // information is used to craft the output path for this logging.
+  //
+  // Each frame contains a map to track how many times a given subpass runs. 
For
+  // example, to keep track of how many times a function pass Foo runs within a
+  // module pass Bar. The first frame of the stack represents the module being
+  // processed rather than any particular pass. This is to create a frequency
+  // table to track module level pass run counts without having to special case
+  // that logic.
+  //
+  // When a change in the module being processed is detecte

[clang] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-03 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/65179:

>From 5d395c85b84e5a554df4b092014d38123e666c6c Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Tue, 29 Aug 2023 10:10:57 -0700
Subject: [PATCH 1/3] Add flags to dump IR to a file before and after LLVM
 passes

Summary:

LLVM offers -print-after and -print-before flags that allow you
to print IR to stderr before and after any pass you want. This can
be useful for debugging LLVM optimization issue, but is far too
noisy for large builds.

This patch adds analogous options -dump-after and -dump-before that
dump the IR to appropriately named files. In addition, it also
introduces flags -dump-after-all, -dump-before-all and -ir-dump-directory
to control where the files are written to.

Test Plan:

Included LIT tests:
```
ninja check-llvm
```
---
 llvm/include/llvm/IR/PrintPasses.h|  21 +++
 .../llvm/Passes/StandardInstrumentations.h|  57 ++
 llvm/lib/IR/PrintPasses.cpp   |  54 ++
 llvm/lib/Passes/PassBuilder.cpp   |   3 +-
 llvm/lib/Passes/StandardInstrumentations.cpp  | 178 ++
 .../Other/dump-before-after-file-contents |  76 
 llvm/test/Other/dump-before-after-filenames   |  71 +++
 .../Other/dump-before-after-multiple-modules  |  36 
 llvm/test/Other/lit.local.cfg |   1 +
 9 files changed, 496 insertions(+), 1 deletion(-)
 create mode 100644 llvm/test/Other/dump-before-after-file-contents
 create mode 100644 llvm/test/Other/dump-before-after-filenames
 create mode 100644 llvm/test/Other/dump-before-after-multiple-modules
 create mode 100644 llvm/test/Other/lit.local.cfg

diff --git a/llvm/include/llvm/IR/PrintPasses.h 
b/llvm/include/llvm/IR/PrintPasses.h
index 95b97e76c867cb2..b2a9017521c6a91 100644
--- a/llvm/include/llvm/IR/PrintPasses.h
+++ b/llvm/include/llvm/IR/PrintPasses.h
@@ -48,6 +48,27 @@ bool shouldPrintAfterAll();
 std::vector printBeforePasses();
 std::vector printAfterPasses();
 
+// Returns true if dumping IR to a file before/after some pass is enabled
+// wether all passes or a specific pass.
+bool shouldDumpBeforeSomePass();
+bool shouldDumpAfterSomePass();
+
+// Returns true if we should dump IR to a file before/after a specific pass. 
The
+// argument should be the pass ID, e.g. "instcombine"
+bool shouldDumpBeforePass(StringRef PassID);
+bool shouldDumpAfterPass(StringRef PassID);
+
+// Returns true if we should dump IR to a file before/after all passes.
+bool shouldDumpBeforeAll();
+bool shouldDumpAfterAll();
+
+// The list of passes to dump IR to a file before/after, if we only want
+// to print before/after specific passes.
+std::vector dumpBeforePasses();
+std::vector dumpAfterPasses();
+
+StringRef irInstrumentationDumpDirectory();
+
 // Returns true if we should always print the entire module.
 bool forcePrintModuleIR();
 
diff --git a/llvm/include/llvm/Passes/StandardInstrumentations.h 
b/llvm/include/llvm/Passes/StandardInstrumentations.h
index 331130c6b22d990..4068a68aa956fd1 100644
--- a/llvm/include/llvm/Passes/StandardInstrumentations.h
+++ b/llvm/include/llvm/Passes/StandardInstrumentations.h
@@ -69,6 +69,62 @@ class PrintIRInstrumentation {
   unsigned CurrentPassNumber = 0;
 };
 
+class DumpIRInstrumentation {
+public:
+  void registerCallbacks(PassInstrumentationCallbacks &PIC);
+
+private:
+  void dumpBeforePass(StringRef PassID, Any IR);
+  void dumpAfterPass(StringRef PassID, Any IR);
+
+  bool shouldDumpBeforePass(StringRef PassID);
+  bool shouldDumpAfterPass(StringRef PassID);
+
+  PassInstrumentationCallbacks *PIC;
+
+  // The module currently being processed in the pipeline.
+  Module const *CurrentModule = nullptr;
+
+  void pushPass(StringRef PassID, Any IR);
+  void popPass(StringRef PassID);
+
+  SmallString<16> InstrumentationDumpDirectory;
+  StringRef fetchInstrumentationDumpDirectory();
+
+  SmallString<16> fetchCurrentInstrumentationDumpFile(StringRef Suffix);
+
+  // A table to store how many times a given pass has run at the current 
"nested
+  // level"
+  using PassRunsFrequencyTableT = DenseMap;
+  // A stack each frame of which (aside from the very first) represents a pass
+  // being run on some unit of IR. The larger, the stack grows, the smaller the
+  // unit of IR. For example, we would first push a module pass, then for each
+  // function pass in that module pass, we would push a frame and so on. This
+  // information is used to craft the output path for this logging.
+  //
+  // Each frame contains a map to track how many times a given subpass runs. 
For
+  // example, to keep track of how many times a function pass Foo runs within a
+  // module pass Bar. The first frame of the stack represents the module being
+  // processed rather than any particular pass. This is to create a frequency
+  // table to track module level pass run counts without having to special case
+  // that logic.
+  //
+  // When a change in the module being processed is detecte

[clang] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-06 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/65179:

>From 5d395c85b84e5a554df4b092014d38123e666c6c Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Tue, 29 Aug 2023 10:10:57 -0700
Subject: [PATCH 1/4] Add flags to dump IR to a file before and after LLVM
 passes

Summary:

LLVM offers -print-after and -print-before flags that allow you
to print IR to stderr before and after any pass you want. This can
be useful for debugging LLVM optimization issue, but is far too
noisy for large builds.

This patch adds analogous options -dump-after and -dump-before that
dump the IR to appropriately named files. In addition, it also
introduces flags -dump-after-all, -dump-before-all and -ir-dump-directory
to control where the files are written to.

Test Plan:

Included LIT tests:
```
ninja check-llvm
```
---
 llvm/include/llvm/IR/PrintPasses.h|  21 +++
 .../llvm/Passes/StandardInstrumentations.h|  57 ++
 llvm/lib/IR/PrintPasses.cpp   |  54 ++
 llvm/lib/Passes/PassBuilder.cpp   |   3 +-
 llvm/lib/Passes/StandardInstrumentations.cpp  | 178 ++
 .../Other/dump-before-after-file-contents |  76 
 llvm/test/Other/dump-before-after-filenames   |  71 +++
 .../Other/dump-before-after-multiple-modules  |  36 
 llvm/test/Other/lit.local.cfg |   1 +
 9 files changed, 496 insertions(+), 1 deletion(-)
 create mode 100644 llvm/test/Other/dump-before-after-file-contents
 create mode 100644 llvm/test/Other/dump-before-after-filenames
 create mode 100644 llvm/test/Other/dump-before-after-multiple-modules
 create mode 100644 llvm/test/Other/lit.local.cfg

diff --git a/llvm/include/llvm/IR/PrintPasses.h 
b/llvm/include/llvm/IR/PrintPasses.h
index 95b97e76c867cb..b2a9017521c6a9 100644
--- a/llvm/include/llvm/IR/PrintPasses.h
+++ b/llvm/include/llvm/IR/PrintPasses.h
@@ -48,6 +48,27 @@ bool shouldPrintAfterAll();
 std::vector printBeforePasses();
 std::vector printAfterPasses();
 
+// Returns true if dumping IR to a file before/after some pass is enabled
+// wether all passes or a specific pass.
+bool shouldDumpBeforeSomePass();
+bool shouldDumpAfterSomePass();
+
+// Returns true if we should dump IR to a file before/after a specific pass. 
The
+// argument should be the pass ID, e.g. "instcombine"
+bool shouldDumpBeforePass(StringRef PassID);
+bool shouldDumpAfterPass(StringRef PassID);
+
+// Returns true if we should dump IR to a file before/after all passes.
+bool shouldDumpBeforeAll();
+bool shouldDumpAfterAll();
+
+// The list of passes to dump IR to a file before/after, if we only want
+// to print before/after specific passes.
+std::vector dumpBeforePasses();
+std::vector dumpAfterPasses();
+
+StringRef irInstrumentationDumpDirectory();
+
 // Returns true if we should always print the entire module.
 bool forcePrintModuleIR();
 
diff --git a/llvm/include/llvm/Passes/StandardInstrumentations.h 
b/llvm/include/llvm/Passes/StandardInstrumentations.h
index 331130c6b22d99..4068a68aa956fd 100644
--- a/llvm/include/llvm/Passes/StandardInstrumentations.h
+++ b/llvm/include/llvm/Passes/StandardInstrumentations.h
@@ -69,6 +69,62 @@ class PrintIRInstrumentation {
   unsigned CurrentPassNumber = 0;
 };
 
+class DumpIRInstrumentation {
+public:
+  void registerCallbacks(PassInstrumentationCallbacks &PIC);
+
+private:
+  void dumpBeforePass(StringRef PassID, Any IR);
+  void dumpAfterPass(StringRef PassID, Any IR);
+
+  bool shouldDumpBeforePass(StringRef PassID);
+  bool shouldDumpAfterPass(StringRef PassID);
+
+  PassInstrumentationCallbacks *PIC;
+
+  // The module currently being processed in the pipeline.
+  Module const *CurrentModule = nullptr;
+
+  void pushPass(StringRef PassID, Any IR);
+  void popPass(StringRef PassID);
+
+  SmallString<16> InstrumentationDumpDirectory;
+  StringRef fetchInstrumentationDumpDirectory();
+
+  SmallString<16> fetchCurrentInstrumentationDumpFile(StringRef Suffix);
+
+  // A table to store how many times a given pass has run at the current 
"nested
+  // level"
+  using PassRunsFrequencyTableT = DenseMap;
+  // A stack each frame of which (aside from the very first) represents a pass
+  // being run on some unit of IR. The larger, the stack grows, the smaller the
+  // unit of IR. For example, we would first push a module pass, then for each
+  // function pass in that module pass, we would push a frame and so on. This
+  // information is used to craft the output path for this logging.
+  //
+  // Each frame contains a map to track how many times a given subpass runs. 
For
+  // example, to keep track of how many times a function pass Foo runs within a
+  // module pass Bar. The first frame of the stack represents the module being
+  // processed rather than any particular pass. This is to create a frequency
+  // table to track module level pass run counts without having to special case
+  // that logic.
+  //
+  // When a change in the module being processed is detected, t

[clang-tools-extra] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-06 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/65179:

>From 5d395c85b84e5a554df4b092014d38123e666c6c Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Tue, 29 Aug 2023 10:10:57 -0700
Subject: [PATCH 1/4] Add flags to dump IR to a file before and after LLVM
 passes

Summary:

LLVM offers -print-after and -print-before flags that allow you
to print IR to stderr before and after any pass you want. This can
be useful for debugging LLVM optimization issue, but is far too
noisy for large builds.

This patch adds analogous options -dump-after and -dump-before that
dump the IR to appropriately named files. In addition, it also
introduces flags -dump-after-all, -dump-before-all and -ir-dump-directory
to control where the files are written to.

Test Plan:

Included LIT tests:
```
ninja check-llvm
```
---
 llvm/include/llvm/IR/PrintPasses.h|  21 +++
 .../llvm/Passes/StandardInstrumentations.h|  57 ++
 llvm/lib/IR/PrintPasses.cpp   |  54 ++
 llvm/lib/Passes/PassBuilder.cpp   |   3 +-
 llvm/lib/Passes/StandardInstrumentations.cpp  | 178 ++
 .../Other/dump-before-after-file-contents |  76 
 llvm/test/Other/dump-before-after-filenames   |  71 +++
 .../Other/dump-before-after-multiple-modules  |  36 
 llvm/test/Other/lit.local.cfg |   1 +
 9 files changed, 496 insertions(+), 1 deletion(-)
 create mode 100644 llvm/test/Other/dump-before-after-file-contents
 create mode 100644 llvm/test/Other/dump-before-after-filenames
 create mode 100644 llvm/test/Other/dump-before-after-multiple-modules
 create mode 100644 llvm/test/Other/lit.local.cfg

diff --git a/llvm/include/llvm/IR/PrintPasses.h 
b/llvm/include/llvm/IR/PrintPasses.h
index 95b97e76c867cb..b2a9017521c6a9 100644
--- a/llvm/include/llvm/IR/PrintPasses.h
+++ b/llvm/include/llvm/IR/PrintPasses.h
@@ -48,6 +48,27 @@ bool shouldPrintAfterAll();
 std::vector printBeforePasses();
 std::vector printAfterPasses();
 
+// Returns true if dumping IR to a file before/after some pass is enabled
+// wether all passes or a specific pass.
+bool shouldDumpBeforeSomePass();
+bool shouldDumpAfterSomePass();
+
+// Returns true if we should dump IR to a file before/after a specific pass. 
The
+// argument should be the pass ID, e.g. "instcombine"
+bool shouldDumpBeforePass(StringRef PassID);
+bool shouldDumpAfterPass(StringRef PassID);
+
+// Returns true if we should dump IR to a file before/after all passes.
+bool shouldDumpBeforeAll();
+bool shouldDumpAfterAll();
+
+// The list of passes to dump IR to a file before/after, if we only want
+// to print before/after specific passes.
+std::vector dumpBeforePasses();
+std::vector dumpAfterPasses();
+
+StringRef irInstrumentationDumpDirectory();
+
 // Returns true if we should always print the entire module.
 bool forcePrintModuleIR();
 
diff --git a/llvm/include/llvm/Passes/StandardInstrumentations.h 
b/llvm/include/llvm/Passes/StandardInstrumentations.h
index 331130c6b22d99..4068a68aa956fd 100644
--- a/llvm/include/llvm/Passes/StandardInstrumentations.h
+++ b/llvm/include/llvm/Passes/StandardInstrumentations.h
@@ -69,6 +69,62 @@ class PrintIRInstrumentation {
   unsigned CurrentPassNumber = 0;
 };
 
+class DumpIRInstrumentation {
+public:
+  void registerCallbacks(PassInstrumentationCallbacks &PIC);
+
+private:
+  void dumpBeforePass(StringRef PassID, Any IR);
+  void dumpAfterPass(StringRef PassID, Any IR);
+
+  bool shouldDumpBeforePass(StringRef PassID);
+  bool shouldDumpAfterPass(StringRef PassID);
+
+  PassInstrumentationCallbacks *PIC;
+
+  // The module currently being processed in the pipeline.
+  Module const *CurrentModule = nullptr;
+
+  void pushPass(StringRef PassID, Any IR);
+  void popPass(StringRef PassID);
+
+  SmallString<16> InstrumentationDumpDirectory;
+  StringRef fetchInstrumentationDumpDirectory();
+
+  SmallString<16> fetchCurrentInstrumentationDumpFile(StringRef Suffix);
+
+  // A table to store how many times a given pass has run at the current 
"nested
+  // level"
+  using PassRunsFrequencyTableT = DenseMap;
+  // A stack each frame of which (aside from the very first) represents a pass
+  // being run on some unit of IR. The larger, the stack grows, the smaller the
+  // unit of IR. For example, we would first push a module pass, then for each
+  // function pass in that module pass, we would push a frame and so on. This
+  // information is used to craft the output path for this logging.
+  //
+  // Each frame contains a map to track how many times a given subpass runs. 
For
+  // example, to keep track of how many times a function pass Foo runs within a
+  // module pass Bar. The first frame of the stack represents the module being
+  // processed rather than any particular pass. This is to create a frequency
+  // table to track module level pass run counts without having to special case
+  // that logic.
+  //
+  // When a change in the module being processed is detected, t

[clang] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-06 Thread Nuri Amari via cfe-commits


@@ -0,0 +1,71 @@
+; RUN: mkdir -p %t/logs
+; RUN: rm -rf %t/logs
+
+; Basic dump before and after a single module pass
+; RUN: opt %s -disable-output -passes='no-op-module' -ir-dump-directory 
%t/logs -dump-after=no-op-module -dump-before=no-op-module
+; RUN: find %t/logs -type f -print | FileCheck %s --check-prefix=SINGLE-RUN 
-DSOURCE_FILE_NAME=%s
+; SINGLE-RUN-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-before.ll
+; SINGLE-RUN-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-after.ll
+; RUN: rm -rf %t/logs
+
+; Dump before and after multiple runs of the same module pass
+; RUN: opt %s -disable-output -passes='no-op-module,no-op-module,no-op-module' 
-ir-dump-directory %t/logs -dump-after=no-op-module -dump-before=no-op-module
+; RUN: find %t/logs -type f -print | FileCheck %s --check-prefix=MULTIPLE-RUNS 
-DSOURCE_FILE_NAME=%s
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-before.ll
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-after.ll
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/1.NoOpModulePass.1-before.ll
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/1.NoOpModulePass.1-after.ll
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/2.NoOpModulePass.2-before.ll
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/2.NoOpModulePass.2-after.ll
+; RUN: rm -rf %t/logs
+
+; Dump before and after multiple passes, some nested
+; RUN: opt %s -disable-output 
-passes='no-op-module,function(no-op-function),function(loop(no-op-loop)),no-op-module'
 -ir-dump-directory %t/logs -dump-after=no-op-module,no-op-function,no-op-loop 
-dump-before=no-op-module,no-op-function,no-op-loop
+; RUN: find %t/logs -type f -print | FileCheck %s 
--check-prefix=MULTIPLE-NESTED-RUNS -DSOURCE_FILE_NAME=%s
+; MULTIPLE-NESTED-RUNS-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-before.ll
+; MULTIPLE-NESTED-RUNS-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-after.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/1.ModuleToFunctionPassAdaptor.0/0.PassManager.0/0.NoOpFunctionPass.0-before.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/1.ModuleToFunctionPassAdaptor.0/0.PassManager.0/0.NoOpFunctionPass.0-after.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/1.ModuleToFunctionPassAdaptor.0/1.PassManager.1/0.NoOpFunctionPass.0-before.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/1.ModuleToFunctionPassAdaptor.0/1.PassManager.1/0.NoOpFunctionPass.0-after.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/2.ModuleToFunctionPassAdaptor.1/1.PassManager.1/0.FunctionToLoopPassAdaptor.0/1.PassManager, 
llvm::LoopStandardAnalysisResults&, 
llvm::LPMUpdater&>.0/0.NoOpLoopPass.0-before.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/2.ModuleToFunctionPassAdaptor.1/1.PassManager.1/0.FunctionToLoopPassAdaptor.0/1.PassManager, 
llvm::LoopStandardAnalysisResults&, 
llvm::LPMUpdater&>.0/0.NoOpLoopPass.0-after.ll

NuriAmari wrote:

I made an attempt at this, the trouble is the new PM instrumentation 
infrastructure only provides callbacks with the PassID, which is this verbose 
sometimes. The existing Class name to pass name mapping does not contain 
entries for all passes (ex. adaptors and PMs). 

I think more consistent names are doable, but perhaps something for another 
patch.

https://github.com/llvm/llvm-project/pull/65179
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang-tools-extra] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-06 Thread Nuri Amari via cfe-commits


@@ -0,0 +1,71 @@
+; RUN: mkdir -p %t/logs
+; RUN: rm -rf %t/logs
+
+; Basic dump before and after a single module pass
+; RUN: opt %s -disable-output -passes='no-op-module' -ir-dump-directory 
%t/logs -dump-after=no-op-module -dump-before=no-op-module
+; RUN: find %t/logs -type f -print | FileCheck %s --check-prefix=SINGLE-RUN 
-DSOURCE_FILE_NAME=%s
+; SINGLE-RUN-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-before.ll
+; SINGLE-RUN-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-after.ll
+; RUN: rm -rf %t/logs
+
+; Dump before and after multiple runs of the same module pass
+; RUN: opt %s -disable-output -passes='no-op-module,no-op-module,no-op-module' 
-ir-dump-directory %t/logs -dump-after=no-op-module -dump-before=no-op-module
+; RUN: find %t/logs -type f -print | FileCheck %s --check-prefix=MULTIPLE-RUNS 
-DSOURCE_FILE_NAME=%s
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-before.ll
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-after.ll
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/1.NoOpModulePass.1-before.ll
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/1.NoOpModulePass.1-after.ll
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/2.NoOpModulePass.2-before.ll
+; MULTIPLE-RUNS-DAG: [[SOURCE_FILE_NAME]]/2.NoOpModulePass.2-after.ll
+; RUN: rm -rf %t/logs
+
+; Dump before and after multiple passes, some nested
+; RUN: opt %s -disable-output 
-passes='no-op-module,function(no-op-function),function(loop(no-op-loop)),no-op-module'
 -ir-dump-directory %t/logs -dump-after=no-op-module,no-op-function,no-op-loop 
-dump-before=no-op-module,no-op-function,no-op-loop
+; RUN: find %t/logs -type f -print | FileCheck %s 
--check-prefix=MULTIPLE-NESTED-RUNS -DSOURCE_FILE_NAME=%s
+; MULTIPLE-NESTED-RUNS-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-before.ll
+; MULTIPLE-NESTED-RUNS-DAG: [[SOURCE_FILE_NAME]]/0.NoOpModulePass.0-after.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/1.ModuleToFunctionPassAdaptor.0/0.PassManager.0/0.NoOpFunctionPass.0-before.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/1.ModuleToFunctionPassAdaptor.0/0.PassManager.0/0.NoOpFunctionPass.0-after.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/1.ModuleToFunctionPassAdaptor.0/1.PassManager.1/0.NoOpFunctionPass.0-before.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/1.ModuleToFunctionPassAdaptor.0/1.PassManager.1/0.NoOpFunctionPass.0-after.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/2.ModuleToFunctionPassAdaptor.1/1.PassManager.1/0.FunctionToLoopPassAdaptor.0/1.PassManager, 
llvm::LoopStandardAnalysisResults&, 
llvm::LPMUpdater&>.0/0.NoOpLoopPass.0-before.ll
+; MULTIPLE-NESTED-RUNS-DAG: 
[[SOURCE_FILE_NAME]]/2.ModuleToFunctionPassAdaptor.1/1.PassManager.1/0.FunctionToLoopPassAdaptor.0/1.PassManager, 
llvm::LoopStandardAnalysisResults&, 
llvm::LPMUpdater&>.0/0.NoOpLoopPass.0-after.ll

NuriAmari wrote:

I made an attempt at this, the trouble is the new PM instrumentation 
infrastructure only provides callbacks with the PassID, which is this verbose 
sometimes. The existing Class name to pass name mapping does not contain 
entries for all passes (ex. adaptors and PMs). 

I think more consistent names are doable, but perhaps something for another 
patch.

https://github.com/llvm/llvm-project/pull/65179
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-06 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari review_requested 
https://github.com/llvm/llvm-project/pull/65179
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-06 Thread Nuri Amari via cfe-commits

NuriAmari wrote:

I've tried this on a real application, and the output paths quickly get 
extremely long. So long that we fail to create the directory structure. Perhaps 
the nesting idea isn't the right choice after all.

https://github.com/llvm/llvm-project/pull/65179
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang-tools-extra] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-06 Thread Nuri Amari via cfe-commits

NuriAmari wrote:

I've tried this on a real application, and the output paths quickly get 
extremely long. So long that we fail to create the directory structure. Perhaps 
the nesting idea isn't the right choice after all.

https://github.com/llvm/llvm-project/pull/65179
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-07 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari closed 
https://github.com/llvm/llvm-project/pull/65179
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang-tools-extra] Add flags to dump IR to a file before and after LLVM passes (PR #65179)

2023-09-07 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari closed 
https://github.com/llvm/llvm-project/pull/65179
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] Run ObjCContractPass in Default Codegen Pipeline (PR #92331)

2024-05-28 Thread Nuri Amari via cfe-commits


@@ -31,6 +31,10 @@
 ; CHECK-NEXT:   AArch64 Stack Tagging
 ; CHECK-NEXT:   SME ABI Pass
 ; CHECK-NEXT:   Exception handling preparation
+; CHECK-NEXT:   Dominator Tree Construction
+; CHECK-NEXT:   Basic Alias Analysis (stateless AA impl)
+; CHECK-NEXT:   Function Alias Analysis Results

NuriAmari wrote:

I suspected this might be an issue. Is it possible to "lazily" request analysis 
results based on some logic in the pass itself? If not, maybe we can't enable 
this pass by default after all.

https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] [LTO] enable `ObjCARCContractPass` only on optimized build (PR #101114)

2024-07-29 Thread Nuri Amari via cfe-commits


@@ -588,12 +588,6 @@ bool EmitAssemblyHelper::AddEmitPasses(legacy::PassManager 
&CodeGenPasses,
   // this also adds codegenerator level optimization passes.
   CodeGenFileType CGFT = getCodeGenFileType(Action);
 
-  // Add ObjC ARC final-cleanup optimizations. This is done as part of the

NuriAmari wrote:

@cachemeifyoucan As far as I can tell, it's already the case that for non-LTO 
builds, this pass is only added for optimization level > 0. Is that ok to do 
for all builds?

I checked and directly providing clang with IR that contains 
`llvm.objc.clang.arc.use` and compiling it with `-O0` fails instruction 
selection as a result of not running this pass, so I'm assuming these 
intrinsics are only introduced at `-O1` or higher.

https://github.com/llvm/llvm-project/pull/101114
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] [LTO] enable `ObjCARCContractPass` only on optimized build (PR #101114)

2024-07-30 Thread Nuri Amari via cfe-commits

NuriAmari wrote:

This pass is added in numerous places: 
https://github.com/search?q=repo%3Allvm%2Fllvm-project%20createObjCARCContractPass&type=code,
 and we were hoping to consolidate all the places and instead schedule it by 
default, or behind some kind of predicate. We haven’t been able to find a 
predicate that satisfies all the use cases, there always seems to be a case 
where the required analysis passes are scheduled when not necessarily always 
required.

In practice, I think this pass is already scheduled by default, or for 
optimized builds in almost all configurations. Auditing the places it’s added, 
the legacy LTO backend adds it unconditionally: 
https://github.com/llvm/llvm-project/blob/8364a6e108faea1a6f05f7bb53720a6fc101fed2/llvm/lib/LTO/ThinLTOCodeGenerator.cpp#L340
https://github.com/llvm/llvm-project/blob/f70f1228035c9610de38e0e376afdacb647c4ad9/llvm/lib/LTO/LTOCodeGenerator.cpp#L140
 LLD for MachO injects it into the new LTO backend via a hook: 
https://github.com/llvm/llvm-project/blob/f70f1228035c9610de38e0e376afdacb647c4ad9/lld/MachO/LTO.cpp#L52
 Clang adds it for non-LTO compiles here: 
https://github.com/llvm/llvm-project/blob/f70f1228035c9610de38e0e376afdacb647c4ad9/clang/lib/CodeGen/BackendUtil.cpp#L595

Granted that leaves LTO builds that are using the new LTO backend (except LLD 
for MachO), and distributed thin-LTO builds which currently don’t schedule it. 
Ultimately, our goal here is to schedule the pass for distributed thin-LTO 
compiles as the new test demonstrates.

We could go back to a change like: 
https://github.com/llvm/llvm-project/pull/92331/commits/66ddf609c0e77867ec48c17136fb80d1e482041d

That would schedule the pass unconditionally for distributed thin-LTO compiles, 
on top of those configurations it already runs for. The analysis passes would 
still be run in some cases that are not required, but I don’t see how to avoid 
that. If we prefer that change, over the current, we could go that direction 
instead.

https://github.com/llvm/llvm-project/pull/101114
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-15 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari created 
https://github.com/llvm/llvm-project/pull/92331

Prior to this patch, when using -fthinlto-index= the ObjCARCContractPass isn't 
run prior to CodeGen, and instruction selection fails on IR containing arc 
intrinstics.

The pass would normally be added here: 
https://github.com/llvm/llvm-project/blob/c00e012bcf5da384a3e7339dc2e046779b339063/clang/lib/CodeGen/BackendUtil.cpp#L1352
 Since because we are using an index file, we return early above after 
`runThinLTOBackend`:
https://github.com/llvm/llvm-project/blob/c00e012bcf5da384a3e7339dc2e046779b339063/clang/lib/CodeGen/BackendUtil.cpp#L1333-L1339


>From 66ddf609c0e77867ec48c17136fb80d1e482041d Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Wed, 15 May 2024 16:33:03 -0700
Subject: [PATCH] Run ObjCContractPass in Distributed Thin-LTO Pipeline

Prior to this patch, when using -fthinlto-index= the
ObjCARCContractPass isn't run prior to CodeGen, and
instruction selection fails on IR containing
arc intrinstics.
---
 clang/lib/CodeGen/BackendUtil.cpp |  3 +++
 .../thinlto-distributed-objc-contract-pass.ll | 19 +++
 2 files changed, 22 insertions(+)
 create mode 100644 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 90985c08fe7f8..03dd1df281d80 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,6 +1289,9 @@ static void runThinLTOBackend(
 };
 break;
   default:
+Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
+  Pm.add(createObjCARCContractPass());
+};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
new file mode 100644
index 0..7d0247555b5c8
--- /dev/null
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -0,0 +1,19 @@
+; RUN: opt -thinlto-bc -o %t.o %s
+
+; RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
+; RUN:   -o %t2.index \
+; RUN:   -r=%t.o,_use_arc,px
+
+; RUN: %clang_cc1 -triple x86_64-apple-darwin \
+; RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
+; RUN:   -o %t.native.o -x ir %t.o
+
+target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @use_arc(ptr %a, ptr %b) {
+  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  ret void
+}
+
+declare void @llvm.objc.clang.arc.use(...) nounwind

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-15 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari edited 
https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-16 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari ready_for_review 
https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-16 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari edited 
https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-16 Thread Nuri Amari via cfe-commits

NuriAmari wrote:

> Shouldn't this be added to the LTO code generator? In `libLTO` (used by 
> Apple's linker) it is added here `llvm/lib/LTO/ThinLTOCodeGenerator.cpp`

Presumably because the pass is likely not useful unless targeting MachO, LLD 
does this via configuration hook: https://reviews.llvm.org/D94547. I'm already 
doing it unconditionally, so what do you think about adding the pass here: 
https://github.com/llvm/llvm-project/blob/main/llvm/lib/LTO/LTOBackend.cpp#L417,
 and deleting this: 
https://github.com/llvm/llvm-project/blob/main/lld/MachO/LTO.cpp#L51-L53 ?

https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-16 Thread Nuri Amari via cfe-commits


@@ -0,0 +1,19 @@
+; RUN: opt -thinlto-bc -o %t.o %s
+
+; RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
+; RUN:   -o %t2.index \
+; RUN:   -r=%t.o,_use_arc,px
+
+; RUN: %clang_cc1 -triple x86_64-apple-darwin \
+; RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
+; RUN:   -o %t.native.o -x ir %t.o
+
+target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @use_arc(ptr %a, ptr %b) {
+  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind

NuriAmari wrote:

TODO: `i8*` -> `ptr`

https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-17 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/92331

>From 66ddf609c0e77867ec48c17136fb80d1e482041d Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Wed, 15 May 2024 16:33:03 -0700
Subject: [PATCH 1/2] Run ObjCContractPass in Distributed Thin-LTO Pipeline

Prior to this patch, when using -fthinlto-index= the
ObjCARCContractPass isn't run prior to CodeGen, and
instruction selection fails on IR containing
arc intrinstics.
---
 clang/lib/CodeGen/BackendUtil.cpp |  3 +++
 .../thinlto-distributed-objc-contract-pass.ll | 19 +++
 2 files changed, 22 insertions(+)
 create mode 100644 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 90985c08fe7f8..03dd1df281d80 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,6 +1289,9 @@ static void runThinLTOBackend(
 };
 break;
   default:
+Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
+  Pm.add(createObjCARCContractPass());
+};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
new file mode 100644
index 0..7d0247555b5c8
--- /dev/null
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -0,0 +1,19 @@
+; RUN: opt -thinlto-bc -o %t.o %s
+
+; RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
+; RUN:   -o %t2.index \
+; RUN:   -r=%t.o,_use_arc,px
+
+; RUN: %clang_cc1 -triple x86_64-apple-darwin \
+; RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
+; RUN:   -o %t.native.o -x ir %t.o
+
+target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @use_arc(ptr %a, ptr %b) {
+  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  ret void
+}
+
+declare void @llvm.objc.clang.arc.use(...) nounwind

>From 142174dff7b12827f7af187e4ff08b3e75486923 Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Fri, 17 May 2024 11:38:16 -0700
Subject: [PATCH 2/2] Address PR Feedback

- Adjust test
- Unconditionally include ARC pass in ThinLTO pipeline
---
 clang/lib/CodeGen/BackendUtil.cpp| 3 ---
 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll | 2 +-
 lld/MachO/LTO.cpp| 3 ---
 llvm/lib/LTO/LTOBackend.cpp  | 2 ++
 4 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 03dd1df281d80..90985c08fe7f8 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,9 +1289,6 @@ static void runThinLTOBackend(
 };
 break;
   default:
-Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
-  Pm.add(createObjCARCContractPass());
-};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
index 7d0247555b5c8..a2a7b62cb7f93 100644
--- a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -12,7 +12,7 @@ target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16
 target triple = "x86_64-apple-darwin"
 
 define void @use_arc(ptr %a, ptr %b) {
-  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  call void (...) @llvm.objc.clang.arc.use(ptr %a, ptr %b) nounwind
   ret void
 }
 
diff --git a/lld/MachO/LTO.cpp b/lld/MachO/LTO.cpp
index 7a9a9223a0322..6527cbb68f249 100644
--- a/lld/MachO/LTO.cpp
+++ b/lld/MachO/LTO.cpp
@@ -48,9 +48,6 @@ static lto::Config createConfig() {
   c.CPU = getCPUStr();
   c.MAttrs = getMAttrs();
   c.DiagHandler = diagnosticHandler;
-  c.PreCodeGenPassesHook = [](legacy::PassManager &pm) {
-pm.add(createObjCARCContractPass());
-  };
 
   c.AlwaysEmitRegularLTOObj = !config->ltoObjPath.empty();
 
diff --git a/llvm/lib/LTO/LTOBackend.cpp b/llvm/lib/LTO/LTOBackend.cpp
index d4b89ede2d713..aef3a7575dfdf 100644
--- a/llvm/lib/LTO/LTOBackend.cpp
+++ b/llvm/lib/LTO/LTOBackend.cpp
@@ -42,6 +42,7 @@
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/TargetParser/SubtargetFeature.h"
 #include "llvm/Transforms/IPO/WholeProgramDevirt.h"
+#include "llvm/Transforms/ObjCARC.h"
 #include "llvm/Transforms/Scalar/LoopPassManager.h"
 #include "llvm/Transforms/Utils/FunctionImportUtils.h"
 #include "llvm/Transforms/Utils/SplitModule.h"
@@ -415,6 +416,7 @@ static void codegen(const Config &Conf, TargetMachine *TM,
   CodeGenPasses.add(new TargetLibraryInfoWrapperPass(TLII));
   CodeGenPasses.add(
   createImmutableModuleSummaryIndexWrapperPass(&CombinedIndex)

[clang] [lld] [llvm] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-17 Thread Nuri Amari via cfe-commits

NuriAmari wrote:

> > Shouldn't this be added to the LTO code generator? In `libLTO` (used by 
> > Apple's linker) it is added here `llvm/lib/LTO/ThinLTOCodeGenerator.cpp`
> 
> Presumably because the pass is likely not useful unless targeting MachO, LLD 
> does this via configuration hook: https://reviews.llvm.org/D94547. I'm 
> already doing it unconditionally, so what do you think about adding the pass 
> here: 
> https://github.com/llvm/llvm-project/blob/main/llvm/lib/LTO/LTOBackend.cpp#L417,
>  and deleting this: 
> https://github.com/llvm/llvm-project/blob/main/lld/MachO/LTO.cpp#L51-L53 ?

I've made this change

https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-20 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/92331

>From 66ddf609c0e77867ec48c17136fb80d1e482041d Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Wed, 15 May 2024 16:33:03 -0700
Subject: [PATCH 1/3] Run ObjCContractPass in Distributed Thin-LTO Pipeline

Prior to this patch, when using -fthinlto-index= the
ObjCARCContractPass isn't run prior to CodeGen, and
instruction selection fails on IR containing
arc intrinstics.
---
 clang/lib/CodeGen/BackendUtil.cpp |  3 +++
 .../thinlto-distributed-objc-contract-pass.ll | 19 +++
 2 files changed, 22 insertions(+)
 create mode 100644 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 90985c08fe7f8..03dd1df281d80 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,6 +1289,9 @@ static void runThinLTOBackend(
 };
 break;
   default:
+Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
+  Pm.add(createObjCARCContractPass());
+};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
new file mode 100644
index 0..7d0247555b5c8
--- /dev/null
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -0,0 +1,19 @@
+; RUN: opt -thinlto-bc -o %t.o %s
+
+; RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
+; RUN:   -o %t2.index \
+; RUN:   -r=%t.o,_use_arc,px
+
+; RUN: %clang_cc1 -triple x86_64-apple-darwin \
+; RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
+; RUN:   -o %t.native.o -x ir %t.o
+
+target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @use_arc(ptr %a, ptr %b) {
+  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  ret void
+}
+
+declare void @llvm.objc.clang.arc.use(...) nounwind

>From 142174dff7b12827f7af187e4ff08b3e75486923 Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Fri, 17 May 2024 11:38:16 -0700
Subject: [PATCH 2/3] Address PR Feedback

- Adjust test
- Unconditionally include ARC pass in ThinLTO pipeline
---
 clang/lib/CodeGen/BackendUtil.cpp| 3 ---
 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll | 2 +-
 lld/MachO/LTO.cpp| 3 ---
 llvm/lib/LTO/LTOBackend.cpp  | 2 ++
 4 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 03dd1df281d80..90985c08fe7f8 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,9 +1289,6 @@ static void runThinLTOBackend(
 };
 break;
   default:
-Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
-  Pm.add(createObjCARCContractPass());
-};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
index 7d0247555b5c8..a2a7b62cb7f93 100644
--- a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -12,7 +12,7 @@ target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16
 target triple = "x86_64-apple-darwin"
 
 define void @use_arc(ptr %a, ptr %b) {
-  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  call void (...) @llvm.objc.clang.arc.use(ptr %a, ptr %b) nounwind
   ret void
 }
 
diff --git a/lld/MachO/LTO.cpp b/lld/MachO/LTO.cpp
index 7a9a9223a0322..6527cbb68f249 100644
--- a/lld/MachO/LTO.cpp
+++ b/lld/MachO/LTO.cpp
@@ -48,9 +48,6 @@ static lto::Config createConfig() {
   c.CPU = getCPUStr();
   c.MAttrs = getMAttrs();
   c.DiagHandler = diagnosticHandler;
-  c.PreCodeGenPassesHook = [](legacy::PassManager &pm) {
-pm.add(createObjCARCContractPass());
-  };
 
   c.AlwaysEmitRegularLTOObj = !config->ltoObjPath.empty();
 
diff --git a/llvm/lib/LTO/LTOBackend.cpp b/llvm/lib/LTO/LTOBackend.cpp
index d4b89ede2d713..aef3a7575dfdf 100644
--- a/llvm/lib/LTO/LTOBackend.cpp
+++ b/llvm/lib/LTO/LTOBackend.cpp
@@ -42,6 +42,7 @@
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/TargetParser/SubtargetFeature.h"
 #include "llvm/Transforms/IPO/WholeProgramDevirt.h"
+#include "llvm/Transforms/ObjCARC.h"
 #include "llvm/Transforms/Scalar/LoopPassManager.h"
 #include "llvm/Transforms/Utils/FunctionImportUtils.h"
 #include "llvm/Transforms/Utils/SplitModule.h"
@@ -415,6 +416,7 @@ static void codegen(const Config &Conf, TargetMachine *TM,
   CodeGenPasses.add(new TargetLibraryInfoWrapperPass(TLII));
   CodeGenPasses.add(
   createImmutableModuleSummaryIndexWrapperPass(&CombinedIndex)

[clang] [lld] [llvm] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-20 Thread Nuri Amari via cfe-commits


@@ -101,6 +101,19 @@ class ARCRuntimeEntryPoints {
 llvm_unreachable("Switch should be a covered switch.");
   }
 
+  bool moduleContainsARCEntryPoints() {
+assert(TheModule != nullptr && "Not initialized.");
+
+for (auto ARCInstricID :
+ enum_seq_inclusive(Intrinsic::objc_arc_annotation_bottomup_bbend,

NuriAmari wrote:

Bit fragile if new arc intrinsics are added, or they are re-arranged, but not 
sure what else to do. Perhaps giving the sequence a name indicating it is the 
canonical representation of "all arc intrinsics" will help it stay up to date.

https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-20 Thread Nuri Amari via cfe-commits

NuriAmari wrote:

> it seems like this should just be in the default codegen pipeline? you'd need 
> to change the pass to bail out early if there are no relevant intrinsics (by 
> checking if the module contains the intrinsic declaration) to not affect 
> compile times

I've done this, and removed the other places the pass is added currently. Let 
me know if the location I've added it looks reasonable. Thanks.

> Maybe that is a good idea since the check is already there and the check is 
> quite cheap (just check the existence of ARC intrinsics).

@cachemeifyoucan I couldn't find this check you're alluding to, so I added my 
own. If the check does exist, could you point me to it? If not, does mine look 
reasonable? Thanks.

https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-20 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/92331

>From 66ddf609c0e77867ec48c17136fb80d1e482041d Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Wed, 15 May 2024 16:33:03 -0700
Subject: [PATCH 1/4] Run ObjCContractPass in Distributed Thin-LTO Pipeline

Prior to this patch, when using -fthinlto-index= the
ObjCARCContractPass isn't run prior to CodeGen, and
instruction selection fails on IR containing
arc intrinstics.
---
 clang/lib/CodeGen/BackendUtil.cpp |  3 +++
 .../thinlto-distributed-objc-contract-pass.ll | 19 +++
 2 files changed, 22 insertions(+)
 create mode 100644 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 90985c08fe7f8..03dd1df281d80 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,6 +1289,9 @@ static void runThinLTOBackend(
 };
 break;
   default:
+Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
+  Pm.add(createObjCARCContractPass());
+};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
new file mode 100644
index 0..7d0247555b5c8
--- /dev/null
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -0,0 +1,19 @@
+; RUN: opt -thinlto-bc -o %t.o %s
+
+; RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
+; RUN:   -o %t2.index \
+; RUN:   -r=%t.o,_use_arc,px
+
+; RUN: %clang_cc1 -triple x86_64-apple-darwin \
+; RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
+; RUN:   -o %t.native.o -x ir %t.o
+
+target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @use_arc(ptr %a, ptr %b) {
+  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  ret void
+}
+
+declare void @llvm.objc.clang.arc.use(...) nounwind

>From 142174dff7b12827f7af187e4ff08b3e75486923 Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Fri, 17 May 2024 11:38:16 -0700
Subject: [PATCH 2/4] Address PR Feedback

- Adjust test
- Unconditionally include ARC pass in ThinLTO pipeline
---
 clang/lib/CodeGen/BackendUtil.cpp| 3 ---
 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll | 2 +-
 lld/MachO/LTO.cpp| 3 ---
 llvm/lib/LTO/LTOBackend.cpp  | 2 ++
 4 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 03dd1df281d80..90985c08fe7f8 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,9 +1289,6 @@ static void runThinLTOBackend(
 };
 break;
   default:
-Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
-  Pm.add(createObjCARCContractPass());
-};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
index 7d0247555b5c8..a2a7b62cb7f93 100644
--- a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -12,7 +12,7 @@ target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16
 target triple = "x86_64-apple-darwin"
 
 define void @use_arc(ptr %a, ptr %b) {
-  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  call void (...) @llvm.objc.clang.arc.use(ptr %a, ptr %b) nounwind
   ret void
 }
 
diff --git a/lld/MachO/LTO.cpp b/lld/MachO/LTO.cpp
index 7a9a9223a0322..6527cbb68f249 100644
--- a/lld/MachO/LTO.cpp
+++ b/lld/MachO/LTO.cpp
@@ -48,9 +48,6 @@ static lto::Config createConfig() {
   c.CPU = getCPUStr();
   c.MAttrs = getMAttrs();
   c.DiagHandler = diagnosticHandler;
-  c.PreCodeGenPassesHook = [](legacy::PassManager &pm) {
-pm.add(createObjCARCContractPass());
-  };
 
   c.AlwaysEmitRegularLTOObj = !config->ltoObjPath.empty();
 
diff --git a/llvm/lib/LTO/LTOBackend.cpp b/llvm/lib/LTO/LTOBackend.cpp
index d4b89ede2d713..aef3a7575dfdf 100644
--- a/llvm/lib/LTO/LTOBackend.cpp
+++ b/llvm/lib/LTO/LTOBackend.cpp
@@ -42,6 +42,7 @@
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/TargetParser/SubtargetFeature.h"
 #include "llvm/Transforms/IPO/WholeProgramDevirt.h"
+#include "llvm/Transforms/ObjCARC.h"
 #include "llvm/Transforms/Scalar/LoopPassManager.h"
 #include "llvm/Transforms/Utils/FunctionImportUtils.h"
 #include "llvm/Transforms/Utils/SplitModule.h"
@@ -415,6 +416,7 @@ static void codegen(const Config &Conf, TargetMachine *TM,
   CodeGenPasses.add(new TargetLibraryInfoWrapperPass(TLII));
   CodeGenPasses.add(
   createImmutableModuleSummaryIndexWrapperPass(&CombinedIndex)

[clang] [lld] [llvm] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-20 Thread Nuri Amari via cfe-commits

NuriAmari wrote:

I've added the check back in, and reverted my custom check. There is too much 
going on in  https://reviews.llvm.org/D92808 for me to quickly understand 
everything, but I don't see an obvious reason the check needed to be removed. 
@ahatanaka Please let me know if I'm missing something.

https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-20 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/92331

>From 66ddf609c0e77867ec48c17136fb80d1e482041d Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Wed, 15 May 2024 16:33:03 -0700
Subject: [PATCH 1/5] Run ObjCContractPass in Distributed Thin-LTO Pipeline

Prior to this patch, when using -fthinlto-index= the
ObjCARCContractPass isn't run prior to CodeGen, and
instruction selection fails on IR containing
arc intrinstics.
---
 clang/lib/CodeGen/BackendUtil.cpp |  3 +++
 .../thinlto-distributed-objc-contract-pass.ll | 19 +++
 2 files changed, 22 insertions(+)
 create mode 100644 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 90985c08fe7f8..03dd1df281d80 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,6 +1289,9 @@ static void runThinLTOBackend(
 };
 break;
   default:
+Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
+  Pm.add(createObjCARCContractPass());
+};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
new file mode 100644
index 0..7d0247555b5c8
--- /dev/null
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -0,0 +1,19 @@
+; RUN: opt -thinlto-bc -o %t.o %s
+
+; RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
+; RUN:   -o %t2.index \
+; RUN:   -r=%t.o,_use_arc,px
+
+; RUN: %clang_cc1 -triple x86_64-apple-darwin \
+; RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
+; RUN:   -o %t.native.o -x ir %t.o
+
+target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @use_arc(ptr %a, ptr %b) {
+  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  ret void
+}
+
+declare void @llvm.objc.clang.arc.use(...) nounwind

>From 142174dff7b12827f7af187e4ff08b3e75486923 Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Fri, 17 May 2024 11:38:16 -0700
Subject: [PATCH 2/5] Address PR Feedback

- Adjust test
- Unconditionally include ARC pass in ThinLTO pipeline
---
 clang/lib/CodeGen/BackendUtil.cpp| 3 ---
 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll | 2 +-
 lld/MachO/LTO.cpp| 3 ---
 llvm/lib/LTO/LTOBackend.cpp  | 2 ++
 4 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 03dd1df281d80..90985c08fe7f8 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,9 +1289,6 @@ static void runThinLTOBackend(
 };
 break;
   default:
-Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
-  Pm.add(createObjCARCContractPass());
-};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
index 7d0247555b5c8..a2a7b62cb7f93 100644
--- a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -12,7 +12,7 @@ target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16
 target triple = "x86_64-apple-darwin"
 
 define void @use_arc(ptr %a, ptr %b) {
-  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  call void (...) @llvm.objc.clang.arc.use(ptr %a, ptr %b) nounwind
   ret void
 }
 
diff --git a/lld/MachO/LTO.cpp b/lld/MachO/LTO.cpp
index 7a9a9223a0322..6527cbb68f249 100644
--- a/lld/MachO/LTO.cpp
+++ b/lld/MachO/LTO.cpp
@@ -48,9 +48,6 @@ static lto::Config createConfig() {
   c.CPU = getCPUStr();
   c.MAttrs = getMAttrs();
   c.DiagHandler = diagnosticHandler;
-  c.PreCodeGenPassesHook = [](legacy::PassManager &pm) {
-pm.add(createObjCARCContractPass());
-  };
 
   c.AlwaysEmitRegularLTOObj = !config->ltoObjPath.empty();
 
diff --git a/llvm/lib/LTO/LTOBackend.cpp b/llvm/lib/LTO/LTOBackend.cpp
index d4b89ede2d713..aef3a7575dfdf 100644
--- a/llvm/lib/LTO/LTOBackend.cpp
+++ b/llvm/lib/LTO/LTOBackend.cpp
@@ -42,6 +42,7 @@
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/TargetParser/SubtargetFeature.h"
 #include "llvm/Transforms/IPO/WholeProgramDevirt.h"
+#include "llvm/Transforms/ObjCARC.h"
 #include "llvm/Transforms/Scalar/LoopPassManager.h"
 #include "llvm/Transforms/Utils/FunctionImportUtils.h"
 #include "llvm/Transforms/Utils/SplitModule.h"
@@ -415,6 +416,7 @@ static void codegen(const Config &Conf, TargetMachine *TM,
   CodeGenPasses.add(new TargetLibraryInfoWrapperPass(TLII));
   CodeGenPasses.add(
   createImmutableModuleSummaryIndexWrapperPass(&CombinedIndex)

[clang] [lld] [llvm] Run ObjCContractPass in Distributed Thin-LTO Pipeline (PR #92331)

2024-05-21 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/92331

>From 66ddf609c0e77867ec48c17136fb80d1e482041d Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Wed, 15 May 2024 16:33:03 -0700
Subject: [PATCH 1/6] Run ObjCContractPass in Distributed Thin-LTO Pipeline

Prior to this patch, when using -fthinlto-index= the
ObjCARCContractPass isn't run prior to CodeGen, and
instruction selection fails on IR containing
arc intrinstics.
---
 clang/lib/CodeGen/BackendUtil.cpp |  3 +++
 .../thinlto-distributed-objc-contract-pass.ll | 19 +++
 2 files changed, 22 insertions(+)
 create mode 100644 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 90985c08fe7f8..03dd1df281d80 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,6 +1289,9 @@ static void runThinLTOBackend(
 };
 break;
   default:
+Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
+  Pm.add(createObjCARCContractPass());
+};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
new file mode 100644
index 0..7d0247555b5c8
--- /dev/null
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -0,0 +1,19 @@
+; RUN: opt -thinlto-bc -o %t.o %s
+
+; RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
+; RUN:   -o %t2.index \
+; RUN:   -r=%t.o,_use_arc,px
+
+; RUN: %clang_cc1 -triple x86_64-apple-darwin \
+; RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
+; RUN:   -o %t.native.o -x ir %t.o
+
+target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @use_arc(ptr %a, ptr %b) {
+  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  ret void
+}
+
+declare void @llvm.objc.clang.arc.use(...) nounwind

>From 142174dff7b12827f7af187e4ff08b3e75486923 Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Fri, 17 May 2024 11:38:16 -0700
Subject: [PATCH 2/6] Address PR Feedback

- Adjust test
- Unconditionally include ARC pass in ThinLTO pipeline
---
 clang/lib/CodeGen/BackendUtil.cpp| 3 ---
 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll | 2 +-
 lld/MachO/LTO.cpp| 3 ---
 llvm/lib/LTO/LTOBackend.cpp  | 2 ++
 4 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 03dd1df281d80..90985c08fe7f8 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,9 +1289,6 @@ static void runThinLTOBackend(
 };
 break;
   default:
-Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
-  Pm.add(createObjCARCContractPass());
-};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
index 7d0247555b5c8..a2a7b62cb7f93 100644
--- a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -12,7 +12,7 @@ target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16
 target triple = "x86_64-apple-darwin"
 
 define void @use_arc(ptr %a, ptr %b) {
-  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  call void (...) @llvm.objc.clang.arc.use(ptr %a, ptr %b) nounwind
   ret void
 }
 
diff --git a/lld/MachO/LTO.cpp b/lld/MachO/LTO.cpp
index 7a9a9223a0322..6527cbb68f249 100644
--- a/lld/MachO/LTO.cpp
+++ b/lld/MachO/LTO.cpp
@@ -48,9 +48,6 @@ static lto::Config createConfig() {
   c.CPU = getCPUStr();
   c.MAttrs = getMAttrs();
   c.DiagHandler = diagnosticHandler;
-  c.PreCodeGenPassesHook = [](legacy::PassManager &pm) {
-pm.add(createObjCARCContractPass());
-  };
 
   c.AlwaysEmitRegularLTOObj = !config->ltoObjPath.empty();
 
diff --git a/llvm/lib/LTO/LTOBackend.cpp b/llvm/lib/LTO/LTOBackend.cpp
index d4b89ede2d713..aef3a7575dfdf 100644
--- a/llvm/lib/LTO/LTOBackend.cpp
+++ b/llvm/lib/LTO/LTOBackend.cpp
@@ -42,6 +42,7 @@
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/TargetParser/SubtargetFeature.h"
 #include "llvm/Transforms/IPO/WholeProgramDevirt.h"
+#include "llvm/Transforms/ObjCARC.h"
 #include "llvm/Transforms/Scalar/LoopPassManager.h"
 #include "llvm/Transforms/Utils/FunctionImportUtils.h"
 #include "llvm/Transforms/Utils/SplitModule.h"
@@ -415,6 +416,7 @@ static void codegen(const Config &Conf, TargetMachine *TM,
   CodeGenPasses.add(new TargetLibraryInfoWrapperPass(TLII));
   CodeGenPasses.add(
   createImmutableModuleSummaryIndexWrapperPass(&CombinedIndex)

[clang] [lld] [llvm] Run ObjCContractPass in Default Codegen Pipeline (PR #92331)

2024-05-22 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari edited 
https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] Run ObjCContractPass in Default Codegen Pipeline (PR #92331)

2024-05-22 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/92331

>From 66ddf609c0e77867ec48c17136fb80d1e482041d Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Wed, 15 May 2024 16:33:03 -0700
Subject: [PATCH 1/7] Run ObjCContractPass in Distributed Thin-LTO Pipeline

Prior to this patch, when using -fthinlto-index= the
ObjCARCContractPass isn't run prior to CodeGen, and
instruction selection fails on IR containing
arc intrinstics.
---
 clang/lib/CodeGen/BackendUtil.cpp |  3 +++
 .../thinlto-distributed-objc-contract-pass.ll | 19 +++
 2 files changed, 22 insertions(+)
 create mode 100644 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 90985c08fe7f8..03dd1df281d80 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,6 +1289,9 @@ static void runThinLTOBackend(
 };
 break;
   default:
+Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
+  Pm.add(createObjCARCContractPass());
+};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
new file mode 100644
index 0..7d0247555b5c8
--- /dev/null
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -0,0 +1,19 @@
+; RUN: opt -thinlto-bc -o %t.o %s
+
+; RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
+; RUN:   -o %t2.index \
+; RUN:   -r=%t.o,_use_arc,px
+
+; RUN: %clang_cc1 -triple x86_64-apple-darwin \
+; RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
+; RUN:   -o %t.native.o -x ir %t.o
+
+target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @use_arc(ptr %a, ptr %b) {
+  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  ret void
+}
+
+declare void @llvm.objc.clang.arc.use(...) nounwind

>From 142174dff7b12827f7af187e4ff08b3e75486923 Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Fri, 17 May 2024 11:38:16 -0700
Subject: [PATCH 2/7] Address PR Feedback

- Adjust test
- Unconditionally include ARC pass in ThinLTO pipeline
---
 clang/lib/CodeGen/BackendUtil.cpp| 3 ---
 clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll | 2 +-
 lld/MachO/LTO.cpp| 3 ---
 llvm/lib/LTO/LTOBackend.cpp  | 2 ++
 4 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/clang/lib/CodeGen/BackendUtil.cpp 
b/clang/lib/CodeGen/BackendUtil.cpp
index 03dd1df281d80..90985c08fe7f8 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -1289,9 +1289,6 @@ static void runThinLTOBackend(
 };
 break;
   default:
-Conf.PreCodeGenPassesHook = [](legacy::PassManager &Pm) {
-  Pm.add(createObjCARCContractPass());
-};
 Conf.CGFileType = getCodeGenFileType(Action);
 break;
   }
diff --git a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll 
b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
index 7d0247555b5c8..a2a7b62cb7f93 100644
--- a/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
+++ b/clang/test/CodeGen/thinlto-distributed-objc-contract-pass.ll
@@ -12,7 +12,7 @@ target datalayout = 
"e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16
 target triple = "x86_64-apple-darwin"
 
 define void @use_arc(ptr %a, ptr %b) {
-  call void (...) @llvm.objc.clang.arc.use(i8* %a, i8* %b) nounwind
+  call void (...) @llvm.objc.clang.arc.use(ptr %a, ptr %b) nounwind
   ret void
 }
 
diff --git a/lld/MachO/LTO.cpp b/lld/MachO/LTO.cpp
index 7a9a9223a0322..6527cbb68f249 100644
--- a/lld/MachO/LTO.cpp
+++ b/lld/MachO/LTO.cpp
@@ -48,9 +48,6 @@ static lto::Config createConfig() {
   c.CPU = getCPUStr();
   c.MAttrs = getMAttrs();
   c.DiagHandler = diagnosticHandler;
-  c.PreCodeGenPassesHook = [](legacy::PassManager &pm) {
-pm.add(createObjCARCContractPass());
-  };
 
   c.AlwaysEmitRegularLTOObj = !config->ltoObjPath.empty();
 
diff --git a/llvm/lib/LTO/LTOBackend.cpp b/llvm/lib/LTO/LTOBackend.cpp
index d4b89ede2d713..aef3a7575dfdf 100644
--- a/llvm/lib/LTO/LTOBackend.cpp
+++ b/llvm/lib/LTO/LTOBackend.cpp
@@ -42,6 +42,7 @@
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/TargetParser/SubtargetFeature.h"
 #include "llvm/Transforms/IPO/WholeProgramDevirt.h"
+#include "llvm/Transforms/ObjCARC.h"
 #include "llvm/Transforms/Scalar/LoopPassManager.h"
 #include "llvm/Transforms/Utils/FunctionImportUtils.h"
 #include "llvm/Transforms/Utils/SplitModule.h"
@@ -415,6 +416,7 @@ static void codegen(const Config &Conf, TargetMachine *TM,
   CodeGenPasses.add(new TargetLibraryInfoWrapperPass(TLII));
   CodeGenPasses.add(
   createImmutableModuleSummaryIndexWrapperPass(&CombinedIndex)

[clang] [lld] [llvm] Run ObjCContractPass in Default Codegen Pipeline (PR #92331)

2024-05-23 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari closed 
https://github.com/llvm/llvm-project/pull/92331
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [CGData] Clang Options (PR #90304)

2024-09-13 Thread Nuri Amari via cfe-commits

NuriAmari wrote:

> Do you see or expect any issues with using distributed ThinLTO?

No seems good to me!

https://github.com/llvm/llvm-project/pull/90304
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [CGData] Clang Options (PR #90304)

2024-09-13 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari approved this pull request.


https://github.com/llvm/llvm-project/pull/90304
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [llvm] [CGData][ThinLTO] Global Outlining with Two-CodeGen Rounds (PR #90933)

2024-09-17 Thread Nuri Amari via cfe-commits


@@ -1558,6 +1562,60 @@ class InProcessThinBackend : public ThinBackendProc {
 return BackendThreadPool.getMaxConcurrency();
   }
 };
+
+/// This Backend will run ThinBackend process but throw away all the output 
from
+/// the codegen. This class facilitates the first codegen round.
+class NoOutputThinBackend : public InProcessThinBackend {
+public:
+  NoOutputThinBackend(
+  const Config &Conf, ModuleSummaryIndex &CombinedIndex,
+  ThreadPoolStrategy ThinLTOParallelism,
+  const DenseMap &ModuleToDefinedGVSummaries,
+  std::unique_ptr>> Scratch)
+  : InProcessThinBackend(
+Conf, CombinedIndex, ThinLTOParallelism, 
ModuleToDefinedGVSummaries,
+// Allocate a scratch buffer for each task to write output to.
+[Allocation = &*Scratch](unsigned Task, const Twine &ModuleName) {
+  return std::make_unique(
+  std::make_unique((*Allocation)[Task]));
+},
+FileCache(), nullptr, false, false),

NuriAmari wrote:

Do you mind labelling these arguments with a comment including the name of the 
parameter like you've done below: `/*CodeGenOnly=*/true`. Same with the 
`OptimizedBitcodeThinBackend` constructor.

https://github.com/llvm/llvm-project/pull/90933
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [llvm] [CGData][ThinLTO] Global Outlining with Two-CodeGen Rounds (PR #90933)

2024-09-17 Thread Nuri Amari via cfe-commits


@@ -1558,6 +1562,60 @@ class InProcessThinBackend : public ThinBackendProc {
 return BackendThreadPool.getMaxConcurrency();
   }
 };
+
+/// This Backend will run ThinBackend process but throw away all the output 
from
+/// the codegen. This class facilitates the first codegen round.
+class NoOutputThinBackend : public InProcessThinBackend {

NuriAmari wrote:

If I'm following correctly, it's not as if this backend really throws results 
away, but it writes the produced object files into the scratch buffer rather 
than to files. It also writes the optimized bitcode files to disk right?

I don't have a strong opinion, but maybe we could name the backend / adjust the 
comment to reflect this. I wouldn't even be opposed to naming the backends 
something specific to two-codegen rounds, as I don't see anyone using them for 
other purposes.

Lastly just an idea:  We could just hold the optimized bitcode in a similar 
buffer in memory, rather than writing them to disk and reading them again 
between rounds. Might run a bit faster.

https://github.com/llvm/llvm-project/pull/90933
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] Integrated Distributed ThinLTO (DTLTO): Initial support (PR #126654)

2025-02-11 Thread Nuri Amari via cfe-commits


@@ -2142,3 +2190,327 @@ std::vector 
lto::generateModulesOrdering(ArrayRef R) {
   });
   return ModulesOrdering;
 }
+
+namespace {
+// For this out-of-process backend no codegen is done when invoked for each
+// task. Instead we generate the required information (e.g. the summary index
+// shard,import list, etc..) to allow for the codegen to be performed
+// externally . This backend's `wait` function then invokes an external
+// distributor process to do backend compilations.
+class OutOfProcessThinBackend : public CGThinBackend {
+  using SString = SmallString<128>;
+
+  AddBufferFn AddBuffer;
+
+  BumpPtrAllocator Alloc;
+  StringSaver Saver{Alloc};
+
+  SString LinkerOutputFile;
+  StringRef LinkerVersion;
+  SString RemoteOptTool;
+  SString DistributorPath;
+  bool SaveTemps;
+
+  SmallVector CodegenOptions;
+  DenseSet AdditionalInputs;
+
+  // Information specific to individual backend compilation job.
+  struct Job {
+unsigned Task;
+StringRef ModuleID;
+StringRef Triple;
+StringRef NativeObjectPath;
+StringRef SummaryIndexPath;
+ImportsFilesContainer ImportFiles;
+  };
+  // The set of backend compilations jobs.
+  SmallVector Jobs;
+
+  // A unique string to identify the current link.
+  SmallString<8> UID;
+
+public:
+  OutOfProcessThinBackend(
+  const Config &Conf, ModuleSummaryIndex &CombinedIndex,
+  ThreadPoolStrategy ThinLTOParallelism,
+  const DenseMap &ModuleToDefinedGVSummaries,
+  AddStreamFn AddStream, AddBufferFn AddBuffer,
+  lto::IndexWriteCallback OnWrite, bool ShouldEmitIndexFiles,
+  bool ShouldEmitImportsFiles, StringRef LinkerOutputFile,
+  StringRef LinkerVersion, StringRef RemoteOptTool, StringRef Distributor,
+  bool SaveTemps)
+  : CGThinBackend(Conf, CombinedIndex, ModuleToDefinedGVSummaries, OnWrite,
+  ShouldEmitIndexFiles, ShouldEmitImportsFiles,
+  ThinLTOParallelism),
+AddBuffer(std::move(AddBuffer)), LinkerOutputFile(LinkerOutputFile),
+LinkerVersion(LinkerVersion), RemoteOptTool(RemoteOptTool),
+DistributorPath(Distributor), SaveTemps(SaveTemps) {}
+
+  virtual void setup(unsigned MaxTasks) override {
+UID = itostr(sys::Process::getProcessId());
+Jobs.resize((size_t)MaxTasks);
+  }
+
+  Error start(
+  unsigned Task, BitcodeModule BM,
+  const FunctionImporter::ImportMapTy &ImportList,
+  const FunctionImporter::ExportSetTy &ExportList,
+  const std::map 
&ResolvedODR,
+  MapVector &ModuleMap,
+  DenseMap &ModuleTriples) override {
+
+StringRef ModulePath = BM.getModuleIdentifier();
+
+SString ObjFilePath = sys::path::parent_path(LinkerOutputFile);
+sys::path::append(ObjFilePath, sys::path::stem(ModulePath) + "." +
+   itostr(Task) + "." + UID + ".native.o");
+
+Job &J = Jobs[Task - 1]; /*Task 0 is reserved*/
+J = {Task,
+ ModulePath,
+ ModuleTriples[ModulePath],
+ Saver.save(ObjFilePath.str()),
+ Saver.save(ObjFilePath.str() + ".thinlto.bc"),
+ {}};
+
+assert(ModuleToDefinedGVSummaries.count(ModulePath));
+BackendThreadPool.async(
+[=](Job &J, const FunctionImporter::ImportMapTy &ImportList) {
+  if (LLVM_ENABLE_THREADS && Conf.TimeTraceEnabled)
+timeTraceProfilerInitialize(Conf.TimeTraceGranularity,
+"thin backend");
+  if (auto E = emitFiles(ImportList, J.ModuleID, J.SummaryIndexPath,
+ J.ModuleID.str(), J.ImportFiles)) {
+std::unique_lock L(ErrMu);
+if (Err)
+  Err = joinErrors(std::move(*Err), std::move(E));
+else
+  Err = std::move(E);
+  }
+  if (LLVM_ENABLE_THREADS && Conf.TimeTraceEnabled)
+timeTraceProfilerFinishThread();
+},
+std::ref(J), std::ref(ImportList));
+
+return Error::success();
+  }
+
+  // Derive a set of Clang options that will be shared/common for all DTLTO

NuriAmari wrote:

With or without this patch to add integrated dthin-lto, I think something like 
this (automatic translation of a thin-link invocation into a matching Clang 
backend invocation) would be useful. Optimization level and flags forwarded 
through Clang directly to the linker (ex: `-Wl,-mllvm,-my_codegen_option`) are 
those I've run into most often.

For Buck we either have this [imho unfortunate script wrapping the backend 
Clang 
invocations](https://github.com/facebook/buck2/blob/main/prelude/cxx/dist_lto/tools/dist_lto_opt_gnu.py)
 that translates args from the thin-link invocation to the Clang backend 
invocation, or we specify both sets of flags manually. @teresajohnson @MaskRay 
How does Bazel handle this problem?

https://github.com/llvm/llvm-project/pull/126654
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
ht

[clang] Avoid Assertion Failure Using -fcs-profile-generate with distributed thin-lto (PR #129736)

2025-03-14 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari updated 
https://github.com/llvm/llvm-project/pull/129736

>From fc06ad49b1b166dd1ef09c8babe65c337487ee65 Mon Sep 17 00:00:00 2001
From: Nuri Amari 
Date: Mon, 3 Mar 2025 16:10:54 -0800
Subject: [PATCH 1/2] Move Clang distributd thin-lto codegen tests into their
 own directory

clang/test/CodeGen includes many tests, some of which pretain to
distributed thin-lto. While they have similar names, it is easier to
find them all in a common directory.
---
 .../backend-skip.ll}| 2 +-
 .../{thinlto-distributed.ll => distributed-thin-lto/basic.ll}   | 0
 .../cfi-devirt.ll}  | 2 +-
 .../{thinlto-distributed-cfi.ll => distributed-thin-lto/cfi.ll} | 2 +-
 .../newpm.ll}   | 0
 .../objc-contract-pass.ll}  | 0
 .../supports-hot-cold-new.ll}   | 0
 7 files changed, 3 insertions(+), 3 deletions(-)
 rename clang/test/CodeGen/{thinlto-distributed-backend-skip.ll => 
distributed-thin-lto/backend-skip.ll} (87%)
 rename clang/test/CodeGen/{thinlto-distributed.ll => 
distributed-thin-lto/basic.ll} (100%)
 rename clang/test/CodeGen/{thinlto-distributed-cfi-devirt.ll => 
distributed-thin-lto/cfi-devirt.ll} (97%)
 rename clang/test/CodeGen/{thinlto-distributed-cfi.ll => 
distributed-thin-lto/cfi.ll} (95%)
 rename clang/test/CodeGen/{thinlto-distributed-newpm.ll => 
distributed-thin-lto/newpm.ll} (100%)
 rename clang/test/CodeGen/{thinlto-distributed-objc-contract-pass.ll => 
distributed-thin-lto/objc-contract-pass.ll} (100%)
 rename clang/test/CodeGen/{thinlto-distributed-supports-hot-cold-new.ll => 
distributed-thin-lto/supports-hot-cold-new.ll} (100%)

diff --git a/clang/test/CodeGen/thinlto-distributed-backend-skip.ll 
b/clang/test/CodeGen/distributed-thin-lto/backend-skip.ll
similarity index 87%
rename from clang/test/CodeGen/thinlto-distributed-backend-skip.ll
rename to clang/test/CodeGen/distributed-thin-lto/backend-skip.ll
index d7b8225ee2693..62aa8aa8e7dc4 100644
--- a/clang/test/CodeGen/thinlto-distributed-backend-skip.ll
+++ b/clang/test/CodeGen/distributed-thin-lto/backend-skip.ll
@@ -6,7 +6,7 @@
 ; RUN: opt -thinlto-bc -o %t.o %s
 
 ; RUN: %clang_cc1 -triple x86_64-grtev4-linux-gnu \
-; RUN:   -fthinlto-index=%S/Inputs/thinlto-distributed-backend-skip.bc \
+; RUN:   -fthinlto-index=%S/../Inputs/thinlto-distributed-backend-skip.bc \
 ; RUN:   -emit-llvm -o - -x ir %t.o | FileCheck %s
 
 target datalayout = 
"e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
diff --git a/clang/test/CodeGen/thinlto-distributed.ll 
b/clang/test/CodeGen/distributed-thin-lto/basic.ll
similarity index 100%
rename from clang/test/CodeGen/thinlto-distributed.ll
rename to clang/test/CodeGen/distributed-thin-lto/basic.ll
diff --git a/clang/test/CodeGen/thinlto-distributed-cfi-devirt.ll 
b/clang/test/CodeGen/distributed-thin-lto/cfi-devirt.ll
similarity index 97%
rename from clang/test/CodeGen/thinlto-distributed-cfi-devirt.ll
rename to clang/test/CodeGen/distributed-thin-lto/cfi-devirt.ll
index acbcdcdb4fd02..a5792eee8abfb 100644
--- a/clang/test/CodeGen/thinlto-distributed-cfi-devirt.ll
+++ b/clang/test/CodeGen/distributed-thin-lto/cfi-devirt.ll
@@ -33,7 +33,7 @@
 ; RUN: llvm-dis %t.o.thinlto.bc -o - | FileCheck %s --check-prefix=CHECK-DIS
 ; Round trip it through llvm-as
 ; RUN: llvm-dis %t.o.thinlto.bc -o - | llvm-as -o - | llvm-dis -o - | 
FileCheck %s --check-prefix=CHECK-DIS
-; CHECK-DIS: ^0 = module: (path: 
"{{.*}}thinlto-distributed-cfi-devirt.ll.tmp.o", hash: ({{.*}}, {{.*}}, {{.*}}, 
{{.*}}, {{.*}}))
+; CHECK-DIS: ^0 = module: (path: "{{.*}}cfi-devirt.ll.tmp.o", hash: ({{.*}}, 
{{.*}}, {{.*}}, {{.*}}, {{.*}}))
 ; CHECK-DIS: ^1 = gv: (guid: 8346051122425466633, summaries: (function: 
(module: ^0, flags: (linkage: external, visibility: default, 
notEligibleToImport: 0, live: 1, dsoLocal: 0, canAutoHide: 0, importType: 
definition), insts: 18, funcFlags: (readNone: 0, readOnly: 0, noRecurse: 0, 
returnDoesNotAlias: 0, noInline: 0, alwaysInline: 0, noUnwind: 0, mayThrow: 0, 
hasUnknownCall: 1, mustBeUnreachable: 0), typeIdInfo: (typeTests: (^2), 
typeCheckedLoadVCalls: (vFuncId: (^2, offset: 8), vFuncId: (^2, offset: 0))
 ; CHECK-DIS: ^2 = typeid: (name: "_ZTS1A", summary: (typeTestRes: (kind: 
allOnes, sizeM1BitWidth: 7), wpdResolutions: ((offset: 0, wpdRes: (kind: 
branchFunnel)), (offset: 8, wpdRes: (kind: singleImpl, singleImplName: 
"_ZN1A1nEi") ; guid = 7004155349499253778
 
diff --git a/clang/test/CodeGen/thinlto-distributed-cfi.ll 
b/clang/test/CodeGen/distributed-thin-lto/cfi.ll
similarity index 95%
rename from clang/test/CodeGen/thinlto-distributed-cfi.ll
rename to clang/test/CodeGen/distributed-thin-lto/cfi.ll
index 6023ba8f32df9..696f2dc4ea634 100644
--- a/clang/test/CodeGen/thinlto-distributed-cfi.ll
+++ b/clang/test/CodeGen/distributed-thin-lto/cfi.ll
@@ -23,7 +23,7 @

[clang] Avoid Assertion Failure Using -fcs-profile-generate with distributed thin-lto (PR #129736)

2025-03-14 Thread Nuri Amari via cfe-commits


@@ -6,7 +6,7 @@
 ; RUN: opt -thinlto-bc -o %t.o %s
 
 ; RUN: %clang_cc1 -triple x86_64-grtev4-linux-gnu \
-; RUN:   -fthinlto-index=%S/Inputs/thinlto-distributed-backend-skip.bc \
+; RUN:   -fthinlto-index=%S/../Inputs/thinlto-distributed-backend-skip.bc \

NuriAmari wrote:

Because the test was moved down a directory, but the checked in sharded summary 
used as input to the test hasn't.

https://github.com/llvm/llvm-project/pull/129736
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] Avoid Assertion Failure Using -fcs-profile-generate with distributed thin-lto (PR #129736)

2025-03-14 Thread Nuri Amari via cfe-commits

https://github.com/NuriAmari edited 
https://github.com/llvm/llvm-project/pull/129736
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits