r340021 - [ThinLTO] Correct documentation on default number of threads

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Fri Aug 17 06:19:36 2018
New Revision: 340021

URL: http://llvm.org/viewvc/llvm-project?rev=340021&view=rev
Log:
[ThinLTO] Correct documentation on default number of threads

Summary:
The number of threads used for ThinLTO backend parallelism was
dropped to the number of cores in r284618 to avoid oversubscribing
physical cores due to hyperthreading. This updates the documentation
to reflect that change.

Fixes PR38610.

Reviewers: pcc

Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, cfe-commits

Differential Revision: https://reviews.llvm.org/D50882

Modified:
cfe/trunk/docs/ThinLTO.rst

Modified: cfe/trunk/docs/ThinLTO.rst
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/ThinLTO.rst?rev=340021&r1=340020&r2=340021&view=diff
==
--- cfe/trunk/docs/ThinLTO.rst (original)
+++ cfe/trunk/docs/ThinLTO.rst Fri Aug 17 06:19:36 2018
@@ -105,7 +105,9 @@ Controlling Backend Parallelism
 ---
 .. _parallelism:
 
-By default, the ThinLTO link step will launch up to
+By default, the ThinLTO link step will launch as many
+threads in parallel as there are cores. If the number of
+cores can't be computed for the architecture, then it will launch
 ``std::thread::hardware_concurrency`` number of threads in parallel.
 For machines with hyper-threading, this is the total number of
 virtual cores. For some applications and machine configurations this


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D47597: IRGen: Write .dwo files when -split-dwarf-file is used together with -fthinlto-index.

[Teresa Johnson via cfe-commits]

On Thu, May 31, 2018 at 12:00 PM Peter Collingbourne 
wrote:

>
>
> On Thu, May 31, 2018 at 11:54 AM, David Blaikie 
> wrote:
>
>>
>>
>> On Thu, May 31, 2018 at 11:20 AM Peter Collingbourne via Phabricator <
>> revi...@reviews.llvm.org> wrote:
>>
>>> pcc created this revision.
>>> pcc added reviewers: tejohnson, dblaikie.
>>> Herald added subscribers: JDevlieghere, hiraditya, eraman, inglorion,
>>> mehdi_amini.
>>>
>>> https://reviews.llvm.org/D47597
>>>
>>> Files:
>>>   clang/lib/CodeGen/BackendUtil.cpp
>>>   clang/test/CodeGen/thinlto-split-dwarf.c
>>>   llvm/include/llvm/LTO/Config.h
>>>   llvm/lib/LTO/LTOBackend.cpp
>>>
>>>
>>> Index: llvm/lib/LTO/LTOBackend.cpp
>>> ===
>>> --- llvm/lib/LTO/LTOBackend.cpp
>>> +++ llvm/lib/LTO/LTOBackend.cpp
>>> @@ -291,14 +291,19 @@
>>>  return;
>>>
>>>std::unique_ptr DwoOut;
>>> +  SmallString<1024> DwoFile(Conf.DwoPath);
>>>if (!Conf.DwoDir.empty()) {
>>>  std::error_code EC;
>>>  if (auto EC = llvm::sys::fs::create_directories(Conf.DwoDir))
>>>report_fatal_error("Failed to create directory " + Conf.DwoDir +
>>> ": " +
>>>   EC.message());
>>>
>>> -SmallString<1024> DwoFile(Conf.DwoDir);
>>> +DwoFile = Conf.DwoDir;
>>>  sys::path::append(DwoFile, std::to_string(Task) + ".dwo");
>>> +  }
>>> +
>>> +  if (!DwoFile.empty()) {
>>> +std::error_code EC;
>>>  TM->Options.MCOptions.SplitDwarfFile = DwoFile.str().str();
>>>  DwoOut = llvm::make_unique(DwoFile, EC,
>>> sys::fs::F_None);
>>>  if (EC)
>>> Index: llvm/include/llvm/LTO/Config.h
>>> ===
>>> --- llvm/include/llvm/LTO/Config.h
>>> +++ llvm/include/llvm/LTO/Config.h
>>> @@ -76,6 +76,11 @@
>>>/// The directory to store .dwo files.
>>>std::string DwoDir;
>>>
>>> +  /// The path to write a .dwo file to. This should generally only be
>>> used when
>>> +  /// running an individual backend directly via thinBackend(), as
>>> otherwise
>>> +  /// all .dwo files will be written to the same path.
>>> +  std::string DwoPath;
>>> +
>>>/// Optimization remarks file path.
>>>std::string RemarksFilename = "";
>>>
>>> Index: clang/test/CodeGen/thinlto-split-dwarf.c
>>> ===
>>> --- /dev/null
>>> +++ clang/test/CodeGen/thinlto-split-dwarf.c
>>> @@ -0,0 +1,21 @@
>>> +// REQUIRES: x86-registered-target
>>> +
>>> +// RUN: %clang_cc1 -debug-info-kind=limited -triple
>>> x86_64-unknown-linux-gnu \
>>> +// RUN:   -flto=thin -emit-llvm-bc \
>>> +// RUN:   -o %t.o %s
>>> +
>>> +// RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
>>> +// RUN:   -o %t2.index \
>>> +// RUN:   -r=%t.o,main,px
>>> +
>>> +// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu \
>>> +// RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
>>> +// RUN:   -o %t.native.o -split-dwarf-file %t.native.dwo -x ir %t.o
>>>
>>
>> Can this be written in a single IR file yet (rather than frontend
>> compiling, indexing, then backend compiling), now that Teresa's implemented
>> some of the summary IR syntax?
>>
>
> I think the parsing part isn't implemented yet, so that wouldn't work.
>

Regarding this - the parsing support is coming along, slowly but surely.
I'd say I am about 75% done.
Teresa


> Besides, we would need two separate files anyway: the bitcode file (%t.o)
> and the combined summary (%t.o.thinlto.bc).
>
> Peter
>
>
>>
>>
>>> +
>>> +// RUN: llvm-readobj -sections %t.native.o | FileCheck --check-prefix=O
>>> %s
>>> +// RUN: llvm-readobj -sections %t.native.dwo | FileCheck
>>> --check-prefix=DWO %s
>>> +
>>> +// O-NOT: .dwo
>>> +// DWO: .dwo
>>> +
>>> +int main() {}
>>> Index: clang/lib/CodeGen/BackendUtil.cpp
>>> ===
>>> --- clang/lib/CodeGen/BackendUtil.cpp
>>> +++ clang/lib/CodeGen/BackendUtil.cpp
>>> @@ -1161,6 +1161,7 @@
>>>Conf.DebugPassManager = CGOpts.DebugPassManager;
>>>Conf.RemarksWithHotness = CGOpts.DiagnosticsWithHotness;
>>>Conf.RemarksFilename = CGOpts.OptRecordFile;
>>> +  Conf.DwoPath = CGOpts.SplitDwarfFile;
>>>switch (Action) {
>>>case Backend_EmitNothing:
>>>  Conf.PreCodeGenModuleHook = [](size_t Task, const Module &Mod) {
>>>
>>>
>>>
>
>
> --
> --
> Peter
>


-- 
Teresa Johnson |  Software Engineer |  tejohn...@google.com |  408-460-2413
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r333966 - [ThinLTO] Add testing of new summary index format to a couple CFI tests

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Mon Jun  4 16:05:24 2018
New Revision: 333966

URL: http://llvm.org/viewvc/llvm-project?rev=333966&view=rev
Log:
[ThinLTO] Add testing of new summary index format to a couple CFI tests

Summary:
Adds testing of combined index summary entries in disassembly format
to CFI tests that were already testing the bitcode format.

Depends on D46699.

Reviewers: pcc

Subscribers: mehdi_amini, inglorion, eraman, cfe-commits

Differential Revision: https://reviews.llvm.org/D46700

Modified:
cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll

Modified: cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll?rev=333966&r1=333965&r2=333966&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll (original)
+++ cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll Mon Jun  4 
16:05:24 2018
@@ -29,6 +29,11 @@
 ; CHECK: blob data = '_ZTS1A_ZN1A1nEi'
 ; CHECK-LABEL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll?rev=333966&r1=333965&r2=333966&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll (original)
+++ cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll Mon Jun  4 16:05:24 2018
@@ -20,6 +20,11 @@
 ; CHECK: blob data = '_ZTS1A'
 ; CHECK-LABEL: http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r350949 - [LTO] Add option to enable LTOUnit splitting, and disable unless needed

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Fri Jan 11 10:32:07 2019
New Revision: 350949

URL: http://llvm.org/viewvc/llvm-project?rev=350949&view=rev
Log:
[LTO] Add option to enable LTOUnit splitting, and disable unless needed

Summary:
Adds a new -f[no]split-lto-unit flag that is disabled by default to
control module splitting during ThinLTO. It is automatically enabled
for -fsanitize=cfi and -fwhole-program-vtables.

The new EnableSplitLTOUnit codegen flag is passed down to llvm
via a new module flag of the same name.

Depends on D53890.

Reviewers: pcc

Subscribers: ormris, mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, 
cfe-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D53891

Added:
cfe/trunk/test/Driver/split-lto-unit.c
Modified:
cfe/trunk/include/clang/Basic/CodeGenOptions.def
cfe/trunk/include/clang/Driver/Options.td
cfe/trunk/include/clang/Driver/SanitizerArgs.h
cfe/trunk/lib/CodeGen/BackendUtil.cpp
cfe/trunk/lib/Driver/SanitizerArgs.cpp
cfe/trunk/lib/Driver/ToolChains/Clang.cpp
cfe/trunk/lib/Frontend/CompilerInvocation.cpp
cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll
cfe/trunk/test/CodeGenCXX/no-lto-unit.cpp
cfe/trunk/test/CodeGenCXX/type-metadata-thinlto.cpp

Modified: cfe/trunk/include/clang/Basic/CodeGenOptions.def
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/CodeGenOptions.def?rev=350949&r1=350948&r2=350949&view=diff
==
--- cfe/trunk/include/clang/Basic/CodeGenOptions.def (original)
+++ cfe/trunk/include/clang/Basic/CodeGenOptions.def Fri Jan 11 10:32:07 2019
@@ -116,6 +116,10 @@ CODEGENOPT(PrepareForThinLTO , 1, 0) ///
  ///< compile step.
 CODEGENOPT(LTOUnit, 1, 0) ///< Emit IR to support LTO unit features (CFI, whole
   ///< program vtable opt).
+CODEGENOPT(EnableSplitLTOUnit, 1, 0) ///< Enable LTO unit splitting to support
+/// CFI and traditional whole program
+/// devirtualization that require whole
+/// program IR support.
 CODEGENOPT(IncrementalLinkerCompatible, 1, 0) ///< Emit an object file which 
can
   ///< be used with an incremental
   ///< linker.

Modified: cfe/trunk/include/clang/Driver/Options.td
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Options.td?rev=350949&r1=350948&r2=350949&view=diff
==
--- cfe/trunk/include/clang/Driver/Options.td (original)
+++ cfe/trunk/include/clang/Driver/Options.td Fri Jan 11 10:32:07 2019
@@ -1776,6 +1776,11 @@ def fwhole_program_vtables : Flag<["-"],
   HelpText<"Enables whole-program vtable optimization. Requires -flto">;
 def fno_whole_program_vtables : Flag<["-"], "fno-whole-program-vtables">, 
Group,
   Flags<[CoreOption]>;
+def fsplit_lto_unit : Flag<["-"], "fsplit-lto-unit">, Group,
+  Flags<[CoreOption, CC1Option]>,
+  HelpText<"Enables splitting of the LTO unit.">;
+def fno_split_lto_unit : Flag<["-"], "fno-split-lto-unit">, Group,
+  Flags<[CoreOption]>;
 def fforce_emit_vtables : Flag<["-"], "fforce-emit-vtables">, Group,
 Flags<[CC1Option]>,
 HelpText<"Emits more virtual tables to improve devirtualization">;

Modified: cfe/trunk/include/clang/Driver/SanitizerArgs.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/SanitizerArgs.h?rev=350949&r1=350948&r2=350949&view=diff
==
--- cfe/trunk/include/clang/Driver/SanitizerArgs.h (original)
+++ cfe/trunk/include/clang/Driver/SanitizerArgs.h Fri Jan 11 10:32:07 2019
@@ -81,6 +81,7 @@ class SanitizerArgs {
 
   bool requiresPIE() const;
   bool needsUnwindTables() const;
+  bool needsLTO() const;
   bool linkCXXRuntimes() const { return LinkCXXRuntimes; }
   bool hasCrossDsoCfi() const { return CfiCrossDso; }
   bool hasAnySanitizer() const { return !Sanitizers.empty(); }

Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=350949&r1=350948&r2=350949&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Fri Jan 11 10:32:07 2019
@@ -814,6 +814,8 @@ void EmitAssemblyHelper::EmitAssembly(Ba
 if (!ThinLinkOS)
   return;
   }
+  TheModule->addModuleFlag(Module::Error, "EnableSplitLTOUnit",
+   CodeGenOpts.EnableSplitLTOUnit);
   PerModulePasses.add(createWriteThinLTOBitcodePass(
   *OS, ThinLinkOS ? &ThinLinkOS->os() : nullptr));
 } else {
@

r347887 - [ThinLTO] Allow importing of multiple symbols with same GUID

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Thu Nov 29 09:02:59 2018
New Revision: 347887

URL: http://llvm.org/viewvc/llvm-project?rev=347887&view=rev
Log:
[ThinLTO] Allow importing of multiple symbols with same GUID

Summary:
The is the clang side of the fix in D55047, to handle the case where
two different modules have local variables with the same GUID because
they had the same source file name at compilation time. Allow multiple
symbols with the same GUID to be imported, and test that this case works
with the distributed backend path.

Depends on D55047.

Reviewers: evgeny777

Subscribers: mehdi_amini, inglorion, eraman, steven_wu, dexonsmith, cfe-commits

Differential Revision: https://reviews.llvm.org/D55048

Added:
cfe/trunk/test/CodeGen/Inputs/thinlto_backend_local_name_conflict1.ll
cfe/trunk/test/CodeGen/Inputs/thinlto_backend_local_name_conflict2.ll
cfe/trunk/test/CodeGen/thinlto_backend_local_name_conflict.ll
Modified:
cfe/trunk/lib/CodeGen/BackendUtil.cpp

Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=347887&r1=347886&r2=347887&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Thu Nov 29 09:02:59 2018
@@ -1155,15 +1155,14 @@ static void runThinLTOBackend(ModuleSumm
   continue;
 
 auto GUID = GlobalList.first;
-assert(GlobalList.second.SummaryList.size() == 1 &&
-   "Expected individual combined index to have one summary per GUID");
-auto &Summary = GlobalList.second.SummaryList[0];
-// Skip the summaries for the importing module. These are included to
-// e.g. record required linkage changes.
-if (Summary->modulePath() == M->getModuleIdentifier())
-  continue;
-// Add an entry to provoke importing by thinBackend.
-ImportList[Summary->modulePath()].insert(GUID);
+for (auto &Summary : GlobalList.second.SummaryList) {
+  // Skip the summaries for the importing module. These are included to
+  // e.g. record required linkage changes.
+  if (Summary->modulePath() == M->getModuleIdentifier())
+continue;
+  // Add an entry to provoke importing by thinBackend.
+  ImportList[Summary->modulePath()].insert(GUID);
+}
   }
 
   std::vector> OwnedImports;

Added: cfe/trunk/test/CodeGen/Inputs/thinlto_backend_local_name_conflict1.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/Inputs/thinlto_backend_local_name_conflict1.ll?rev=347887&view=auto
==
--- cfe/trunk/test/CodeGen/Inputs/thinlto_backend_local_name_conflict1.ll 
(added)
+++ cfe/trunk/test/CodeGen/Inputs/thinlto_backend_local_name_conflict1.ll Thu 
Nov 29 09:02:59 2018
@@ -0,0 +1,13 @@
+; ModuleID = 'local_name_conflict_var.o'
+source_filename = "local_name_conflict_var.c"
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@baz = internal global i32 10, align 4
+
+; Function Attrs: noinline nounwind uwtable
+define i32 @a() {
+entry:
+  %0 = load i32, i32* @baz, align 4
+  ret i32 %0
+}

Added: cfe/trunk/test/CodeGen/Inputs/thinlto_backend_local_name_conflict2.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/Inputs/thinlto_backend_local_name_conflict2.ll?rev=347887&view=auto
==
--- cfe/trunk/test/CodeGen/Inputs/thinlto_backend_local_name_conflict2.ll 
(added)
+++ cfe/trunk/test/CodeGen/Inputs/thinlto_backend_local_name_conflict2.ll Thu 
Nov 29 09:02:59 2018
@@ -0,0 +1,13 @@
+; ModuleID = 'local_name_conflict_var.o'
+source_filename = "local_name_conflict_var.c"
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@baz = internal global i32 10, align 4
+
+; Function Attrs: noinline nounwind uwtable
+define i32 @b() {
+entry:
+  %0 = load i32, i32* @baz, align 4
+  ret i32 %0
+}

Added: cfe/trunk/test/CodeGen/thinlto_backend_local_name_conflict.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto_backend_local_name_conflict.ll?rev=347887&view=auto
==
--- cfe/trunk/test/CodeGen/thinlto_backend_local_name_conflict.ll (added)
+++ cfe/trunk/test/CodeGen/thinlto_backend_local_name_conflict.ll Thu Nov 29 
09:02:59 2018
@@ -0,0 +1,34 @@
+; Test handling when two files with the same source file name contain
+; static read only variables with the same name (which will have the same GUID
+; in the combined index).
+
+; Do setup work for all below tests: generate bitcode and combined index
+; RUN: opt -module-summary -module-hash %s -o %t.bc
+; RUN: opt -module-summary -module-hash 
%p/Inputs/thinlto_backend_local_name_conflict1.ll -o %t2.bc
+; RUN: opt -module-summary -modu

r347892 - Add missing REQUIRES to new test

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Thu Nov 29 10:02:31 2018
New Revision: 347892

URL: http://llvm.org/viewvc/llvm-project?rev=347892&view=rev
Log:
Add missing REQUIRES to new test

Test added in r347887 requires an x86 target.

Modified:
cfe/trunk/test/CodeGen/thinlto_backend_local_name_conflict.ll

Modified: cfe/trunk/test/CodeGen/thinlto_backend_local_name_conflict.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto_backend_local_name_conflict.ll?rev=347892&r1=347891&r2=347892&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto_backend_local_name_conflict.ll (original)
+++ cfe/trunk/test/CodeGen/thinlto_backend_local_name_conflict.ll Thu Nov 29 
10:02:31 2018
@@ -2,6 +2,8 @@
 ; static read only variables with the same name (which will have the same GUID
 ; in the combined index).
 
+; REQUIRES: x86-registered-target
+
 ; Do setup work for all below tests: generate bitcode and combined index
 ; RUN: opt -module-summary -module-hash %s -o %t.bc
 ; RUN: opt -module-summary -module-hash 
%p/Inputs/thinlto_backend_local_name_conflict1.ll -o %t2.bc


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r359025 - [ThinLTO] Pass down opt level to LTO backend and handle -O0 LTO in new PM

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Tue Apr 23 11:56:19 2019
New Revision: 359025

URL: http://llvm.org/viewvc/llvm-project?rev=359025&view=rev
Log:
[ThinLTO] Pass down opt level to LTO backend and handle -O0 LTO in new PM

Summary:
The opt level was not being passed down to the ThinLTO backend when
invoked via clang (for distributed ThinLTO).

This exposed an issue where the new PM was asserting if the Thin or
regular LTO backend pipelines were invoked with -O0 (not a new issue,
could be provoked by invoking in-process *LTO backends via linker using
new PM and -O0). Fix this similar to the old PM where -O0 only does the
necessary lowering of type metadata (WPD and LowerTypeTest passes) and
then quits, rather than asserting.

Reviewers: xur

Subscribers: mehdi_amini, inglorion, eraman, hiraditya, steven_wu, dexonsmith, 
cfe-commits, llvm-commits, pcc

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D61022

Modified:
cfe/trunk/lib/CodeGen/BackendUtil.cpp
cfe/trunk/test/CodeGen/thinlto-debug-pm.c
cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll

Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=359025&r1=359024&r2=359025&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Tue Apr 23 11:56:19 2019
@@ -1325,6 +1325,7 @@ static void runThinLTOBackend(ModuleSumm
   Conf.MAttrs = TOpts.Features;
   Conf.RelocModel = CGOpts.RelocationModel;
   Conf.CGOptLevel = getCGOptLevel(CGOpts);
+  Conf.OptLevel = CGOpts.OptimizationLevel;
   initTargetOptions(Conf.Options, CGOpts, TOpts, LOpts, HeaderOpts);
   Conf.SampleProfile = std::move(SampleProfile);
 

Modified: cfe/trunk/test/CodeGen/thinlto-debug-pm.c
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-debug-pm.c?rev=359025&r1=359024&r2=359025&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto-debug-pm.c (original)
+++ cfe/trunk/test/CodeGen/thinlto-debug-pm.c Tue Apr 23 11:56:19 2019
@@ -1,10 +1,17 @@
-// Test to ensure -fdebug-pass-manager works when invoking the
-// ThinLTO backend path with the new PM.
+// Test to ensure the opt level is passed down to the ThinLTO backend.
 // REQUIRES: x86-registered-target
 // RUN: %clang_cc1 -o %t.o -flto=thin -fexperimental-new-pass-manager -triple 
x86_64-unknown-linux-gnu -emit-llvm-bc %s
 // RUN: llvm-lto -thinlto -o %t %t.o
-// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-obj -O2 -o %t2.o -x 
ir %t.o -fthinlto-index=%t.thinlto.bc -fdebug-pass-manager 
-fexperimental-new-pass-manager 2>&1 | FileCheck %s
-// CHECK: Running pass:
+
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-obj -O2 -o %t2.o -x 
ir %t.o -fthinlto-index=%t.thinlto.bc -fdebug-pass-manager 
-fexperimental-new-pass-manager 2>&1 | FileCheck %s --check-prefix=O2-NEWPM
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-obj -O0 -o %t2.o -x 
ir %t.o -fthinlto-index=%t.thinlto.bc -fdebug-pass-manager 
-fexperimental-new-pass-manager 2>&1 | FileCheck %s --check-prefix=O0-NEWPM
+// O2-NEWPM: Running pass: LoopVectorizePass
+// O0-NEWPM-NOT: Running pass: LoopVectorizePass
+
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-obj -O2 -o %t2.o -x 
ir %t.o -fthinlto-index=%t.thinlto.bc -mllvm -debug-pass=Structure 2>&1 | 
FileCheck %s --check-prefix=O2-OLDPM
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-obj -O0 -o %t2.o -x 
ir %t.o -fthinlto-index=%t.thinlto.bc -mllvm -debug-pass=Structure 2>&1 | 
FileCheck %s --check-prefix=O0-OLDPM
+// O2-OLDPM: Loop Vectorization
+// O0-OLDPM-NOT: Loop Vectorization
 
 void foo() {
 }

Modified: cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll?rev=359025&r1=359024&r2=359025&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll (original)
+++ cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll Tue Apr 23 
11:56:19 2019
@@ -39,12 +39,12 @@
 ; CHECK-DIS: ^2 = typeid: (name: "_ZTS1A", summary: (typeTestRes: (kind: 
allOnes, sizeM1BitWidth: 7), wpdResolutions: ((offset: 0, wpdRes: (kind: 
branchFunnel)), (offset: 8, wpdRes: (kind: singleImpl, singleImplName: 
"_ZN1A1nEi") ; guid = 7004155349499253778
 
 ; RUN: %clang_cc1 -triple x86_64-grtev4-linux-gnu \
-; RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
+; RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc -O2 \
 ; RUN:   -emit-llvm -o - -x ir %t.o | FileCheck %s --check-prefixes=CHECK-IR
 
 ; Check that backend does not fail generating native code.
 ; RUN: %clang_cc1 -triple x86_64-grtev4-linux-gnu \
-; RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
+; RUN:   -emit-obj -fthinlto-index=%t.o.thi

r366623 - [LTO] Always mark regular LTO units with EnableSplitLTOUnit=1

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Fri Jul 19 16:02:58 2019
New Revision: 366623

URL: http://llvm.org/viewvc/llvm-project?rev=366623&view=rev
Log:
[LTO] Always mark regular LTO units with EnableSplitLTOUnit=1

Summary:
Regular LTO modules do not need LTO Unit splitting, only ThinLTO does
(they must be consistently split into regular and Thin units for
optimizations such as whole program devirtualization and lower type
tests). In order to avoid spurious errors from LTO when combining with
split ThinLTO modules, always set this flag for regular LTO modules.

Reviewers: pcc

Subscribers: mehdi_amini, Prazek, inglorion, steven_wu, dexonsmith, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D65009

Added:
cfe/trunk/test/CodeGen/split-lto-unit.c
Modified:
cfe/trunk/lib/CodeGen/BackendUtil.cpp

Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=366623&r1=366622&r2=366623&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Fri Jul 19 16:02:58 2019
@@ -852,7 +852,7 @@ void EmitAssemblyHelper::EmitAssembly(Ba
 if (!TheModule->getModuleFlag("ThinLTO"))
   TheModule->addModuleFlag(Module::Error, "ThinLTO", uint32_t(0));
 TheModule->addModuleFlag(Module::Error, "EnableSplitLTOUnit",
- CodeGenOpts.EnableSplitLTOUnit);
+ uint32_t(1));
   }
 
   PerModulePasses.add(createBitcodeWriterPass(

Added: cfe/trunk/test/CodeGen/split-lto-unit.c
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/split-lto-unit.c?rev=366623&view=auto
==
--- cfe/trunk/test/CodeGen/split-lto-unit.c (added)
+++ cfe/trunk/test/CodeGen/split-lto-unit.c Fri Jul 19 16:02:58 2019
@@ -0,0 +1,12 @@
+// ; Check that -flto=thin without -fsplit-lto-unit has EnableSplitLTOUnit = 0
+// RUN: %clang_cc1 -flto=thin -emit-llvm-bc < %s | llvm-dis -o - | FileCheck %s
+// CHECK: !{i32 1, !"EnableSplitLTOUnit", i32 0}
+//
+// ; Check that -flto=thin with -fsplit-lto-unit has EnableSplitLTOUnit = 1
+// RUN: %clang_cc1 -flto=thin -fsplit-lto-unit -emit-llvm-bc < %s | llvm-dis 
-o - | FileCheck %s --check-prefix=SPLIT
+// SPLIT: !{i32 1, !"EnableSplitLTOUnit", i32 1}
+//
+// ; Check that regular LTO has EnableSplitLTOUnit = 1
+// RUN: %clang_cc1 -flto -triple x86_64-pc-linux-gnu -emit-llvm-bc < %s | 
llvm-dis -o - | FileCheck %s --implicit-check-not="EnableSplitLTOUnit" 
--check-prefix=SPLIT
+
+int main() {}


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r360468 - [ThinLTO] Clang test changes for new CanAutoHide flag

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Fri May 10 13:38:31 2019
New Revision: 360468

URL: http://llvm.org/viewvc/llvm-project?rev=360468&view=rev
Log:
[ThinLTO] Clang test changes for new CanAutoHide flag

Modified:
cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll

Modified: cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll?rev=360468&r1=360467&r2=360468&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll (original)
+++ cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll Fri May 10 
13:38:31 2019
@@ -35,7 +35,7 @@
 ; Round trip it through llvm-as
 ; RUN: llvm-dis %t.o.thinlto.bc -o - | llvm-as -o - | llvm-dis -o - | 
FileCheck %s --check-prefix=CHECK-DIS
 ; CHECK-DIS: ^0 = module: (path: 
"{{.*}}thinlto-distributed-cfi-devirt.ll.tmp.o", hash: ({{.*}}, {{.*}}, {{.*}}, 
{{.*}}, {{.*}}))
-; CHECK-DIS: ^1 = gv: (guid: 8346051122425466633, summaries: (function: 
(module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 1, 
dsoLocal: 0), insts: 18, typeIdInfo: (typeTests: (^2), typeCheckedLoadVCalls: 
(vFuncId: (^2, offset: 8), vFuncId: (^2, offset: 0))
+; CHECK-DIS: ^1 = gv: (guid: 8346051122425466633, summaries: (function: 
(module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 1, 
dsoLocal: 0, canAutoHide: 0), insts: 18, typeIdInfo: (typeTests: (^2), 
typeCheckedLoadVCalls: (vFuncId: (^2, offset: 8), vFuncId: (^2, offset: 0))
 ; CHECK-DIS: ^2 = typeid: (name: "_ZTS1A", summary: (typeTestRes: (kind: 
allOnes, sizeM1BitWidth: 7), wpdResolutions: ((offset: 0, wpdRes: (kind: 
branchFunnel)), (offset: 8, wpdRes: (kind: singleImpl, singleImplName: 
"_ZN1A1nEi") ; guid = 7004155349499253778
 
 ; RUN: %clang_cc1 -triple x86_64-grtev4-linux-gnu \

Modified: cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll?rev=360468&r1=360467&r2=360468&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll (original)
+++ cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll Fri May 10 13:38:31 2019
@@ -24,7 +24,7 @@
 ; Round trip it through llvm-as
 ; RUN: llvm-dis %t.o.thinlto.bc -o - | llvm-as -o - | llvm-dis -o - | 
FileCheck %s --check-prefix=CHECK-DIS
 ; CHECK-DIS: ^0 = module: (path: "{{.*}}thinlto-distributed-cfi.ll.tmp.o", 
hash: ({{.*}}, {{.*}}, {{.*}}, {{.*}}, {{.*}}))
-; CHECK-DIS: ^1 = gv: (guid: 8346051122425466633, summaries: (function: 
(module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 1, 
dsoLocal: 0), insts: 7, typeIdInfo: (typeTests: (^2)
+; CHECK-DIS: ^1 = gv: (guid: 8346051122425466633, summaries: (function: 
(module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 1, 
dsoLocal: 0, canAutoHide: 0), insts: 7, typeIdInfo: (typeTests: (^2)
 ; CHECK-DIS: ^2 = typeid: (name: "_ZTS1A", summary: (typeTestRes: (kind: 
single, sizeM1BitWidth: 0))) ; guid = 7004155349499253778
 
 ; RUN: %clang_cc1 -triple x86_64-grtev4-linux-gnu \


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r350424 - [ThinLTO] Clang changes to utilize new pass to handle chains of aliases

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Fri Jan  4 11:05:01 2019
New Revision: 350424

URL: http://llvm.org/viewvc/llvm-project?rev=350424&view=rev
Log:
[ThinLTO] Clang changes to utilize new pass to handle chains of aliases

Summary:
As with NameAnonGlobals, invoke the new CanonicalizeAliases via clang
when using the new PM.

Depends on D54507.

Reviewers: pcc, davidxl

Subscribers: mehdi_amini, inglorion, steven_wu, dexonsmith, cfe-commits

Differential Revision: https://reviews.llvm.org/D55620

Modified:
cfe/trunk/lib/CodeGen/BackendUtil.cpp
cfe/trunk/test/CodeGen/lto-newpm-pipeline.c

Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=350424&r1=350423&r2=350424&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Fri Jan  4 11:05:01 2019
@@ -60,6 +60,7 @@
 #include "llvm/Transforms/Scalar.h"
 #include "llvm/Transforms/Scalar/GVN.h"
 #include "llvm/Transforms/Utils.h"
+#include "llvm/Transforms/Utils/CanonicalizeAliases.h"
 #include "llvm/Transforms/Utils/NameAnonGlobals.h"
 #include "llvm/Transforms/Utils/SymbolRewriter.h"
 #include 
@@ -996,9 +997,11 @@ void EmitAssemblyHelper::EmitAssemblyWit
   if (LangOpts.Sanitize.has(SanitizerKind::LocalBounds))
 MPM.addPass(createModuleToFunctionPassAdaptor(BoundsCheckingPass()));
 
-  // Lastly, add a semantically necessary pass for LTO.
-  if (IsLTO || IsThinLTO)
+  // Lastly, add semantically necessary passes for LTO.
+  if (IsLTO || IsThinLTO) {
+MPM.addPass(CanonicalizeAliasesPass());
 MPM.addPass(NameAnonGlobalPass());
+  }
 } else {
   // Map our optimization levels into one of the distinct levels used to
   // configure the pipeline.
@@ -1019,10 +1022,12 @@ void EmitAssemblyHelper::EmitAssemblyWit
   if (IsThinLTO) {
 MPM = PB.buildThinLTOPreLinkDefaultPipeline(
 Level, CodeGenOpts.DebugPassManager);
+MPM.addPass(CanonicalizeAliasesPass());
 MPM.addPass(NameAnonGlobalPass());
   } else if (IsLTO) {
 MPM = PB.buildLTOPreLinkDefaultPipeline(Level,
 CodeGenOpts.DebugPassManager);
+MPM.addPass(CanonicalizeAliasesPass());
 MPM.addPass(NameAnonGlobalPass());
   } else {
 MPM = PB.buildPerModuleDefaultPipeline(Level,

Modified: cfe/trunk/test/CodeGen/lto-newpm-pipeline.c
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/lto-newpm-pipeline.c?rev=350424&r1=350423&r2=350424&view=diff
==
--- cfe/trunk/test/CodeGen/lto-newpm-pipeline.c (original)
+++ cfe/trunk/test/CodeGen/lto-newpm-pipeline.c Fri Jan  4 11:05:01 2019
@@ -27,12 +27,14 @@
 
 // CHECK-FULL-O0: Starting llvm::Module pass manager run.
 // CHECK-FULL-O0: Running pass: AlwaysInlinerPass
+// CHECK-FULL-O0-NEXT: Running pass: CanonicalizeAliasesPass
 // CHECK-FULL-O0-NEXT: Running pass: NameAnonGlobalPass
 // CHECK-FULL-O0-NEXT: Running pass: BitcodeWriterPass
 // CHECK-FULL-O0: Finished llvm::Module pass manager run.
 
 // CHECK-THIN-O0: Starting llvm::Module pass manager run.
 // CHECK-THIN-O0: Running pass: AlwaysInlinerPass
+// CHECK-THIN-O0-NEXT: Running pass: CanonicalizeAliasesPass
 // CHECK-THIN-O0-NEXT: Running pass: NameAnonGlobalPass
 // CHECK-THIN-O0-NEXT: Running pass: ThinLTOBitcodeWriterPass
 // CHECK-THIN-O0: Finished llvm::Module pass manager run.
@@ -48,6 +50,7 @@
 // LoopVectorizePass.
 // CHECK-THIN-OPTIMIZED: Starting llvm::Function pass manager run.
 // CHECK-THIN-OPTIMIZED-NOT: Running pass: LoopVectorizePass
+// CHECK-THIN-OPTIMIZED: Running pass: CanonicalizeAliasesPass
 // CHECK-THIN-OPTIMIZED: Running pass: NameAnonGlobalPass
 // CHECK-THIN-OPTIMIZED: Running pass: ThinLTOBitcodeWriterPass
 


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r373217 - [Clang] Use -main-file-name for source filename if not set

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Mon Sep 30 08:05:35 2019
New Revision: 373217

URL: http://llvm.org/viewvc/llvm-project?rev=373217&view=rev
Log:
[Clang] Use -main-file-name for source filename if not set

-main-file-name is currently used to set the source name used in debug
information.

If the source filename is "-" and -main-file-name is set, then use the
filename also for source_filename and ModuleID of the output.

The argument is generally used outside the internal clang calls when
running clang in a wrapper like icecc which gives the source via stdin
but still wants to get a object file with the original source filename
both in debug info and IR code.

Patch by: the_jk (Joel Klinghed)

Differential Revision: https://reviews.llvm.org/D67592

Added:
cfe/trunk/test/Frontend/stdin-input.c
Modified:
cfe/trunk/include/clang/Driver/CC1Options.td
cfe/trunk/lib/CodeGen/ModuleBuilder.cpp

Modified: cfe/trunk/include/clang/Driver/CC1Options.td
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/CC1Options.td?rev=373217&r1=373216&r2=373217&view=diff
==
--- cfe/trunk/include/clang/Driver/CC1Options.td (original)
+++ cfe/trunk/include/clang/Driver/CC1Options.td Mon Sep 30 08:05:35 2019
@@ -687,7 +687,7 @@ let Flags = [CC1Option, CC1AsOption, NoD
 def version : Flag<["-"], "version">,
   HelpText<"Print the compiler version">;
 def main_file_name : Separate<["-"], "main-file-name">,
-  HelpText<"Main file name to use for debug info">;
+  HelpText<"Main file name to use for debug info and source if missing">;
 def split_dwarf_output : Separate<["-"], "split-dwarf-output">,
   HelpText<"File name to use for split dwarf debug info output">;
 

Modified: cfe/trunk/lib/CodeGen/ModuleBuilder.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/ModuleBuilder.cpp?rev=373217&r1=373216&r2=373217&view=diff
==
--- cfe/trunk/lib/CodeGen/ModuleBuilder.cpp (original)
+++ cfe/trunk/lib/CodeGen/ModuleBuilder.cpp Mon Sep 30 08:05:35 2019
@@ -65,6 +65,13 @@ namespace {
   private:
 SmallVector DeferredInlineMemberFuncDefs;
 
+static llvm::StringRef ExpandModuleName(llvm::StringRef ModuleName,
+const CodeGenOptions &CGO) {
+  if (ModuleName == "-" && !CGO.MainFileName.empty())
+return CGO.MainFileName;
+  return ModuleName;
+}
+
   public:
 CodeGeneratorImpl(DiagnosticsEngine &diags, llvm::StringRef ModuleName,
   const HeaderSearchOptions &HSO,
@@ -73,7 +80,8 @@ namespace {
   CoverageSourceInfo *CoverageInfo = nullptr)
 : Diags(diags), Ctx(nullptr), HeaderSearchOpts(HSO),
   PreprocessorOpts(PPO), CodeGenOpts(CGO), HandlingTopLevelDecls(0),
-  CoverageInfo(CoverageInfo), M(new llvm::Module(ModuleName, C)) {
+  CoverageInfo(CoverageInfo),
+  M(new llvm::Module(ExpandModuleName(ModuleName, CGO), C)) {
   C.setDiscardValueNames(CGO.DiscardValueNames);
 }
 
@@ -121,7 +129,7 @@ namespace {
 llvm::Module *StartModule(llvm::StringRef ModuleName,
   llvm::LLVMContext &C) {
   assert(!M && "Replacing existing Module?");
-  M.reset(new llvm::Module(ModuleName, C));
+  M.reset(new llvm::Module(ExpandModuleName(ModuleName, CodeGenOpts), C));
   Initialize(*Ctx);
   return M.get();
 }

Added: cfe/trunk/test/Frontend/stdin-input.c
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Frontend/stdin-input.c?rev=373217&view=auto
==
--- cfe/trunk/test/Frontend/stdin-input.c (added)
+++ cfe/trunk/test/Frontend/stdin-input.c Mon Sep 30 08:05:35 2019
@@ -0,0 +1,7 @@
+// RUN: cat %s | %clang -emit-llvm -g -S \
+// RUN: -Xclang -main-file-name -Xclang test/foo.c -x c - -o - | FileCheck %s
+// CHECK: ; ModuleID = 'test/foo.c'
+// CHECK: source_filename = "test/foo.c"
+// CHECK: !1 = !DIFile(filename: "test/foo.c"
+
+int main() {}


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r373237 - Fix buildbot failure from r373217 (don't match metadata id exactly)

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Mon Sep 30 10:26:48 2019
New Revision: 373237

URL: http://llvm.org/viewvc/llvm-project?rev=373237&view=rev
Log:
Fix buildbot failure from r373217 (don't match metadata id exactly)

Fix this failure by ignoring the id of the metadata being checked:

http://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-incremental/3046/consoleFull#-21332887158254eaf0-7326-4999-85b0-388101f2d404

Modified:
cfe/trunk/test/Frontend/stdin-input.c

Modified: cfe/trunk/test/Frontend/stdin-input.c
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Frontend/stdin-input.c?rev=373237&r1=373236&r2=373237&view=diff
==
--- cfe/trunk/test/Frontend/stdin-input.c (original)
+++ cfe/trunk/test/Frontend/stdin-input.c Mon Sep 30 10:26:48 2019
@@ -2,6 +2,6 @@
 // RUN: -Xclang -main-file-name -Xclang test/foo.c -x c - -o - | FileCheck %s
 // CHECK: ; ModuleID = 'test/foo.c'
 // CHECK: source_filename = "test/foo.c"
-// CHECK: !1 = !DIFile(filename: "test/foo.c"
+// CHECK: !DIFile(filename: "test/foo.c"
 
 int main() {}


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r373370 - [ThinLTO] Enable index-only WPD from clang

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Tue Oct  1 11:08:29 2019
New Revision: 373370

URL: http://llvm.org/viewvc/llvm-project?rev=373370&view=rev
Log:
[ThinLTO] Enable index-only WPD from clang

Summary:
To trigger the index-only Whole Program Devirt support added to LLVM, we
need to be able to specify -fno-split-lto-unit in conjunction with
-fwhole-program-vtables. Keep the default for -fwhole-program-vtables as
-fsplit-lto-unit, but don't error on that option combination.

Reviewers: pcc

Subscribers: mehdi_amini, inglorion, steven_wu, dexonsmith, arphaman, 
cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68029

Modified:
cfe/trunk/lib/Driver/ToolChains/Clang.cpp
cfe/trunk/test/Driver/split-lto-unit.c

Modified: cfe/trunk/lib/Driver/ToolChains/Clang.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/ToolChains/Clang.cpp?rev=373370&r1=373369&r2=373370&view=diff
==
--- cfe/trunk/lib/Driver/ToolChains/Clang.cpp (original)
+++ cfe/trunk/lib/Driver/ToolChains/Clang.cpp Tue Oct  1 11:08:29 2019
@@ -5456,14 +5456,13 @@ void Clang::ConstructJob(Compilation &C,
 CmdArgs.push_back("-fwhole-program-vtables");
   }
 
-  bool RequiresSplitLTOUnit = WholeProgramVTables || Sanitize.needsLTO();
+  bool DefaultsSplitLTOUnit = WholeProgramVTables || Sanitize.needsLTO();
   bool SplitLTOUnit =
   Args.hasFlag(options::OPT_fsplit_lto_unit,
-   options::OPT_fno_split_lto_unit, RequiresSplitLTOUnit);
-  if (RequiresSplitLTOUnit && !SplitLTOUnit)
-D.Diag(diag::err_drv_argument_not_allowed_with)
-<< "-fno-split-lto-unit"
-<< (WholeProgramVTables ? "-fwhole-program-vtables" : 
"-fsanitize=cfi");
+   options::OPT_fno_split_lto_unit, DefaultsSplitLTOUnit);
+  if (Sanitize.needsLTO() && !SplitLTOUnit)
+D.Diag(diag::err_drv_argument_not_allowed_with) << "-fno-split-lto-unit"
+<< "-fsanitize=cfi";
   if (SplitLTOUnit)
 CmdArgs.push_back("-fsplit-lto-unit");
 

Modified: cfe/trunk/test/Driver/split-lto-unit.c
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/split-lto-unit.c?rev=373370&r1=373369&r2=373370&view=diff
==
--- cfe/trunk/test/Driver/split-lto-unit.c (original)
+++ cfe/trunk/test/Driver/split-lto-unit.c Tue Oct  1 11:08:29 2019
@@ -6,5 +6,5 @@
 
 // UNIT: "-fsplit-lto-unit"
 // NOUNIT-NOT: "-fsplit-lto-unit"
-// ERROR1: error: invalid argument '-fno-split-lto-unit' not allowed with 
'-fwhole-program-vtables'
+// ERROR1-NOT: error: invalid argument
 // ERROR2: error: invalid argument '-fno-split-lto-unit' not allowed with 
'-fsanitize=cfi'


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r330194 - [ThinLTO] Pass -save-temps to LTO backend for distributed ThinLTO builds

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Tue Apr 17 09:39:25 2018
New Revision: 330194

URL: http://llvm.org/viewvc/llvm-project?rev=330194&view=rev
Log:
[ThinLTO] Pass -save-temps to LTO backend for distributed ThinLTO builds

Summary:
The clang driver option -save-temps was not passed to the LTO config,
so when invoking the ThinLTO backends via clang during distributed
builds there was no way to get LTO to save temp files.

Getting this to work with ThinLTO distributed builds also required
changing the driver to avoid a separate compile step to emit unoptimized
bitcode when the input was already bitcode under -save-temps. Not only is
this unnecessary in general, it is problematic for ThinLTO backends since
the temporary bitcode file to the backend would not match the module path
in the combined index, leading to incorrect ThinLTO backend index-based
optimizations.

Reviewers: pcc

Subscribers: mehdi_amini, inglorion, eraman, cfe-commits

Differential Revision: https://reviews.llvm.org/D45217

Modified:
cfe/trunk/include/clang/Driver/Options.td
cfe/trunk/include/clang/Frontend/CodeGenOptions.h
cfe/trunk/lib/CodeGen/BackendUtil.cpp
cfe/trunk/lib/Driver/Driver.cpp
cfe/trunk/lib/Driver/ToolChains/Clang.cpp
cfe/trunk/lib/Frontend/CompilerInvocation.cpp
cfe/trunk/test/CodeGen/thinlto_backend.ll
cfe/trunk/test/Driver/thinlto_backend.c

Modified: cfe/trunk/include/clang/Driver/Options.td
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Options.td?rev=330194&r1=330193&r2=330194&view=diff
==
--- cfe/trunk/include/clang/Driver/Options.td (original)
+++ cfe/trunk/include/clang/Driver/Options.td Tue Apr 17 09:39:25 2018
@@ -2275,7 +2275,7 @@ def fno_rtlib_add_rpath: Flag<["-"], "fn
   HelpText<"Do not add -rpath with architecture-specific resource directory to 
the linker flags">;
 def r : Flag<["-"], "r">, Flags<[LinkerInput,NoArgumentUnused]>,
 Group;
-def save_temps_EQ : Joined<["-", "--"], "save-temps=">, Flags<[DriverOption]>,
+def save_temps_EQ : Joined<["-", "--"], "save-temps=">, Flags<[CC1Option, 
DriverOption]>,
   HelpText<"Save intermediate compilation results.">;
 def save_temps : Flag<["-", "--"], "save-temps">, Flags<[DriverOption]>,
   Alias, AliasArgs<["cwd"]>,

Modified: cfe/trunk/include/clang/Frontend/CodeGenOptions.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/CodeGenOptions.h?rev=330194&r1=330193&r2=330194&view=diff
==
--- cfe/trunk/include/clang/Frontend/CodeGenOptions.h (original)
+++ cfe/trunk/include/clang/Frontend/CodeGenOptions.h Tue Apr 17 09:39:25 2018
@@ -203,6 +203,9 @@ public:
   /// the summary and module symbol table (and not, e.g. any debug metadata).
   std::string ThinLinkBitcodeFile;
 
+  /// Prefix to use for -save-temps output.
+  std::string SaveTempsFilePrefix;
+
   /// Name of file passed with -fcuda-include-gpubinary option to forward to
   /// CUDA runtime back-end for incorporating them into host-side object file.
   std::string CudaGpuBinaryFileName;

Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=330194&r1=330193&r2=330194&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Tue Apr 17 09:39:25 2018
@@ -1117,6 +1117,15 @@ static void runThinLTOBackend(ModuleSumm
 return llvm::make_unique(std::move(OS));
   };
   lto::Config Conf;
+  if (CGOpts.SaveTempsFilePrefix != "") {
+if (Error E = Conf.addSaveTemps(CGOpts.SaveTempsFilePrefix + ".",
+/* UseInputModulePath */ false)) {
+  handleAllErrors(std::move(E), [&](ErrorInfoBase &EIB) {
+errs() << "Error setting up ThinLTO save-temps: " << EIB.message()
+   << '\n';
+  });
+}
+  }
   Conf.CPU = TOpts.CPU;
   Conf.CodeModel = getCodeModel(CGOpts);
   Conf.MAttrs = TOpts.Features;

Modified: cfe/trunk/lib/Driver/Driver.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Driver.cpp?rev=330194&r1=330193&r2=330194&view=diff
==
--- cfe/trunk/lib/Driver/Driver.cpp (original)
+++ cfe/trunk/lib/Driver/Driver.cpp Tue Apr 17 09:39:25 2018
@@ -3379,19 +3379,34 @@ class ToolSelector final {
   const Tool *combineBackendCompile(ArrayRef ActionInfo,
 const ActionList *&Inputs,
 ActionList &CollapsedOffloadAction) {
-if (ActionInfo.size() < 2 || !canCollapsePreprocessorAction())
+if (ActionInfo.size() < 2)
   return nullptr;
 auto *BJ = dyn_cast(ActionInfo[0].JA);
 auto *CJ = dyn_cast(ActionInfo[1].JA);
 if (!BJ || !CJ)
   return nullptr;
 

r330210 - Remove unused variable

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Tue Apr 17 13:21:53 2018
New Revision: 330210

URL: http://llvm.org/viewvc/llvm-project?rev=330210&view=rev
Log:
Remove unused variable

Fixes unused variable error introduced in r330194.

Modified:
cfe/trunk/lib/Driver/ToolChains/Clang.cpp

Modified: cfe/trunk/lib/Driver/ToolChains/Clang.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/ToolChains/Clang.cpp?rev=330210&r1=330209&r2=330210&view=diff
==
--- cfe/trunk/lib/Driver/ToolChains/Clang.cpp (original)
+++ cfe/trunk/lib/Driver/ToolChains/Clang.cpp Tue Apr 17 13:21:53 2018
@@ -3267,7 +3267,7 @@ void Clang::ConstructJob(Compilation &C,
 Args.AddLastArg(CmdArgs, options::OPT_fthinlto_index_EQ);
   }
 
-  if (const Arg *A = Args.getLastArg(options::OPT_save_temps_EQ))
+  if (Args.getLastArg(options::OPT_save_temps_EQ))
 Args.AddLastArg(CmdArgs, options::OPT_save_temps_EQ);
 
   // Embed-bitcode option.


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r330212 - Require shell for test

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Tue Apr 17 13:36:51 2018
New Revision: 330212

URL: http://llvm.org/viewvc/llvm-project?rev=330212&view=rev
Log:
Require shell for test

Attempt to fix windows bot which doesn't like the "(cd .." invocation
added in r330194:
http://lab.llvm.org:8011/builders/clang-with-thin-lto-windows/builds/8704/steps/stage%202%20check/logs/stdio

Modified:
cfe/trunk/test/CodeGen/thinlto_backend.ll

Modified: cfe/trunk/test/CodeGen/thinlto_backend.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto_backend.ll?rev=330212&r1=330211&r2=330212&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto_backend.ll (original)
+++ cfe/trunk/test/CodeGen/thinlto_backend.ll Tue Apr 17 13:36:51 2018
@@ -1,4 +1,5 @@
-; REQUIRES: x86-registered-target
+; shell required since the windows bot does not like the "(cd ..."
+; REQUIRES: x86-registered-target, shell
 
 ; RUN: opt -module-summary -o %t1.o %s
 ; RUN: opt -module-summary -o %t2.o %S/Inputs/thinlto_backend.ll


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r331592 - [ThinLTO] Support opt remarks options with distributed ThinLTO backends

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Sat May  5 07:37:29 2018
New Revision: 331592

URL: http://llvm.org/viewvc/llvm-project?rev=331592&view=rev
Log:
[ThinLTO] Support opt remarks options with distributed ThinLTO backends

Summary:
Passes down the necessary code ge options to the LTO Config to enable
-fdiagnostics-show-hotness and -fsave-optimization-record in the ThinLTO
backend for a distributed build.

Also, remove warning about not having PGO when the input is IR.

Reviewers: pcc

Subscribers: mehdi_amini, inglorion, eraman, cfe-commits

Differential Revision: https://reviews.llvm.org/D46464

Added:
cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll
Modified:
cfe/trunk/lib/CodeGen/BackendUtil.cpp
cfe/trunk/lib/Frontend/CompilerInvocation.cpp
cfe/trunk/test/CodeGen/thinlto_backend.ll

Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=331592&r1=331591&r2=331592&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Sat May  5 07:37:29 2018
@@ -1136,6 +1136,8 @@ static void runThinLTOBackend(ModuleSumm
   Conf.SampleProfile = std::move(SampleProfile);
   Conf.UseNewPM = CGOpts.ExperimentalNewPassManager;
   Conf.DebugPassManager = CGOpts.DebugPassManager;
+  Conf.RemarksWithHotness = CGOpts.DiagnosticsWithHotness;
+  Conf.RemarksFilename = CGOpts.OptRecordFile;
   switch (Action) {
   case Backend_EmitNothing:
 Conf.PreCodeGenModuleHook = [](size_t Task, const Module &Mod) {
@@ -1159,7 +1161,7 @@ static void runThinLTOBackend(ModuleSumm
 break;
   }
   if (Error E = thinBackend(
-  Conf, 0, AddStream, *M, *CombinedIndex, ImportList,
+  Conf, -1, AddStream, *M, *CombinedIndex, ImportList,
   ModuleToDefinedGVSummaries[M->getModuleIdentifier()], ModuleMap)) {
 handleAllErrors(std::move(E), [&](ErrorInfoBase &EIB) {
   errs() << "Error running ThinLTO backend: " << EIB.message() << '\n';

Modified: cfe/trunk/lib/Frontend/CompilerInvocation.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CompilerInvocation.cpp?rev=331592&r1=331591&r2=331592&view=diff
==
--- cfe/trunk/lib/Frontend/CompilerInvocation.cpp (original)
+++ cfe/trunk/lib/Frontend/CompilerInvocation.cpp Sat May  5 07:37:29 2018
@@ -1075,7 +1075,9 @@ static bool ParseCodeGenArgs(CodeGenOpti
   bool UsingProfile = UsingSampleProfile ||
   (Opts.getProfileUse() != CodeGenOptions::ProfileNone);
 
-  if (Opts.DiagnosticsWithHotness && !UsingProfile)
+  if (Opts.DiagnosticsWithHotness && !UsingProfile &&
+  // An IR file will contain PGO as metadata
+  IK.getLanguage() != InputKind::LLVM_IR)
 Diags.Report(diag::warn_drv_diagnostics_hotness_requires_pgo)
 << "-fdiagnostics-show-hotness";
 

Added: cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll?rev=331592&view=auto
==
--- cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll 
(added)
+++ cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll 
Sat May  5 07:37:29 2018
@@ -0,0 +1,47 @@
+; Test to ensure -fdiagnostics-show-hotness and -fsave-optimization-record
+; work when invoking the ThinLTO backend path.
+; RUN: opt -module-summary -o %t.o %s
+; RUN: llvm-lto -thinlto -o %t %t.o
+
+; First try with pass remarks to stderr
+; RUN: %clang -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -mllvm 
-pass-remarks=inline -fdiagnostics-show-hotness -o %t2.o -c 2>&1 | FileCheck %s
+
+; CHECK: tinkywinky inlined into main with cost=0 (threshold=337) (hotness: 
300)
+
+; Next try YAML pass remarks file
+; RUN: rm -f %t2.opt.yaml
+; RUN: %clang -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc 
-fsave-optimization-record -fdiagnostics-show-hotness -o %t2.o -c
+; RUN: cat %t2.opt.yaml | FileCheck %s -check-prefix=YAML
+
+; YAML: --- !Passed
+; YAML-NEXT: Pass:inline
+; YAML-NEXT: Name:Inlined
+; YAML-NEXT: Function:main
+; YAML-NEXT: Hotness: 300
+; YAML-NEXT: Args:
+; YAML-NEXT:   - Callee:  tinkywinky
+; YAML-NEXT:   - String:  ' inlined into '
+; YAML-NEXT:   - Caller:  main
+; YAML-NEXT:   - String:  ' with cost='
+; YAML-NEXT:   - Cost:'0'
+; YAML-NEXT:   - String:  ' (threshold='
+; YAML-NEXT:   - Threshold:   '337'
+; YAML-NEXT:   - String:  ')'
+; YAML-NEXT: ...
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-scei-ps4"
+
+declare i32 @patatino()
+
+define i32 @tinkywinky() {
+  %a = call i32 @patatino()
+  ret i32 %a
+}
+
+define i32 @mai

r331593 - Add required target to address bot failures from r331592

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Sat May  5 08:15:04 2018
New Revision: 331593

URL: http://llvm.org/viewvc/llvm-project?rev=331593&view=rev
Log:
Add required target to address bot failures from r331592

Failing on non-x86 bots, needs x86 target for code gen.

Modified:
cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll

Modified: 
cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll?rev=331593&r1=331592&r2=331593&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll 
(original)
+++ cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll 
Sat May  5 08:15:04 2018
@@ -1,5 +1,7 @@
 ; Test to ensure -fdiagnostics-show-hotness and -fsave-optimization-record
 ; work when invoking the ThinLTO backend path.
+; REQUIRES: x86-registered-target
+
 ; RUN: opt -module-summary -o %t.o %s
 ; RUN: llvm-lto -thinlto -o %t %t.o
 


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r331596 - Skip part of test added in r331592 to help debug bot failures

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Sat May  5 08:54:57 2018
New Revision: 331596

URL: http://llvm.org/viewvc/llvm-project?rev=331596&view=rev
Log:
Skip part of test added in r331592 to help debug bot failures

Trying to debug why/where a few bots getting exit code 256 e.g.
http://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-incremental/48471/testReport/Clang/CodeGen/thinlto_diagnostic_handler_remarks_with_hotness_ll/

and a few windows bots getting no output from that RUN line e.g.
http://lab.llvm.org:8011/builders/clang-x86-windows-msvc2015/builds/11865/steps/ninja%20check%201/logs/FAIL%3A%20Clang%3A%3Athinlto-diagnostic-handler-remarks-with-hotness.ll

Modified:
cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll

Modified: 
cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll?rev=331596&r1=331595&r2=331596&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll 
(original)
+++ cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll 
Sat May  5 08:54:57 2018
@@ -6,9 +6,10 @@
 ; RUN: llvm-lto -thinlto -o %t %t.o
 
 ; First try with pass remarks to stderr
-; RUN: %clang -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -mllvm 
-pass-remarks=inline -fdiagnostics-show-hotness -o %t2.o -c 2>&1 | FileCheck %s
+; Temporarily skip the next RUN/CHECK to debug bot failures
+; RUN %clang -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -mllvm 
-pass-remarks=inline -fdiagnostics-show-hotness -o %t2.o -c 2>&1 | FileCheck %s
 
-; CHECK: tinkywinky inlined into main with cost=0 (threshold=337) (hotness: 
300)
+; CHECK tinkywinky inlined into main with cost=0 (threshold=337) (hotness: 300)
 
 ; Next try YAML pass remarks file
 ; RUN: rm -f %t2.opt.yaml


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r331597 - Add -target to address errors in test from r331592

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Sat May  5 09:37:31 2018
New Revision: 331597

URL: http://llvm.org/viewvc/llvm-project?rev=331597&view=rev
Log:
Add -target to address errors in test from r331592

The error turns out to be:
Assertion failed: (Target.isCompatibleDataLayout(getDataLayout()) && "Can't 
create a MachineFunction using a Module with a " "Target-incompatible 
DataLayout attached\n"), function init, file 
/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-incremental/llvm/lib/CodeGen/MachineFunction.cpp,
 line 180.

Add -target to address this. Also re-enable the test I had temporarily
commented, and move it further down in case there is still a failure
(since it pipes stderr to FileCheck).

Modified:
cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll

Modified: 
cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll?rev=331597&r1=331596&r2=331597&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll 
(original)
+++ cfe/trunk/test/CodeGen/thinlto-diagnostic-handler-remarks-with-hotness.ll 
Sat May  5 09:37:31 2018
@@ -5,15 +5,9 @@
 ; RUN: opt -module-summary -o %t.o %s
 ; RUN: llvm-lto -thinlto -o %t %t.o
 
-; First try with pass remarks to stderr
-; Temporarily skip the next RUN/CHECK to debug bot failures
-; RUN %clang -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -mllvm 
-pass-remarks=inline -fdiagnostics-show-hotness -o %t2.o -c 2>&1 | FileCheck %s
-
-; CHECK tinkywinky inlined into main with cost=0 (threshold=337) (hotness: 300)
-
-; Next try YAML pass remarks file
+; First try YAML pass remarks file
 ; RUN: rm -f %t2.opt.yaml
-; RUN: %clang -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc 
-fsave-optimization-record -fdiagnostics-show-hotness -o %t2.o -c
+; RUN: %clang -target x86_64-scei-ps4 -O2 -x ir %t.o 
-fthinlto-index=%t.thinlto.bc -fsave-optimization-record 
-fdiagnostics-show-hotness -o %t2.o -c
 ; RUN: cat %t2.opt.yaml | FileCheck %s -check-prefix=YAML
 
 ; YAML: --- !Passed
@@ -32,6 +26,11 @@
 ; YAML-NEXT:   - String:  ')'
 ; YAML-NEXT: ...
 
+; Next try with pass remarks to stderr
+; RUN: %clang -target x86_64-scei-ps4 -O2 -x ir %t.o 
-fthinlto-index=%t.thinlto.bc -mllvm -pass-remarks=inline 
-fdiagnostics-show-hotness -o %t2.o -c 2>&1 | FileCheck %s
+
+; CHECK: tinkywinky inlined into main with cost=0 (threshold=337) (hotness: 
300)
+
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-scei-ps4"
 


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r335618 - [ThinLTO] Add testing of summary index parsing to a couple CFI tests

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Tue Jun 26 08:50:34 2018
New Revision: 335618

URL: http://llvm.org/viewvc/llvm-project?rev=335618&view=rev
Log:
[ThinLTO] Add testing of summary index parsing to a couple CFI tests

Summary:
Changes to some clang side tests to go with the summary parsing patch.

Depends on D47905.

Reviewers: pcc, dexonsmith, mehdi_amini

Subscribers: inglorion, eraman, cfe-commits, steven_wu

Differential Revision: https://reviews.llvm.org/D47906

Modified:
cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll

Modified: cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll?rev=335618&r1=335617&r2=335618&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll (original)
+++ cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll Tue Jun 26 
08:50:34 2018
@@ -30,6 +30,8 @@
 ; CHECK-LABEL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll?rev=335618&r1=335617&r2=335618&view=diff
==
--- cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll (original)
+++ cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll Tue Jun 26 08:50:34 2018
@@ -21,6 +21,8 @@
 ; CHECK-LABEL: http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: r335618 - [ThinLTO] Add testing of summary index parsing to a couple CFI tests

[Teresa Johnson via cfe-commits]

I haven't seen any bot failures for the change. The failing check was there
before, so presumably llvm-dis on its own is producing the expected lines,
just not when it goes through llvm-as. Can you let me know whether llvm-as
has been updated and whether it gives an error. Note that there were a
number of other similar test changes committed in the main llvm-side
patch D47905 shortly before this one. Are any of those failing for you?
Teresa

On Tue, Jun 26, 2018, 3:47 PM Shoaib Meenai  wrote:

> The check lines added here are failing for me on CentOS 7.
> https://reviews.llvm.org/P8091 has the lit output and the actual output
> being produced by one of the added RUN lines. My llvm and clang
> repositories are in sync. Any idea what might be going on here?
>
>
>
> *From: *cfe-commits  on behalf of
> Teresa Johnson via cfe-commits 
> *Reply-To: *Teresa Johnson 
> *Date: *Tuesday, June 26, 2018 at 8:55 AM
> *To: *"cfe-commits@lists.llvm.org" 
> *Subject: *r335618 - [ThinLTO] Add testing of summary index parsing to a
> couple CFI tests
>
>
>
> Author: tejohnson
>
> Date: Tue Jun 26 08:50:34 2018
>
> New Revision: 335618
>
>
>
> URL:
> https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_viewvc_llvm-2Dproject-3Frev-3D335618-26view-3Drev&d=DwIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=t3QlYWXjMDXjKJ9zRV9gsXH-VakqyyLmPocT9T_axY8&s=sFcyz-6EIWKbnsSlIcghgfqpuW73fEF48XuoUDzbohk&e=
>
> Log:
>
> [ThinLTO] Add testing of summary index parsing to a couple CFI tests
>
>
>
> Summary:
>
> Changes to some clang side tests to go with the summary parsing patch.
>
>
>
> Depends on D47905.
>
>
>
> Reviewers: pcc, dexonsmith, mehdi_amini
>
>
>
> Subscribers: inglorion, eraman, cfe-commits, steven_wu
>
>
>
> Differential Revision:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.llvm.org_D47906&d=DwIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=t3QlYWXjMDXjKJ9zRV9gsXH-VakqyyLmPocT9T_axY8&s=jQWcXM3uTGYSu5wgYQ_0NcHaJj9V9DOBoJcL8bxbjc4&e=
>
>
>
> Modified:
>
> cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
>
> cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll
>
>
>
> Modified: cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
>
> URL:
> https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_viewvc_llvm-2Dproject_cfe_trunk_test_CodeGen_thinlto-2Ddistributed-2Dcfi-2Ddevirt.ll-3Frev-3D335618-26r1-3D335617-26r2-3D335618-26view-3Ddiff&d=DwIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=t3QlYWXjMDXjKJ9zRV9gsXH-VakqyyLmPocT9T_axY8&s=DQBuDzMNex1JeoqLkc4S9ZTXRVWN2wCcbmk3U7hzKDQ&e=
>
>
> ==
>
> --- cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll (original)
>
> +++ cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll Tue Jun 26
> 08:50:34 2018
>
> @@ -30,6 +30,8 @@
>
> ; CHECK-LABEL: 
> ; RUN: llvm-dis %t.o.thinlto.bc -o - | FileCheck %s
> --check-prefix=CHECK-DIS
>
> +; Round trip it through llvm-as
>
> +; RUN: llvm-dis %t.o.thinlto.bc -o - | llvm-as -o - | llvm-dis -o - |
> FileCheck %s --check-prefix=CHECK-DIS
>
> ; CHECK-DIS: ^0 = module: (path:
> "{{.*}}thinlto-distributed-cfi-devirt.ll.tmp.o", hash: ({{.*}}, {{.*}},
> {{.*}}, {{.*}}, {{.*}}))
>
> ; CHECK-DIS: ^1 = gv: (guid: 8346051122425466633, summaries: (function:
> (module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 1,
> dsoLocal: 0), insts: 18, typeIdInfo: (typeTests: (^2),
> typeCheckedLoadVCalls: (vFuncId: (^2, offset: 8), vFuncId: (^2, offset:
> 0))
>
> ; CHECK-DIS: ^2 = typeid: (name: "_ZTS1A", summary: (typeTestRes: (kind:
> allOnes, sizeM1BitWidth: 7), wpdResolutions: ((offset: 0, wpdRes: (kind:
> branchFunnel)), (offset: 8, wpdRes: (kind: singleImpl, singleImplName:
> "_ZN1A1nEi") ; guid = 7004155349499253778
>
>
>
> Modified: cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll
>
> URL:
> https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_viewvc_llvm-2Dproject_cfe_trunk_test_CodeGen_thinlto-2Ddistributed-2Dcfi.ll-3Frev-3D335618-26r1-3D335617-26r2-3D335618-26view-3Ddiff&d=DwIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=t3QlYWXjMDXjKJ9zRV9gsXH-VakqyyLmPocT9T_axY8&s=PAH2s9gPirL0F8r0JQYFEoT0zKAnd6n9XnhXUvluugU&e=
>
>
> ==
>
> --- cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll (original)
>
> +++ cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll Tue Jun 26 08:50:34
> 2018
>
> @@ -21,6 +21,8 

Re: r335618 - [ThinLTO] Add testing of summary index parsing to a couple CFI tests

[Teresa Johnson via cfe-commits]

Ah - thanks for fixing that. Surprised none of the bots hit it.
Teresa

On Tue, Jun 26, 2018 at 4:24 PM Shoaib Meenai  wrote:

> Ah, my llvm-as was indeed out of date. I made check-clang depend on
> llvm-as in r335686 to avoid this issue in the future.
>
>
>
> *From: *Teresa Johnson 
> *Date: *Tuesday, June 26, 2018 at 4:11 PM
> *To: *Shoaib Meenai 
> *Cc: *"cfe-commits@lists.llvm.org" 
> *Subject: *Re: r335618 - [ThinLTO] Add testing of summary index parsing
> to a couple CFI tests
>
>
>
> I haven't seen any bot failures for the change. The failing check was
> there before, so presumably llvm-dis on its own is producing the expected
> lines, just not when it goes through llvm-as. Can you let me know whether
> llvm-as has been updated and whether it gives an error. Note that there
> were a number of other similar test changes committed in the main llvm-side
> patch D47905 shortly before this one. Are any of those failing for you?
>
> Teresa
>
> On Tue, Jun 26, 2018, 3:47 PM Shoaib Meenai  wrote:
>
> The check lines added here are failing for me on CentOS 7.
> https://reviews.llvm.org/P8091
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.llvm.org_P8091&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=Jy48KWhDHTfiRkUh6D6LWptKpx92vz5VpkWaHVCjHCE&s=rLSAJBQEXnuVpqtESj1BaYt_bapa7spVBuaonbJHWZ8&e=>
> has the lit output and the actual output being produced by one of the added
> RUN lines. My llvm and clang repositories are in sync. Any idea what might
> be going on here?
>
>
>
> *From: *cfe-commits  on behalf of
> Teresa Johnson via cfe-commits 
> *Reply-To: *Teresa Johnson 
> *Date: *Tuesday, June 26, 2018 at 8:55 AM
> *To: *"cfe-commits@lists.llvm.org" 
> *Subject: *r335618 - [ThinLTO] Add testing of summary index parsing to a
> couple CFI tests
>
>
>
> Author: tejohnson
>
> Date: Tue Jun 26 08:50:34 2018
>
> New Revision: 335618
>
>
>
> URL:
> https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_viewvc_llvm-2Dproject-3Frev-3D335618-26view-3Drev&d=DwIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=t3QlYWXjMDXjKJ9zRV9gsXH-VakqyyLmPocT9T_axY8&s=sFcyz-6EIWKbnsSlIcghgfqpuW73fEF48XuoUDzbohk&e=
>
> Log:
>
> [ThinLTO] Add testing of summary index parsing to a couple CFI tests
>
>
>
> Summary:
>
> Changes to some clang side tests to go with the summary parsing patch.
>
>
>
> Depends on D47905.
>
>
>
> Reviewers: pcc, dexonsmith, mehdi_amini
>
>
>
> Subscribers: inglorion, eraman, cfe-commits, steven_wu
>
>
>
> Differential Revision:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.llvm.org_D47906&d=DwIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=t3QlYWXjMDXjKJ9zRV9gsXH-VakqyyLmPocT9T_axY8&s=jQWcXM3uTGYSu5wgYQ_0NcHaJj9V9DOBoJcL8bxbjc4&e=
>
>
>
> Modified:
>
> cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
>
> cfe/trunk/test/CodeGen/thinlto-distributed-cfi.ll
>
>
>
> Modified: cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll
>
> URL:
> https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_viewvc_llvm-2Dproject_cfe_trunk_test_CodeGen_thinlto-2Ddistributed-2Dcfi-2Ddevirt.ll-3Frev-3D335618-26r1-3D335617-26r2-3D335618-26view-3Ddiff&d=DwIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=t3QlYWXjMDXjKJ9zRV9gsXH-VakqyyLmPocT9T_axY8&s=DQBuDzMNex1JeoqLkc4S9ZTXRVWN2wCcbmk3U7hzKDQ&e=
>
>
> ==
>
> --- cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll (original)
>
> +++ cfe/trunk/test/CodeGen/thinlto-distributed-cfi-devirt.ll Tue Jun 26
> 08:50:34 2018
>
> @@ -30,6 +30,8 @@
>
> ; CHECK-LABEL: 
> ; RUN: llvm-dis %t.o.thinlto.bc -o - | FileCheck %s
> --check-prefix=CHECK-DIS
>
> +; Round trip it through llvm-as
>
> +; RUN: llvm-dis %t.o.thinlto.bc -o - | llvm-as -o - | llvm-dis -o - |
> FileCheck %s --check-prefix=CHECK-DIS
>
> ; CHECK-DIS: ^0 = module: (path:
> "{{.*}}thinlto-distributed-cfi-devirt.ll.tmp.o", hash: ({{.*}}, {{.*}},
> {{.*}}, {{.*}}, {{.*}}))
>
> ; CHECK-DIS: ^1 = gv: (guid: 8346051122425466633, summaries: (function:
> (module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 1,
> dsoLocal: 0), insts: 18, typeIdInfo: (typeTests: (^2),
> typeCheckedLoadVCalls: (vFuncId: (^2, offset: 8), vFuncId: (^2, offset:
> 0))
>
> ; CHECK-DIS: ^2 = typeid: (name: "_ZTS1A", summary: (typeTestRes: (kind:
> allOnes, sizeM1BitWidth: 7), wpdResolutions: ((offset: 0, wpdRes: (kind:
> branchFunn

r337051 - [ThinLTO] Ensure we always select the same function copy to import

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Fri Jul 13 14:35:58 2018
New Revision: 337051

URL: http://llvm.org/viewvc/llvm-project?rev=337051&view=rev
Log:
[ThinLTO] Ensure we always select the same function copy to import

Clang change to reflect the FunctionsToImportTy type change
in the llvm changes for D48670.

Modified:
cfe/trunk/lib/CodeGen/BackendUtil.cpp

Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=337051&r1=337050&r2=337051&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Fri Jul 13 14:35:58 2018
@@ -1127,9 +1127,8 @@ static void runThinLTOBackend(ModuleSumm
 // e.g. record required linkage changes.
 if (Summary->modulePath() == M->getModuleIdentifier())
   continue;
-// Doesn't matter what value we plug in to the map, just needs an entry
-// to provoke importing by thinBackend.
-ImportList[Summary->modulePath()][GUID] = 1;
+// Add an entry to provoke importing by thinBackend.
+ImportList[Summary->modulePath()].insert(GUID);
   }
 
   std::vector> OwnedImports;


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r337082 - Revert "[ThinLTO] Ensure we always select the same function copy to import"

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Fri Jul 13 18:50:14 2018
New Revision: 337082

URL: http://llvm.org/viewvc/llvm-project?rev=337082&view=rev
Log:
Revert "[ThinLTO] Ensure we always select the same function copy to import"

This reverts commit r337051.

Modified:
cfe/trunk/lib/CodeGen/BackendUtil.cpp

Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=337082&r1=337081&r2=337082&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Fri Jul 13 18:50:14 2018
@@ -1127,8 +1127,9 @@ static void runThinLTOBackend(ModuleSumm
 // e.g. record required linkage changes.
 if (Summary->modulePath() == M->getModuleIdentifier())
   continue;
-// Add an entry to provoke importing by thinBackend.
-ImportList[Summary->modulePath()].insert(GUID);
+// Doesn't matter what value we plug in to the map, just needs an entry
+// to provoke importing by thinBackend.
+ImportList[Summary->modulePath()][GUID] = 1;
   }
 
   std::vector> OwnedImports;


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r337185 - Restore "[ThinLTO] Ensure we always select the same function copy to import"

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Mon Jul 16 08:30:36 2018
New Revision: 337185

URL: http://llvm.org/viewvc/llvm-project?rev=337185&view=rev
Log:
Restore "[ThinLTO] Ensure we always select the same function copy to import"

This reverts commit r337082, restoring r337051, since the LLVM side
patch has been restored.

Modified:
cfe/trunk/lib/CodeGen/BackendUtil.cpp

Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=337185&r1=337184&r2=337185&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Mon Jul 16 08:30:36 2018
@@ -1127,9 +1127,8 @@ static void runThinLTOBackend(ModuleSumm
 // e.g. record required linkage changes.
 if (Summary->modulePath() == M->getModuleIdentifier())
   continue;
-// Doesn't matter what value we plug in to the map, just needs an entry
-// to provoke importing by thinBackend.
-ImportList[Summary->modulePath()][GUID] = 1;
+// Add an entry to provoke importing by thinBackend.
+ImportList[Summary->modulePath()].insert(GUID);
   }
 
   std::vector> OwnedImports;


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 33ffb62 - Allow disabling of vectorization using internal options

[Teresa Johnson via cfe-commits]

Author: Teresa Johnson
Date: 2020-04-14T18:09:10-07:00
New Revision: 33ffb62e23e7a7bece5618d5a7b54bdb401d0bcf

URL: 
https://github.com/llvm/llvm-project/commit/33ffb62e23e7a7bece5618d5a7b54bdb401d0bcf
DIFF: 
https://github.com/llvm/llvm-project/commit/33ffb62e23e7a7bece5618d5a7b54bdb401d0bcf.diff

LOG: Allow disabling of vectorization using internal options

Summary:
Currently, the internal options -vectorize-loops, -vectorize-slp, and
-interleave-loops do not have much practical effect. This is because
they are used to initialize the corresponding flags in the pass
managers, and those flags are then unconditionally overwritten when
compiling via clang or via LTO from the linkers. The only exception was
-vectorize-loops via opt because of some special hackery there.

While vectorization could still be disabled when compiling via clang,
using -fno-[slp-]vectorize, this meant that there was no way to disable
it when compiling in LTO mode via the linkers. This only affected
ThinLTO, since for regular LTO vectorization is done during the compile
step for scalability reasons. For ThinLTO it is invoked in the LTO
backends. See also the discussion on PR45434.

This patch makes it so the internal options can actually be used to
disable these optimizations. Ultimately, the best long term solution is
to mark the loops with metadata (similar to the approach used to fix
-fno-unroll-loops in D77058), but this enables a shorter term
workaround, and actually makes these internal options useful.

I constant propagated the initial values of these internal flags into
the pass manager flags (for some reasons vectorize-loops and
interleave-loops were initialized to true, while vectorize-slp was
initialized to false). As mentioned above, they are overwritten
unconditionally so this doesn't have any real impact, and these initial
values aren't particularly meaningful.

I then changed the passes to check the internl values and return without
performing the associated optimization when false (I changed the default
of -vectorize-slp to true so the options behave similarly). I was able
to remove the hackery in opt used to get -vectorize-loops=false to work,
as well as a special option there used to disable SLP vectorization.

Finally, I changed thinlto-slp-vectorize-pm.c to:
a) Only test SLP (moved the loop vectorization checking to a new test).
b) Use code that is slp vectorized when it is enabled, and check that
instead of whether the pass is enabled.
c) Test the new behavior of -vectorize-slp.
d) Test both pass managers.

The loop vectorization (and associated interleaving) testing I moved to
a new thinlto-loop-vectorize-pm.c test, with several changes:
a) Changed the flags on the interleaving testing so that it will
actually interleave, and check that.
b) Test the new behavior of -vectorize-loops and -interleave-loops.
c) Test both pass managers.

Reviewers: fhahn, wmi

Subscribers: hiraditya, steven_wu, dexonsmith, cfe-commits, davezarzycki, 
llvm-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77989

Added: 
clang/test/CodeGen/thinlto-loop-vectorize-pm.c

Modified: 
clang/test/CodeGen/thinlto-slp-vectorize-pm.c
llvm/include/llvm/Passes/PassBuilder.h
llvm/include/llvm/Transforms/Vectorize/LoopVectorize.h
llvm/include/llvm/Transforms/Vectorize/SLPVectorizer.h
llvm/lib/Passes/PassBuilder.cpp
llvm/lib/Transforms/IPO/PassManagerBuilder.cpp
llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
llvm/test/Transforms/SLPVectorizer/X86/opt.ll
llvm/tools/opt/opt.cpp

Removed: 




diff  --git a/clang/test/CodeGen/thinlto-loop-vectorize-pm.c 
b/clang/test/CodeGen/thinlto-loop-vectorize-pm.c
new file mode 100644
index ..7933b98c246d
--- /dev/null
+++ b/clang/test/CodeGen/thinlto-loop-vectorize-pm.c
@@ -0,0 +1,44 @@
+// REQUIRES: x86-registered-target
+// RUN: %clang_cc1 -o %t.o -O2 -flto=thin -fexperimental-new-pass-manager 
-triple x86_64-unknown-linux-gnu -emit-llvm-bc %s
+// RUN: llvm-lto -thinlto -o %t %t.o
+
+// Test to ensure the loop vectorize codegen option is passed down to the
+// ThinLTO backend. -vectorize-loops is a cc1 option and will be added
+// automatically when O2/O3/Os is available for clang. Also check that
+// "-mllvm -vectorize-loops=false" will disable loop vectorization, overriding
+// the cc1 option.
+//
+// Check both the new and old PMs.
+//
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-obj -O2 
-vectorize-loops -mllvm -force-vector-width=2 -mllvm -force-vector-interleave=1 
-emit-llvm -o - -x ir %t.o -fthinlto-index=%t.thinlto.bc 
-fexperimental-new-pass-manager 2>&1 | FileCheck %s --check-prefix=O2-LPV
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-obj -O2 
-vectorize-loops -mllvm -force-vector-width=2 -mllvm -force-vector-interleave=1 
-emit-llvm -o - -x ir %t.o -fthinlto-index=%

[PATCH] D24806: [docs] Add ThinLTO user documentation

[Teresa Johnson via cfe-commits]

tejohnson created this revision.
tejohnson added a reviewer: mehdi_amini.
tejohnson added a subscriber: cfe-commits.
Herald added a subscriber: mehdi_amini.

Add some user facing documentation on ThinLTO and how to use it.

https://reviews.llvm.org/D24806

Files:
  docs/CommandGuide/clang.rst
  docs/ThinLTO.rst
  docs/index.rst

Index: docs/index.rst
===
--- docs/index.rst
+++ docs/index.rst
@@ -37,6 +37,7 @@
SourceBasedCodeCoverage
Modules
MSVCCompatibility
+   ThinLTO
CommandGuide/index
FAQ
 
Index: docs/ThinLTO.rst
===
--- /dev/null
+++ docs/ThinLTO.rst
@@ -0,0 +1,146 @@
+===
+ThinLTO
+===
+
+.. contents::
+   :local:
+
+Introduction
+
+
+*ThinLTO* compilation is a new type of LTO that is both scalable and
+incremental. *LTO* (Link Time Optimization) achieves better
+runtime performance through whole-program analysis and cross-module
+optimization. However, regular LTO implements this by merging all
+input into a single monolithic module, which is not scalable
+in time or memory, and also prevents fast incremental compiles.
+
+In ThinLTO mode, as with regular LTO, clang emits LLVM bitcode after the
+compile phase. The ThinLTO bitcode is augmented with a compact summary
+of the module. During the link step, only the summaries are read and
+merged into a combined summary index, which includes an index of function
+locations for later cross-module function importing. Fast and efficient
+whole-program analysis is then performed on the combined summary index.
+
+However, all transformations, including function importing, occur
+later when the modules are optimized in fully parallel backends.
+By default, linkers_ that support ThinLTO are set up to launch
+the ThinLTO backends in threads. So the usage model is not affected
+as the distinction between the fast serial thin link step and the backends
+is transparent to the user.
+
+For more information on the ThinLTO design and current performance,
+see the LLVM blog post `ThinLTO: Scalable and Incremental LTO
+`_.
+While tuning is still in progress, results in the blog post show that
+ThinLTO already performs well compared to LTO, in many cases matching
+the performance improvement.
+
+Current Status
+==
+
+Clang/LLVM
+--
+.. _compiler:
+
+The 3.9 release of clang includes ThinLTO support. However, ThinLTO
+is under active development, and new features, improvements and bugfixes
+are being added for the next release. For the latest ThinLTO support,
+`build a recent version of clang and LLVM
+`_.
+
+Linkers
+---
+.. _linkers:
+.. _linker:
+
+ThinLTO is currently supported for the following linkers:
+
+- **gold (via the gold-plugin)**:
+  Similar to regular LTO, this requires using
+  a `gold linker configured with plugins enabled
+  `_.
+- **ld64**:
+  Starting with `Xcode 8 `_.
+
+Additionally, support is being added to the *lld* linker.
+
+Usage
+=
+
+Basic
+-
+
+To utilize ThinLTO, simply add the -flto=thin option to compile and link. E.g.
+
+.. code-block:: console
+
+  % clang -flto=thin -O2 file1.c file2.c -c
+  % clang -flto=thin -O2 file1.o file2.o -o a.out
+
+As mentioned earlier, by default the linkers will launch the ThinLTO backend
+threads in parallel, passing the resulting native object files back to the
+linker for the final native link.  As such, the usage model the same as
+non-LTO.
+
+Controlling Backend Parallelism
+---
+.. _parallelism:
+
+By default, the ThinLTO link step will launch up to
+``std::thread::hardware_concurrency`` number of threads in parallel.
+For machines with hyper-threading, this is the total number of
+virtual cores. For some applications and machine configurations this
+may be too aggressive, in which case the amount of parallelism can
+be reduced to ``N`` via:
+
+- gold:
+  ``-Wl,-plugin-opt,jobs=N``
+- ld64:
+  ``-Wl,-mllvm,-threads=N``
+
+Incremental
+---
+.. _incremental:
+
+ThinLTO supports fast incremental builds through the use of a cache,
+which currently must be enabled through a linker option.
+
+- gold (as of LLVM r279883):
+  ``-Wl,-plugin-opt,cache-dir=/path/to/cache``
+- ld64 (support in clang 3.9 and Xcode 8):
+  ``-Wl,-cache_path_lto,/path/to/cache``
+
+Clang Bootstrap
+---
+
+To bootstrap clang/LLVM with ThinLTO, follow these steps:
+
+1. The host compiler_ must be a version of clang that supports ThinLTO.
+#. The host linker_ must support ThinLTO (and in the case of gold, must be
+   `configured with plugins enabled `_.
+#. Use the following additional `CMake variables
+   `_
+   when configur

Re: [PATCH] D24806: [docs] Add ThinLTO user documentation

[Teresa Johnson via cfe-commits]

tejohnson updated this revision to Diff 72078.
tejohnson added a comment.

Add some troubleshooting info for gold plugin misconfigurations


https://reviews.llvm.org/D24806

Files:
  docs/CommandGuide/clang.rst
  docs/ThinLTO.rst
  docs/index.rst

Index: docs/index.rst
===
--- docs/index.rst
+++ docs/index.rst
@@ -37,6 +37,7 @@
SourceBasedCodeCoverage
Modules
MSVCCompatibility
+   ThinLTO
CommandGuide/index
FAQ
 
Index: docs/ThinLTO.rst
===
--- /dev/null
+++ docs/ThinLTO.rst
@@ -0,0 +1,157 @@
+===
+ThinLTO
+===
+
+.. contents::
+   :local:
+
+Introduction
+
+
+*ThinLTO* compilation is a new type of LTO that is both scalable and
+incremental. *LTO* (Link Time Optimization) achieves better
+runtime performance through whole-program analysis and cross-module
+optimization. However, regular LTO implements this by merging all
+input into a single monolithic module, which is not scalable
+in time or memory, and also prevents fast incremental compiles.
+
+In ThinLTO mode, as with regular LTO, clang emits LLVM bitcode after the
+compile phase. The ThinLTO bitcode is augmented with a compact summary
+of the module. During the link step, only the summaries are read and
+merged into a combined summary index, which includes an index of function
+locations for later cross-module function importing. Fast and efficient
+whole-program analysis is then performed on the combined summary index.
+
+However, all transformations, including function importing, occur
+later when the modules are optimized in fully parallel backends.
+By default, linkers_ that support ThinLTO are set up to launch
+the ThinLTO backends in threads. So the usage model is not affected
+as the distinction between the fast serial thin link step and the backends
+is transparent to the user.
+
+For more information on the ThinLTO design and current performance,
+see the LLVM blog post `ThinLTO: Scalable and Incremental LTO
+`_.
+While tuning is still in progress, results in the blog post show that
+ThinLTO already performs well compared to LTO, in many cases matching
+the performance improvement.
+
+Current Status
+==
+
+Clang/LLVM
+--
+.. _compiler:
+
+The 3.9 release of clang includes ThinLTO support. However, ThinLTO
+is under active development, and new features, improvements and bugfixes
+are being added for the next release. For the latest ThinLTO support,
+`build a recent version of clang and LLVM
+`_.
+
+Linkers
+---
+.. _linkers:
+.. _linker:
+
+ThinLTO is currently supported for the following linkers:
+
+- **gold (via the gold-plugin)**:
+  Similar to regular LTO, this requires using
+  a `gold linker configured with plugins enabled
+  `_.
+- **ld64**:
+  Starting with `Xcode 8 `_.
+
+Additionally, support is being added to the *lld* linker.
+
+Usage
+=
+
+Basic
+-
+
+To utilize ThinLTO, simply add the -flto=thin option to compile and link. E.g.
+
+.. code-block:: console
+
+  % clang -flto=thin -O2 file1.c file2.c -c
+  % clang -flto=thin -O2 file1.o file2.o -o a.out
+
+As mentioned earlier, by default the linkers will launch the ThinLTO backend
+threads in parallel, passing the resulting native object files back to the
+linker for the final native link.  As such, the usage model the same as
+non-LTO.
+
+With gold, if you see an error during the link of the form:
+
+.. code-block:: console
+
+  /usr/bin/ld: error: /path/to/clang/bin/../lib/LLVMgold.so: could not load plugin library: /path/to/clang/bin/../lib/LLVMgold.so: cannot open shared object file: No such file or directory
+
+Then either gold was not configured with plugins enabled, or clang
+was not built with ``-DLLVM_BINUTILS_INCDIR`` set properly. See
+the instructions for the
+`LLVM gold plugin `_.
+
+Controlling Backend Parallelism
+---
+.. _parallelism:
+
+By default, the ThinLTO link step will launch up to
+``std::thread::hardware_concurrency`` number of threads in parallel.
+For machines with hyper-threading, this is the total number of
+virtual cores. For some applications and machine configurations this
+may be too aggressive, in which case the amount of parallelism can
+be reduced to ``N`` via:
+
+- gold:
+  ``-Wl,-plugin-opt,jobs=N``
+- ld64:
+  ``-Wl,-mllvm,-threads=N``
+
+Incremental
+---
+.. _incremental:
+
+ThinLTO supports fast incremental builds through the use of a cache,
+which currently must be enabled through a linker option.
+
+- gold (as of LLVM r279883):
+  ``-Wl,-plugin-opt,cache-dir=/path/to/cache``
+- ld64 (support in clang 3.9 and Xcode 8):
+  ``-Wl,-cache_path_lto,/path/to/cache``
+
+Clang Bootstrap

Re: [PATCH] D24806: [docs] Add ThinLTO user documentation

[Teresa Johnson via cfe-commits]

tejohnson updated this revision to Diff 72081.
tejohnson added a comment.

Replace 'regular LTO' with 'monolithic LTO' to be more descriptive.


https://reviews.llvm.org/D24806

Files:
  docs/CommandGuide/clang.rst
  docs/ThinLTO.rst
  docs/index.rst

Index: docs/index.rst
===
--- docs/index.rst
+++ docs/index.rst
@@ -37,6 +37,7 @@
SourceBasedCodeCoverage
Modules
MSVCCompatibility
+   ThinLTO
CommandGuide/index
FAQ
 
Index: docs/ThinLTO.rst
===
--- /dev/null
+++ docs/ThinLTO.rst
@@ -0,0 +1,157 @@
+===
+ThinLTO
+===
+
+.. contents::
+   :local:
+
+Introduction
+
+
+*ThinLTO* compilation is a new type of LTO that is both scalable and
+incremental. *LTO* (Link Time Optimization) achieves better
+runtime performance through whole-program analysis and cross-module
+optimization. However, monolithic LTO implements this by merging all
+input into a single module, which is not scalable
+in time or memory, and also prevents fast incremental compiles.
+
+In ThinLTO mode, as with regular LTO, clang emits LLVM bitcode after the
+compile phase. The ThinLTO bitcode is augmented with a compact summary
+of the module. During the link step, only the summaries are read and
+merged into a combined summary index, which includes an index of function
+locations for later cross-module function importing. Fast and efficient
+whole-program analysis is then performed on the combined summary index.
+
+However, all transformations, including function importing, occur
+later when the modules are optimized in fully parallel backends.
+By default, linkers_ that support ThinLTO are set up to launch
+the ThinLTO backends in threads. So the usage model is not affected
+as the distinction between the fast serial thin link step and the backends
+is transparent to the user.
+
+For more information on the ThinLTO design and current performance,
+see the LLVM blog post `ThinLTO: Scalable and Incremental LTO
+`_.
+While tuning is still in progress, results in the blog post show that
+ThinLTO already performs well compared to LTO, in many cases matching
+the performance improvement.
+
+Current Status
+==
+
+Clang/LLVM
+--
+.. _compiler:
+
+The 3.9 release of clang includes ThinLTO support. However, ThinLTO
+is under active development, and new features, improvements and bugfixes
+are being added for the next release. For the latest ThinLTO support,
+`build a recent version of clang and LLVM
+`_.
+
+Linkers
+---
+.. _linkers:
+.. _linker:
+
+ThinLTO is currently supported for the following linkers:
+
+- **gold (via the gold-plugin)**:
+  Similar to monolithic LTO, this requires using
+  a `gold linker configured with plugins enabled
+  `_.
+- **ld64**:
+  Starting with `Xcode 8 `_.
+
+Additionally, support is being added to the *lld* linker.
+
+Usage
+=
+
+Basic
+-
+
+To utilize ThinLTO, simply add the -flto=thin option to compile and link. E.g.
+
+.. code-block:: console
+
+  % clang -flto=thin -O2 file1.c file2.c -c
+  % clang -flto=thin -O2 file1.o file2.o -o a.out
+
+As mentioned earlier, by default the linkers will launch the ThinLTO backend
+threads in parallel, passing the resulting native object files back to the
+linker for the final native link.  As such, the usage model the same as
+non-LTO.
+
+With gold, if you see an error during the link of the form:
+
+.. code-block:: console
+
+  /usr/bin/ld: error: /path/to/clang/bin/../lib/LLVMgold.so: could not load plugin library: /path/to/clang/bin/../lib/LLVMgold.so: cannot open shared object file: No such file or directory
+
+Then either gold was not configured with plugins enabled, or clang
+was not built with ``-DLLVM_BINUTILS_INCDIR`` set properly. See
+the instructions for the
+`LLVM gold plugin `_.
+
+Controlling Backend Parallelism
+---
+.. _parallelism:
+
+By default, the ThinLTO link step will launch up to
+``std::thread::hardware_concurrency`` number of threads in parallel.
+For machines with hyper-threading, this is the total number of
+virtual cores. For some applications and machine configurations this
+may be too aggressive, in which case the amount of parallelism can
+be reduced to ``N`` via:
+
+- gold:
+  ``-Wl,-plugin-opt,jobs=N``
+- ld64:
+  ``-Wl,-mllvm,-threads=N``
+
+Incremental
+---
+.. _incremental:
+
+ThinLTO supports fast incremental builds through the use of a cache,
+which currently must be enabled through a linker option.
+
+- gold (as of LLVM r279883):
+  ``-Wl,-plugin-opt,cache-dir=/path/to/cache``
+- ld64 (support in clang 3.9 and Xcode 8):
+  ``-Wl,-cache_path_lto,/path/to/cache``
+
+Clang Bootstrap
+

r282089 - [docs] Add ThinLTO user documentation

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Wed Sep 21 11:57:03 2016
New Revision: 282089

URL: http://llvm.org/viewvc/llvm-project?rev=282089&view=rev
Log:
[docs] Add ThinLTO user documentation

Summary: Add some user facing documentation on ThinLTO and how to use it.

Reviewers: mehdi_amini

Subscribers: mehdi_amini, cfe-commits

Differential Revision: https://reviews.llvm.org/D24806

Added:
cfe/trunk/docs/ThinLTO.rst
Modified:
cfe/trunk/docs/CommandGuide/clang.rst
cfe/trunk/docs/index.rst

Modified: cfe/trunk/docs/CommandGuide/clang.rst
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/CommandGuide/clang.rst?rev=282089&r1=282088&r2=282089&view=diff
==
--- cfe/trunk/docs/CommandGuide/clang.rst (original)
+++ cfe/trunk/docs/CommandGuide/clang.rst Wed Sep 21 11:57:03 2016
@@ -328,13 +328,19 @@ Code Generation Options
   model can be overridden with the tls_model attribute. The compiler will try
   to choose a more efficient model if possible.
 
-.. option:: -flto, -emit-llvm
+.. option:: -flto[=full,thin], -emit-llvm
 
   Generate output files in LLVM formats, suitable for link time optimization.
   When used with :option:`-S` this generates LLVM intermediate language
   assembly files, otherwise this generates LLVM bitcode format object files
   (which may be passed to the linker depending on the stage selection options).
 
+  The default for :option:`-flto` is :option:`-flto=full`, in which the
+  LLVM bitcode is suitable for monolithic Link Time Optimization (LTO), where
+  the linker merges all such modules into a single combined module for
+  optimization. With :option:`-flto=thin`, :doc:`ThinLTO <../ThinLTO>`
+  compilation is invoked instead.
+
 Driver Options
 ~~
 

Added: cfe/trunk/docs/ThinLTO.rst
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/ThinLTO.rst?rev=282089&view=auto
==
--- cfe/trunk/docs/ThinLTO.rst (added)
+++ cfe/trunk/docs/ThinLTO.rst Wed Sep 21 11:57:03 2016
@@ -0,0 +1,157 @@
+===
+ThinLTO
+===
+
+.. contents::
+   :local:
+
+Introduction
+
+
+*ThinLTO* compilation is a new type of LTO that is both scalable and
+incremental. *LTO* (Link Time Optimization) achieves better
+runtime performance through whole-program analysis and cross-module
+optimization. However, monolithic LTO implements this by merging all
+input into a single module, which is not scalable
+in time or memory, and also prevents fast incremental compiles.
+
+In ThinLTO mode, as with regular LTO, clang emits LLVM bitcode after the
+compile phase. The ThinLTO bitcode is augmented with a compact summary
+of the module. During the link step, only the summaries are read and
+merged into a combined summary index, which includes an index of function
+locations for later cross-module function importing. Fast and efficient
+whole-program analysis is then performed on the combined summary index.
+
+However, all transformations, including function importing, occur
+later when the modules are optimized in fully parallel backends.
+By default, linkers_ that support ThinLTO are set up to launch
+the ThinLTO backends in threads. So the usage model is not affected
+as the distinction between the fast serial thin link step and the backends
+is transparent to the user.
+
+For more information on the ThinLTO design and current performance,
+see the LLVM blog post `ThinLTO: Scalable and Incremental LTO
+`_.
+While tuning is still in progress, results in the blog post show that
+ThinLTO already performs well compared to LTO, in many cases matching
+the performance improvement.
+
+Current Status
+==
+
+Clang/LLVM
+--
+.. _compiler:
+
+The 3.9 release of clang includes ThinLTO support. However, ThinLTO
+is under active development, and new features, improvements and bugfixes
+are being added for the next release. For the latest ThinLTO support,
+`build a recent version of clang and LLVM
+`_.
+
+Linkers
+---
+.. _linkers:
+.. _linker:
+
+ThinLTO is currently supported for the following linkers:
+
+- **gold (via the gold-plugin)**:
+  Similar to monolithic LTO, this requires using
+  a `gold linker configured with plugins enabled
+  `_.
+- **ld64**:
+  Starting with `Xcode 8 `_.
+
+Additionally, support is being added to the *lld* linker.
+
+Usage
+=
+
+Basic
+-
+
+To utilize ThinLTO, simply add the -flto=thin option to compile and link. E.g.
+
+.. code-block:: console
+
+  % clang -flto=thin -O2 file1.c file2.c -c
+  % clang -flto=thin -O2 file1.o file2.o -o a.out
+
+As mentioned earlier, by default the linkers will launch the ThinLTO backend
+threads in parallel, passing the resulting native object files back to the
+linker for the final nativ

Re: [PATCH] D24806: [docs] Add ThinLTO user documentation

[Teresa Johnson via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rL282089: [docs] Add ThinLTO user documentation (authored by 
tejohnson).

Changed prior to commit:
  https://reviews.llvm.org/D24806?vs=72081&id=72082#toc

Repository:
  rL LLVM

https://reviews.llvm.org/D24806

Files:
  cfe/trunk/docs/CommandGuide/clang.rst
  cfe/trunk/docs/ThinLTO.rst
  cfe/trunk/docs/index.rst

Index: cfe/trunk/docs/ThinLTO.rst
===
--- cfe/trunk/docs/ThinLTO.rst
+++ cfe/trunk/docs/ThinLTO.rst
@@ -0,0 +1,157 @@
+===
+ThinLTO
+===
+
+.. contents::
+   :local:
+
+Introduction
+
+
+*ThinLTO* compilation is a new type of LTO that is both scalable and
+incremental. *LTO* (Link Time Optimization) achieves better
+runtime performance through whole-program analysis and cross-module
+optimization. However, monolithic LTO implements this by merging all
+input into a single module, which is not scalable
+in time or memory, and also prevents fast incremental compiles.
+
+In ThinLTO mode, as with regular LTO, clang emits LLVM bitcode after the
+compile phase. The ThinLTO bitcode is augmented with a compact summary
+of the module. During the link step, only the summaries are read and
+merged into a combined summary index, which includes an index of function
+locations for later cross-module function importing. Fast and efficient
+whole-program analysis is then performed on the combined summary index.
+
+However, all transformations, including function importing, occur
+later when the modules are optimized in fully parallel backends.
+By default, linkers_ that support ThinLTO are set up to launch
+the ThinLTO backends in threads. So the usage model is not affected
+as the distinction between the fast serial thin link step and the backends
+is transparent to the user.
+
+For more information on the ThinLTO design and current performance,
+see the LLVM blog post `ThinLTO: Scalable and Incremental LTO
+`_.
+While tuning is still in progress, results in the blog post show that
+ThinLTO already performs well compared to LTO, in many cases matching
+the performance improvement.
+
+Current Status
+==
+
+Clang/LLVM
+--
+.. _compiler:
+
+The 3.9 release of clang includes ThinLTO support. However, ThinLTO
+is under active development, and new features, improvements and bugfixes
+are being added for the next release. For the latest ThinLTO support,
+`build a recent version of clang and LLVM
+`_.
+
+Linkers
+---
+.. _linkers:
+.. _linker:
+
+ThinLTO is currently supported for the following linkers:
+
+- **gold (via the gold-plugin)**:
+  Similar to monolithic LTO, this requires using
+  a `gold linker configured with plugins enabled
+  `_.
+- **ld64**:
+  Starting with `Xcode 8 `_.
+
+Additionally, support is being added to the *lld* linker.
+
+Usage
+=
+
+Basic
+-
+
+To utilize ThinLTO, simply add the -flto=thin option to compile and link. E.g.
+
+.. code-block:: console
+
+  % clang -flto=thin -O2 file1.c file2.c -c
+  % clang -flto=thin -O2 file1.o file2.o -o a.out
+
+As mentioned earlier, by default the linkers will launch the ThinLTO backend
+threads in parallel, passing the resulting native object files back to the
+linker for the final native link.  As such, the usage model the same as
+non-LTO.
+
+With gold, if you see an error during the link of the form:
+
+.. code-block:: console
+
+  /usr/bin/ld: error: /path/to/clang/bin/../lib/LLVMgold.so: could not load plugin library: /path/to/clang/bin/../lib/LLVMgold.so: cannot open shared object file: No such file or directory
+
+Then either gold was not configured with plugins enabled, or clang
+was not built with ``-DLLVM_BINUTILS_INCDIR`` set properly. See
+the instructions for the
+`LLVM gold plugin `_.
+
+Controlling Backend Parallelism
+---
+.. _parallelism:
+
+By default, the ThinLTO link step will launch up to
+``std::thread::hardware_concurrency`` number of threads in parallel.
+For machines with hyper-threading, this is the total number of
+virtual cores. For some applications and machine configurations this
+may be too aggressive, in which case the amount of parallelism can
+be reduced to ``N`` via:
+
+- gold:
+  ``-Wl,-plugin-opt,jobs=N``
+- ld64:
+  ``-Wl,-mllvm,-threads=N``
+
+Incremental
+---
+.. _incremental:
+
+ThinLTO supports fast incremental builds through the use of a cache,
+which currently must be enabled through a linker option.
+
+- gold (as of LLVM r279883):
+  ``-Wl,-plugin-opt,cache-dir=/path/to/cache``
+- ld64 (support in clang 3.9 and Xcode 8):
+  ``-Wl,-cache_path_lto,/path/to/cache``
+
+Clang Bootstrap
+---
+
+To bootstrap clang/LLVM with ThinLTO, fol

Re: r282089 - [docs] Add ThinLTO user documentation

[Teresa Johnson via cfe-commits]

Anyone know how to get these changes on http://clang.llvm.org/docs/? Is
that site periodically refreshed or do I need to do something explicit?

Thanks,
Teresa

On Wed, Sep 21, 2016 at 9:57 AM, Teresa Johnson via cfe-commits <
cfe-commits@lists.llvm.org> wrote:

> Author: tejohnson
> Date: Wed Sep 21 11:57:03 2016
> New Revision: 282089
>
> URL: http://llvm.org/viewvc/llvm-project?rev=282089&view=rev
> Log:
> [docs] Add ThinLTO user documentation
>
> Summary: Add some user facing documentation on ThinLTO and how to use it.
>
> Reviewers: mehdi_amini
>
> Subscribers: mehdi_amini, cfe-commits
>
> Differential Revision: https://reviews.llvm.org/D24806
>
> Added:
> cfe/trunk/docs/ThinLTO.rst
> Modified:
> cfe/trunk/docs/CommandGuide/clang.rst
> cfe/trunk/docs/index.rst
>
> Modified: cfe/trunk/docs/CommandGuide/clang.rst
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/
> CommandGuide/clang.rst?rev=282089&r1=282088&r2=282089&view=diff
> 
> ==
> --- cfe/trunk/docs/CommandGuide/clang.rst (original)
> +++ cfe/trunk/docs/CommandGuide/clang.rst Wed Sep 21 11:57:03 2016
> @@ -328,13 +328,19 @@ Code Generation Options
>model can be overridden with the tls_model attribute. The compiler will
> try
>to choose a more efficient model if possible.
>
> -.. option:: -flto, -emit-llvm
> +.. option:: -flto[=full,thin], -emit-llvm
>
>Generate output files in LLVM formats, suitable for link time
> optimization.
>When used with :option:`-S` this generates LLVM intermediate language
>assembly files, otherwise this generates LLVM bitcode format object
> files
>(which may be passed to the linker depending on the stage selection
> options).
>
> +  The default for :option:`-flto` is :option:`-flto=full`, in which the
> +  LLVM bitcode is suitable for monolithic Link Time Optimization (LTO),
> where
> +  the linker merges all such modules into a single combined module for
> +  optimization. With :option:`-flto=thin`, :doc:`ThinLTO <../ThinLTO>`
> +  compilation is invoked instead.
> +
>  Driver Options
>  ~~
>
>
> Added: cfe/trunk/docs/ThinLTO.rst
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/
> ThinLTO.rst?rev=282089&view=auto
> 
> ==
> --- cfe/trunk/docs/ThinLTO.rst (added)
> +++ cfe/trunk/docs/ThinLTO.rst Wed Sep 21 11:57:03 2016
> @@ -0,0 +1,157 @@
> +===
> +ThinLTO
> +===
> +
> +.. contents::
> +   :local:
> +
> +Introduction
> +
> +
> +*ThinLTO* compilation is a new type of LTO that is both scalable and
> +incremental. *LTO* (Link Time Optimization) achieves better
> +runtime performance through whole-program analysis and cross-module
> +optimization. However, monolithic LTO implements this by merging all
> +input into a single module, which is not scalable
> +in time or memory, and also prevents fast incremental compiles.
> +
> +In ThinLTO mode, as with regular LTO, clang emits LLVM bitcode after the
> +compile phase. The ThinLTO bitcode is augmented with a compact summary
> +of the module. During the link step, only the summaries are read and
> +merged into a combined summary index, which includes an index of function
> +locations for later cross-module function importing. Fast and efficient
> +whole-program analysis is then performed on the combined summary index.
> +
> +However, all transformations, including function importing, occur
> +later when the modules are optimized in fully parallel backends.
> +By default, linkers_ that support ThinLTO are set up to launch
> +the ThinLTO backends in threads. So the usage model is not affected
> +as the distinction between the fast serial thin link step and the backends
> +is transparent to the user.
> +
> +For more information on the ThinLTO design and current performance,
> +see the LLVM blog post `ThinLTO: Scalable and Incremental LTO
> +<http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html
> >`_.
> +While tuning is still in progress, results in the blog post show that
> +ThinLTO already performs well compared to LTO, in many cases matching
> +the performance improvement.
> +
> +Current Status
> +==
> +
> +Clang/LLVM
> +--
> +.. _compiler:
> +
> +The 3.9 release of clang includes ThinLTO support. However, ThinLTO
> +is under active development, and new features, improvements and bugfixes
> +are being added for the next release. For the latest ThinLTO support,
> +`build a recent version of clang and LLVM
> +<http://llvm.o

Re: r282148 - Fixing sphinx build due to diagnostic:

[Teresa Johnson via cfe-commits]

Thank you!
Teresa

On Thu, Sep 22, 2016 at 5:15 AM, Aaron Ballman via cfe-commits <
cfe-commits@lists.llvm.org> wrote:

> Author: aaronballman
> Date: Thu Sep 22 07:15:18 2016
> New Revision: 282148
>
> URL: http://llvm.org/viewvc/llvm-project?rev=282148&view=rev
> Log:
> Fixing sphinx build due to diagnostic:
>
> /opt/llvm/build.attributes.src/tools/clang/docs/CommandGuide/clang.rst:338:
> WARNING: unknown option: -flto=full
>
> Modified:
> cfe/trunk/docs/CommandGuide/clang.rst
>
> Modified: cfe/trunk/docs/CommandGuide/clang.rst
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/
> CommandGuide/clang.rst?rev=282148&r1=282147&r2=282148&view=diff
> 
> ==
> --- cfe/trunk/docs/CommandGuide/clang.rst (original)
> +++ cfe/trunk/docs/CommandGuide/clang.rst Thu Sep 22 07:15:18 2016
> @@ -328,7 +328,7 @@ Code Generation Options
>model can be overridden with the tls_model attribute. The compiler will
> try
>to choose a more efficient model if possible.
>
> -.. option:: -flto[=full,thin], -emit-llvm
> +.. option:: -flto, -flto=full, -flto=thin, -emit-llvm
>
>Generate output files in LLVM formats, suitable for link time
> optimization.
>When used with :option:`-S` this generates LLVM intermediate language
>
>
> ___
> cfe-commits mailing list
> cfe-commits@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>



-- 
Teresa Johnson |  Software Engineer |  tejohn...@google.com |  408-460-2413
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: r282148 - Fixing sphinx build due to diagnostic:

[Teresa Johnson via cfe-commits]

Hi Aaron,

I just went back to the bot, which had first failed with a process error
when I made my commit which is why I didn't see this error. It looks like
it is still failing with the same error:

http://lab.llvm.org:8011/builders/clang-sphinx-docs/builds/16313/steps/docs-clang-html/logs/stdio
(that build has your fix).

Unfortunately, I can't reproduce this locally, which is why I didn't see it
in my testing. I confirmed my build is using the same command line, so I
can only surmise that I must be using a different version of sphinx.

AFAICT your change should have fixed this error. Any idea why it is still
giving it?

Thanks,
Teresa


On Thu, Sep 22, 2016 at 6:10 AM, Teresa Johnson 
wrote:

> Thank you!
> Teresa
>
> On Thu, Sep 22, 2016 at 5:15 AM, Aaron Ballman via cfe-commits <
> cfe-commits@lists.llvm.org> wrote:
>
>> Author: aaronballman
>> Date: Thu Sep 22 07:15:18 2016
>> New Revision: 282148
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=282148&view=rev
>> Log:
>> Fixing sphinx build due to diagnostic:
>>
>> /opt/llvm/build.attributes.src/tools/clang/docs/CommandGuide/clang.rst:338:
>> WARNING: unknown option: -flto=full
>>
>> Modified:
>> cfe/trunk/docs/CommandGuide/clang.rst
>>
>> Modified: cfe/trunk/docs/CommandGuide/clang.rst
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/CommandGu
>> ide/clang.rst?rev=282148&r1=282147&r2=282148&view=diff
>> 
>> ==
>> --- cfe/trunk/docs/CommandGuide/clang.rst (original)
>> +++ cfe/trunk/docs/CommandGuide/clang.rst Thu Sep 22 07:15:18 2016
>> @@ -328,7 +328,7 @@ Code Generation Options
>>model can be overridden with the tls_model attribute. The compiler
>> will try
>>to choose a more efficient model if possible.
>>
>> -.. option:: -flto[=full,thin], -emit-llvm
>> +.. option:: -flto, -flto=full, -flto=thin, -emit-llvm
>>
>>Generate output files in LLVM formats, suitable for link time
>> optimization.
>>When used with :option:`-S` this generates LLVM intermediate language
>>
>>
>> ___
>> cfe-commits mailing list
>> cfe-commits@lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>>
>
>
>
> --
> Teresa Johnson |  Software Engineer |  tejohn...@google.com |
> 408-460-2413
>



-- 
Teresa Johnson |  Software Engineer |  tejohn...@google.com |  408-460-2413
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: r282148 - Fixing sphinx build due to diagnostic:

[Teresa Johnson via cfe-commits]

On Thu, Sep 22, 2016 at 6:38 AM, Aaron Ballman 
wrote:

> On Thu, Sep 22, 2016 at 9:34 AM, Teresa Johnson 
> wrote:
> > Hi Aaron,
> >
> > I just went back to the bot, which had first failed with a process error
> > when I made my commit which is why I didn't see this error. It looks
> like it
> > is still failing with the same error:
> >
> > http://lab.llvm.org:8011/builders/clang-sphinx-docs/
> builds/16313/steps/docs-clang-html/logs/stdio
> > (that build has your fix).
> >
> > Unfortunately, I can't reproduce this locally, which is why I didn't see
> it
> > in my testing. I confirmed my build is using the same command line, so I
> can
> > only surmise that I must be using a different version of sphinx.
>
> I run into the same problem -- I cannot reproduce it locally either.
> I've just hit this problem several times before and was speculatively
> fixing it. I have no idea how this bot is configured and why
> configuration is different. :-(
>
> > AFAICT your change should have fixed this error. Any idea why it is still
> > giving it?
>
> The option directives are really persnickety about how things are
> spelled. It's possible that it just doesn't like the "=blah" part at
> all. We could remove the :option: usage from the paragraph and leave
> the .. option:: directive in the title. That's another way I've solved
> this in the past.
>

Ok, I'll go ahead and do that. It's odd that it doesn't seem to be
complaining about the -flto=thin reference a few lines below.

Do you know if the pages at http://clang.llvm.org/docs/ will automatically
be refreshed once this is fixed?

Teresa


> ~Aaron
>
> >
> > Thanks,
> > Teresa
> >
> >
> > On Thu, Sep 22, 2016 at 6:10 AM, Teresa Johnson 
> > wrote:
> >>
> >> Thank you!
> >> Teresa
> >>
> >> On Thu, Sep 22, 2016 at 5:15 AM, Aaron Ballman via cfe-commits
> >>  wrote:
> >>>
> >>> Author: aaronballman
> >>> Date: Thu Sep 22 07:15:18 2016
> >>> New Revision: 282148
> >>>
> >>> URL: http://llvm.org/viewvc/llvm-project?rev=282148&view=rev
> >>> Log:
> >>> Fixing sphinx build due to diagnostic:
> >>>
> >>>
> >>> /opt/llvm/build.attributes.src/tools/clang/docs/
> CommandGuide/clang.rst:338:
> >>> WARNING: unknown option: -flto=full
> >>>
> >>> Modified:
> >>> cfe/trunk/docs/CommandGuide/clang.rst
> >>>
> >>> Modified: cfe/trunk/docs/CommandGuide/clang.rst
> >>> URL:
> >>> http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/
> CommandGuide/clang.rst?rev=282148&r1=282147&r2=282148&view=diff
> >>>
> >>> 
> ==
> >>> --- cfe/trunk/docs/CommandGuide/clang.rst (original)
> >>> +++ cfe/trunk/docs/CommandGuide/clang.rst Thu Sep 22 07:15:18 2016
> >>> @@ -328,7 +328,7 @@ Code Generation Options
> >>>model can be overridden with the tls_model attribute. The compiler
> >>> will try
> >>>to choose a more efficient model if possible.
> >>>
> >>> -.. option:: -flto[=full,thin], -emit-llvm
> >>> +.. option:: -flto, -flto=full, -flto=thin, -emit-llvm
> >>>
> >>>Generate output files in LLVM formats, suitable for link time
> >>> optimization.
> >>>When used with :option:`-S` this generates LLVM intermediate
> language
> >>>
> >>>
> >>> ___
> >>> cfe-commits mailing list
> >>> cfe-commits@lists.llvm.org
> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
> >>
> >>
> >>
> >>
> >> --
> >> Teresa Johnson | Software Engineer | tejohn...@google.com |
> 408-460-2413
> >
> >
> >
> >
> > --
> > Teresa Johnson | Software Engineer | tejohn...@google.com | 408-460-2413
>



-- 
Teresa Johnson |  Software Engineer |  tejohn...@google.com |  408-460-2413
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r282151 - Second attempt to fix Sphinx bot

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Thu Sep 22 08:41:10 2016
New Revision: 282151

URL: http://llvm.org/viewvc/llvm-project?rev=282151&view=rev
Log:
Second attempt to fix Sphinx bot

The fix in r282148 was not enough to fix the following error:
/home/llvmbb/llvm-build-dir/clang-sphinx-docs/llvm/src/tools/clang/docs/CommandGuide/clang.rst:338:
WARNING: unknown option: -flto=full

on the sphinx bot:
  http://lab.llvm.org:8011/builders/clang-sphinx-docs/builds/16313

(not reproducible locally).

This time, simply remove the option reference.

Modified:
cfe/trunk/docs/CommandGuide/clang.rst

Modified: cfe/trunk/docs/CommandGuide/clang.rst
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/CommandGuide/clang.rst?rev=282151&r1=282150&r2=282151&view=diff
==
--- cfe/trunk/docs/CommandGuide/clang.rst (original)
+++ cfe/trunk/docs/CommandGuide/clang.rst Thu Sep 22 08:41:10 2016
@@ -335,7 +335,7 @@ Code Generation Options
   assembly files, otherwise this generates LLVM bitcode format object files
   (which may be passed to the linker depending on the stage selection options).
 
-  The default for :option:`-flto` is :option:`-flto=full`, in which the
+  The default for :option:`-flto` is "full", in which the
   LLVM bitcode is suitable for monolithic Link Time Optimization (LTO), where
   the linker merges all such modules into a single combined module for
   optimization. With :option:`-flto=thin`, :doc:`ThinLTO <../ThinLTO>`


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: r282148 - Fixing sphinx build due to diagnostic:

[Teresa Johnson via cfe-commits]

On Thu, Sep 22, 2016 at 6:43 AM, Aaron Ballman 
wrote:

> On Thu, Sep 22, 2016 at 9:41 AM, Teresa Johnson 
> wrote:
> >
> >
> >
> > On Thu, Sep 22, 2016 at 6:38 AM, Aaron Ballman 
> wrote:
> >>
> >> On Thu, Sep 22, 2016 at 9:34 AM, Teresa Johnson 
> wrote:
> >> > Hi Aaron,
> >> >
> >> > I just went back to the bot, which had first failed with a process
> error
> >> > when I made my commit which is why I didn't see this error. It looks
> like it
> >> > is still failing with the same error:
> >> >
> >> > http://lab.llvm.org:8011/builders/clang-sphinx-docs/
> builds/16313/steps/docs-clang-html/logs/stdio
> >> > (that build has your fix).
> >> >
> >> > Unfortunately, I can't reproduce this locally, which is why I didn't
> see it
> >> > in my testing. I confirmed my build is using the same command line,
> so I can
> >> > only surmise that I must be using a different version of sphinx.
> >>
> >> I run into the same problem -- I cannot reproduce it locally either.
> >> I've just hit this problem several times before and was speculatively
> >> fixing it. I have no idea how this bot is configured and why
> >> configuration is different. :-(
> >>
> >> > AFAICT your change should have fixed this error. Any idea why it is
> still
> >> > giving it?
> >>
> >> The option directives are really persnickety about how things are
> >> spelled. It's possible that it just doesn't like the "=blah" part at
> >> all. We could remove the :option: usage from the paragraph and leave
> >> the .. option:: directive in the title. That's another way I've solved
> >> this in the past.
> >
> >
> > Ok, I'll go ahead and do that. It's odd that it doesn't seem to be
> complaining about the -flto=thin reference a few lines below.
>
> Thank you for taking care of it! Also, welcome to the world of Sphinx,
> where that's not odd, it's The Way Things Work. :-D Once Sphinx hits
> an error, it seems to stop reporting more errors.
>

I only fixed the -flto=full reference (r282151) - will keep an eye on the
bot and if it starts complaining about the -flto=thin reference next I will
do the same there!


> > Do you know if the pages at http://clang.llvm.org/docs/ will
> automatically be refreshed once this is fixed?
>
> I believe they should, yes.
>

Ok, great, thanks

Teresa


>
> ~Aaron
>
> >
> > Teresa
> >
> >>
> >> ~Aaron
> >>
> >> >
> >> > Thanks,
> >> > Teresa
> >> >
> >> >
> >> > On Thu, Sep 22, 2016 at 6:10 AM, Teresa Johnson  >
> >> > wrote:
> >> >>
> >> >> Thank you!
> >> >> Teresa
> >> >>
> >> >> On Thu, Sep 22, 2016 at 5:15 AM, Aaron Ballman via cfe-commits
> >> >>  wrote:
> >> >>>
> >> >>> Author: aaronballman
> >> >>> Date: Thu Sep 22 07:15:18 2016
> >> >>> New Revision: 282148
> >> >>>
> >> >>> URL: http://llvm.org/viewvc/llvm-project?rev=282148&view=rev
> >> >>> Log:
> >> >>> Fixing sphinx build due to diagnostic:
> >> >>>
> >> >>>
> >> >>> /opt/llvm/build.attributes.src/tools/clang/docs/
> CommandGuide/clang.rst:338:
> >> >>> WARNING: unknown option: -flto=full
> >> >>>
> >> >>> Modified:
> >> >>> cfe/trunk/docs/CommandGuide/clang.rst
> >> >>>
> >> >>> Modified: cfe/trunk/docs/CommandGuide/clang.rst
> >> >>> URL:
> >> >>> http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/
> CommandGuide/clang.rst?rev=282148&r1=282147&r2=282148&view=diff
> >> >>>
> >> >>> 
> ==
> >> >>> --- cfe/trunk/docs/CommandGuide/clang.rst (original)
> >> >>> +++ cfe/trunk/docs/CommandGuide/clang.rst Thu Sep 22 07:15:18 2016
> >> >>> @@ -328,7 +328,7 @@ Code Generation Options
> >> >>>model can be overridden with the tls_model attribute. The
> compiler
> >> >>> will try
> >> >>>to choose a more efficient model if possible.
> >> >>>
> >> >>> -.. option:: -flto[=full,thin], -emit-llvm
> >> >>> +.. option:: -flto, -flto=full, -flto=thin, -emit-llvm
> >> >>>
> >> >>>Generate output files in LLVM formats, suitable for link time
> >> >>> optimization.
> >> >>>When used with :option:`-S` this generates LLVM intermediate
> language
> >> >>>
> >> >>>
> >> >>> ___
> >> >>> cfe-commits mailing list
> >> >>> cfe-commits@lists.llvm.org
> >> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Teresa Johnson | Software Engineer | tejohn...@google.com |
> 408-460-2413
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Teresa Johnson | Software Engineer | tejohn...@google.com |
> 408-460-2413
> >
> >
> >
> >
> > --
> > Teresa Johnson | Software Engineer | tejohn...@google.com | 408-460-2413
>



-- 
Teresa Johnson |  Software Engineer |  tejohn...@google.com |  408-460-2413
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D24826: [LTO] Add -flto-jobs=N to control backend parallelism

[Teresa Johnson via cfe-commits]

tejohnson created this revision.
tejohnson added a reviewer: mehdi_amini.
tejohnson added a subscriber: cfe-commits.
Herald added a subscriber: mehdi_amini.

Currently, a linker option must be used to control the backend
parallelism of ThinLTO or full LTO (where it only applies to
parallel code gen). The linker option varies depending on the
linker (e.g. gold vs ld64). Add a new clang option -flto-jobs=N
to control this.

I've added in the wiring to pass this to the gold plugin. I also
added in the logic to pass this down in the form I understand that
ld64 uses on MacOS, for the darwin target.

https://reviews.llvm.org/D24826

Files:
  include/clang/Driver/Options.td
  lib/Driver/Tools.cpp
  test/Driver/lto-jobs.c

Index: test/Driver/lto-jobs.c
===
--- /dev/null
+++ test/Driver/lto-jobs.c
@@ -0,0 +1,11 @@
+// Confirm that -flto-jobs=N is passed to linker
+
+// RUN: %clang -target x86_64-unknown-linux -### %s -flto=thin -flto-jobs=5 2> %t
+// RUN: FileCheck -check-prefix=CHECK-LINK-THIN-JOBS-ACTION < %t %s
+//
+// CHECK-LINK-THIN-JOBS-ACTION: "-plugin-opt=jobs=5"
+
+// RUN: %clang -target x86_64-apple-darwin13.3.0 -### %s -flto=thin -flto-jobs=5 2> %t
+// RUN: FileCheck -check-prefix=CHECK-LINK-THIN-JOBS2-ACTION < %t %s
+//
+// CHECK-LINK-THIN-JOBS2-ACTION: "-mllvm" "-threads=5"
Index: lib/Driver/Tools.cpp
===
--- lib/Driver/Tools.cpp
+++ lib/Driver/Tools.cpp
@@ -1998,8 +1998,19 @@
   }
 }
 
+static unsigned getLTOParallelism(const ArgList &Args, const Driver &D) {
+  unsigned Parallelism = 0;
+  Arg *LtoJobsArg = Args.getLastArg(options::OPT_flto_jobs_EQ);
+  if (LtoJobsArg &&
+  StringRef(LtoJobsArg->getValue()).getAsInteger(10, Parallelism))
+D.Diag(diag::err_drv_invalid_int_value) << LtoJobsArg->getAsString(Args)
+<< LtoJobsArg->getValue();
+  return Parallelism;
+}
+
 static void AddGoldPlugin(const ToolChain &ToolChain, const ArgList &Args,
-  ArgStringList &CmdArgs, bool IsThinLTO) {
+  ArgStringList &CmdArgs, bool IsThinLTO,
+  const Driver &D) {
   // Tell the linker to load the plugin. This has to come before AddLinkerInputs
   // as gold requires -plugin to come before any -plugin-opt that -Wl might
   // forward.
@@ -2032,6 +2043,10 @@
   if (IsThinLTO)
 CmdArgs.push_back("-plugin-opt=thinlto");
 
+  if (unsigned Parallelism = getLTOParallelism(Args, D))
+CmdArgs.push_back(Args.MakeArgString(Twine("-plugin-opt=jobs=") +
+ std::to_string(Parallelism)));
+
   // If an explicit debugger tuning argument appeared, pass it along.
   if (Arg *A = Args.getLastArg(options::OPT_gTune_Group,
options::OPT_ggdbN_Group)) {
@@ -7639,7 +7654,7 @@
options::OPT_t, options::OPT_Z_Flag, options::OPT_r});
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs);
 
@@ -8060,6 +8075,13 @@
 
   getMachOToolChain().addProfileRTLibs(Args, CmdArgs);
 
+  if (unsigned Parallelism =
+  getLTOParallelism(Args, getToolChain().getDriver())) {
+CmdArgs.push_back("-mllvm");
+CmdArgs.push_back(
+Args.MakeArgString(Twine("-threads=") + std::to_string(Parallelism)));
+  }
+
   if (!Args.hasArg(options::OPT_nostdlib, options::OPT_nodefaultlibs)) {
 if (getToolChain().getDriver().CCCIsCXX())
   getToolChain().AddCXXStdlibLibArgs(Args, CmdArgs);
@@ -8790,7 +8812,7 @@
   Args.AddAllArgs(CmdArgs, options::OPT_r);
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   bool NeedsSanitizerDeps = addSanitizerRuntimes(ToolChain, Args, CmdArgs);
   AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs);
@@ -9623,7 +9645,7 @@
   ToolChain.AddFilePathLibArgs(Args, CmdArgs);
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   if (Args.hasArg(options::OPT_Z_Xlinker__no_demangle))
 CmdArgs.push_back("--no-demangle");
Index: include/clang/Driver/Options.td
===
--- include/clang/Driver/Options.td
+++ include/clang/Driver/Options.td
@@ -812,6 +812,11 @@
   HelpText<"Enable LTO in 'full' mode">;
 def fno_lto : Flag<["-"], "fno-lto">, Group,
   HelpText<"Disable LTO mode (default)">;
+def flto_jobs_EQ : Joined<["-"], "flto-jobs=">,
+  Flags<[CC1Option]>, Group,
+  HelpText<"Controls the backend parallelism of -flto=thin (default "
+   "std::thread::hardware_concurrency) or the co

r282154 - Third attempt to fix Sphinx bot

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Thu Sep 22 08:58:33 2016
New Revision: 282154

URL: http://llvm.org/viewvc/llvm-project?rev=282154&view=rev
Log:
Third attempt to fix Sphinx bot

Bot now complaining about -flto=thin reference, used similar workaround
for that failure.

Modified:
cfe/trunk/docs/CommandGuide/clang.rst

Modified: cfe/trunk/docs/CommandGuide/clang.rst
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/CommandGuide/clang.rst?rev=282154&r1=282153&r2=282154&view=diff
==
--- cfe/trunk/docs/CommandGuide/clang.rst (original)
+++ cfe/trunk/docs/CommandGuide/clang.rst Thu Sep 22 08:58:33 2016
@@ -338,7 +338,7 @@ Code Generation Options
   The default for :option:`-flto` is "full", in which the
   LLVM bitcode is suitable for monolithic Link Time Optimization (LTO), where
   the linker merges all such modules into a single combined module for
-  optimization. With :option:`-flto=thin`, :doc:`ThinLTO <../ThinLTO>`
+  optimization. With "thin", :doc:`ThinLTO <../ThinLTO>`
   compilation is invoked instead.
 
 Driver Options


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D24826: [LTO] Add -flto-jobs=N to control backend parallelism

[Teresa Johnson via cfe-commits]

tejohnson added a comment.

In https://reviews.llvm.org/D24826#549788, @mehdi_amini wrote:

> The Gold path looks fine. 
>  On OSX, we would have the clang driver relying on a LLVM cl::opt, for which 
> I don't think there is any precedent. CC Duncan for advice.


I do see other uses of -mllvm in lib/Driver/Tools.cpp, but are you talking 
about something else?

> Also I don't think the same option should be used for the parallel LTO 
> codegen: it actually does not generate the same binary, which should deserve 
> a dedicated opt-in (What if I mix ThinLTO and LTO, and I don't want // 
> codegen?)


Ok good point. I can change this to -fthinlto_jobs. However, while the two 
parallel settings are separate in the LTO API, currently the gold-plugin jobs 
option controls both, so I will need to do a preparatory gold-plugin patch to 
split this into thinlto_jobs and lto_jobs. On the libLTO/ld64 path, looks like 
the current -mllvm -threads only affects ThinLTO so there is no work to do 
there.


https://reviews.llvm.org/D24826



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D24826: [LTO] Add -flto-jobs=N to control backend parallelism

[Teresa Johnson via cfe-commits]

tejohnson updated this revision to Diff 72301.
tejohnson added a comment.

Update option description as per decision to split from parallel code gen.


https://reviews.llvm.org/D24826

Files:
  include/clang/Driver/Options.td
  lib/Driver/Tools.cpp
  test/Driver/lto-jobs.c

Index: test/Driver/lto-jobs.c
===
--- /dev/null
+++ test/Driver/lto-jobs.c
@@ -0,0 +1,11 @@
+// Confirm that -flto-jobs=N is passed to linker
+
+// RUN: %clang -target x86_64-unknown-linux -### %s -flto=thin -flto-jobs=5 2> %t
+// RUN: FileCheck -check-prefix=CHECK-LINK-THIN-JOBS-ACTION < %t %s
+//
+// CHECK-LINK-THIN-JOBS-ACTION: "-plugin-opt=jobs=5"
+
+// RUN: %clang -target x86_64-apple-darwin13.3.0 -### %s -flto=thin -flto-jobs=5 2> %t
+// RUN: FileCheck -check-prefix=CHECK-LINK-THIN-JOBS2-ACTION < %t %s
+//
+// CHECK-LINK-THIN-JOBS2-ACTION: "-mllvm" "-threads=5"
Index: lib/Driver/Tools.cpp
===
--- lib/Driver/Tools.cpp
+++ lib/Driver/Tools.cpp
@@ -1998,8 +1998,19 @@
   }
 }
 
+static unsigned getLTOParallelism(const ArgList &Args, const Driver &D) {
+  unsigned Parallelism = 0;
+  Arg *LtoJobsArg = Args.getLastArg(options::OPT_flto_jobs_EQ);
+  if (LtoJobsArg &&
+  StringRef(LtoJobsArg->getValue()).getAsInteger(10, Parallelism))
+D.Diag(diag::err_drv_invalid_int_value) << LtoJobsArg->getAsString(Args)
+<< LtoJobsArg->getValue();
+  return Parallelism;
+}
+
 static void AddGoldPlugin(const ToolChain &ToolChain, const ArgList &Args,
-  ArgStringList &CmdArgs, bool IsThinLTO) {
+  ArgStringList &CmdArgs, bool IsThinLTO,
+  const Driver &D) {
   // Tell the linker to load the plugin. This has to come before AddLinkerInputs
   // as gold requires -plugin to come before any -plugin-opt that -Wl might
   // forward.
@@ -2032,6 +2043,10 @@
   if (IsThinLTO)
 CmdArgs.push_back("-plugin-opt=thinlto");
 
+  if (unsigned Parallelism = getLTOParallelism(Args, D))
+CmdArgs.push_back(Args.MakeArgString(Twine("-plugin-opt=jobs=") +
+ std::to_string(Parallelism)));
+
   // If an explicit debugger tuning argument appeared, pass it along.
   if (Arg *A = Args.getLastArg(options::OPT_gTune_Group,
options::OPT_ggdbN_Group)) {
@@ -7639,7 +7654,7 @@
options::OPT_t, options::OPT_Z_Flag, options::OPT_r});
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs);
 
@@ -8060,6 +8075,13 @@
 
   getMachOToolChain().addProfileRTLibs(Args, CmdArgs);
 
+  if (unsigned Parallelism =
+  getLTOParallelism(Args, getToolChain().getDriver())) {
+CmdArgs.push_back("-mllvm");
+CmdArgs.push_back(
+Args.MakeArgString(Twine("-threads=") + std::to_string(Parallelism)));
+  }
+
   if (!Args.hasArg(options::OPT_nostdlib, options::OPT_nodefaultlibs)) {
 if (getToolChain().getDriver().CCCIsCXX())
   getToolChain().AddCXXStdlibLibArgs(Args, CmdArgs);
@@ -8790,7 +8812,7 @@
   Args.AddAllArgs(CmdArgs, options::OPT_r);
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   bool NeedsSanitizerDeps = addSanitizerRuntimes(ToolChain, Args, CmdArgs);
   AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs);
@@ -9623,7 +9645,7 @@
   ToolChain.AddFilePathLibArgs(Args, CmdArgs);
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   if (Args.hasArg(options::OPT_Z_Xlinker__no_demangle))
 CmdArgs.push_back("--no-demangle");
Index: include/clang/Driver/Options.td
===
--- include/clang/Driver/Options.td
+++ include/clang/Driver/Options.td
@@ -812,6 +812,10 @@
   HelpText<"Enable LTO in 'full' mode">;
 def fno_lto : Flag<["-"], "fno-lto">, Group,
   HelpText<"Disable LTO mode (default)">;
+def flto_jobs_EQ : Joined<["-"], "flto-jobs=">,
+  Flags<[CC1Option]>, Group,
+  HelpText<"Controls the backend parallelism of -flto=thin (default "
+   "of 0 means use std::thread::hardware_concurrency)">;
 def fthinlto_index_EQ : Joined<["-"], "fthinlto-index=">,
   Flags<[CC1Option]>, Group,
   HelpText<"Perform ThinLTO importing using provided function summary index">;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D24826: [LTO] Add -flto-jobs=N to control backend parallelism

[Teresa Johnson via cfe-commits]

tejohnson added a comment.

> > I do see other uses of -mllvm in lib/Driver/Tools.cpp, but are you talking 
> > about something else?

> 

> I think this is okay, since clang is talking to the same version of 
> libLTO.dylib.  I feel like there might be another case where 

>  clang talks to libLTO.dylib through ld64 using -mllvm... perhaps, -O0?

> 

> Let's ask around though to be sure.


Ok, let me know what you find out.

> > Ok good point. I can change this to -fthinlto_jobs. However, while the two 
> > parallel settings are separate in the LTO API, currently the gold-plugin 
> > jobs option controls both, so I will need to do a preparatory gold-plugin 
> > patch to split this into thinlto_jobs and lto_jobs. On the libLTO/ld64 
> > path, looks like the current -mllvm -threads only affects ThinLTO so there 
> > is no work to do there.

> 




> I actually like -flto-jobs=N better for this.  I expect "jobs" not to affect 
> output at all.

> 

> I think the current parallel FullLTO CodeGen (where it *does* affect output) 
> should have a special name that calls this out, perhaps -flto-partitions=N?  
> -flto-slices=N?  -flto-random-partitions=N?  Is it urgent to add that flag 
> now though?

> 

> Note that I imagine someone will parallelizing FullLTO the hard way in the 
> future, which won't affect output.  That implementation should use 
> -flto-jobs=N.


Ok, sure that seems reasonable. I changed the option documentation to note that 
this is currently just for ThinLTO. See also https://reviews.llvm.org/D24873 
where I split the gold-plugin options.


https://reviews.llvm.org/D24826



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D24826: [LTO] Add -flto-jobs=N to control backend parallelism

[Teresa Johnson via cfe-commits]

tejohnson updated this revision to Diff 72317.
tejohnson added a comment.

Update option help message per Mehdi's suggestion


https://reviews.llvm.org/D24826

Files:
  include/clang/Driver/Options.td
  lib/Driver/Tools.cpp
  test/Driver/lto-jobs.c

Index: test/Driver/lto-jobs.c
===
--- /dev/null
+++ test/Driver/lto-jobs.c
@@ -0,0 +1,11 @@
+// Confirm that -flto-jobs=N is passed to linker
+
+// RUN: %clang -target x86_64-unknown-linux -### %s -flto=thin -flto-jobs=5 2> %t
+// RUN: FileCheck -check-prefix=CHECK-LINK-THIN-JOBS-ACTION < %t %s
+//
+// CHECK-LINK-THIN-JOBS-ACTION: "-plugin-opt=jobs=5"
+
+// RUN: %clang -target x86_64-apple-darwin13.3.0 -### %s -flto=thin -flto-jobs=5 2> %t
+// RUN: FileCheck -check-prefix=CHECK-LINK-THIN-JOBS2-ACTION < %t %s
+//
+// CHECK-LINK-THIN-JOBS2-ACTION: "-mllvm" "-threads=5"
Index: lib/Driver/Tools.cpp
===
--- lib/Driver/Tools.cpp
+++ lib/Driver/Tools.cpp
@@ -1998,8 +1998,19 @@
   }
 }
 
+static unsigned getLTOParallelism(const ArgList &Args, const Driver &D) {
+  unsigned Parallelism = 0;
+  Arg *LtoJobsArg = Args.getLastArg(options::OPT_flto_jobs_EQ);
+  if (LtoJobsArg &&
+  StringRef(LtoJobsArg->getValue()).getAsInteger(10, Parallelism))
+D.Diag(diag::err_drv_invalid_int_value) << LtoJobsArg->getAsString(Args)
+<< LtoJobsArg->getValue();
+  return Parallelism;
+}
+
 static void AddGoldPlugin(const ToolChain &ToolChain, const ArgList &Args,
-  ArgStringList &CmdArgs, bool IsThinLTO) {
+  ArgStringList &CmdArgs, bool IsThinLTO,
+  const Driver &D) {
   // Tell the linker to load the plugin. This has to come before AddLinkerInputs
   // as gold requires -plugin to come before any -plugin-opt that -Wl might
   // forward.
@@ -2032,6 +2043,10 @@
   if (IsThinLTO)
 CmdArgs.push_back("-plugin-opt=thinlto");
 
+  if (unsigned Parallelism = getLTOParallelism(Args, D))
+CmdArgs.push_back(Args.MakeArgString(Twine("-plugin-opt=jobs=") +
+ std::to_string(Parallelism)));
+
   // If an explicit debugger tuning argument appeared, pass it along.
   if (Arg *A = Args.getLastArg(options::OPT_gTune_Group,
options::OPT_ggdbN_Group)) {
@@ -7639,7 +7654,7 @@
options::OPT_t, options::OPT_Z_Flag, options::OPT_r});
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs);
 
@@ -8060,6 +8075,13 @@
 
   getMachOToolChain().addProfileRTLibs(Args, CmdArgs);
 
+  if (unsigned Parallelism =
+  getLTOParallelism(Args, getToolChain().getDriver())) {
+CmdArgs.push_back("-mllvm");
+CmdArgs.push_back(
+Args.MakeArgString(Twine("-threads=") + std::to_string(Parallelism)));
+  }
+
   if (!Args.hasArg(options::OPT_nostdlib, options::OPT_nodefaultlibs)) {
 if (getToolChain().getDriver().CCCIsCXX())
   getToolChain().AddCXXStdlibLibArgs(Args, CmdArgs);
@@ -8790,7 +8812,7 @@
   Args.AddAllArgs(CmdArgs, options::OPT_r);
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   bool NeedsSanitizerDeps = addSanitizerRuntimes(ToolChain, Args, CmdArgs);
   AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs);
@@ -9623,7 +9645,7 @@
   ToolChain.AddFilePathLibArgs(Args, CmdArgs);
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   if (Args.hasArg(options::OPT_Z_Xlinker__no_demangle))
 CmdArgs.push_back("--no-demangle");
Index: include/clang/Driver/Options.td
===
--- include/clang/Driver/Options.td
+++ include/clang/Driver/Options.td
@@ -812,6 +812,11 @@
   HelpText<"Enable LTO in 'full' mode">;
 def fno_lto : Flag<["-"], "fno-lto">, Group,
   HelpText<"Disable LTO mode (default)">;
+def flto_jobs_EQ : Joined<["-"], "flto-jobs=">,
+  Flags<[CC1Option]>, Group,
+  HelpText<"Controls the backend parallelism of -flto=thin (default "
+   "of 0 means the number of threads will be derived from "
+   "the number of CPUs detected)">;
 def fthinlto_index_EQ : Joined<["-"], "fthinlto-index=">,
   Flags<[CC1Option]>, Group,
   HelpText<"Perform ThinLTO importing using provided function summary index">;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D24826: [LTO] Add -flto-jobs=N to control backend parallelism

[Teresa Johnson via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rL282291: [LTO] Add -flto-jobs=N to control backend 
parallelism (authored by tejohnson).

Changed prior to commit:
  https://reviews.llvm.org/D24826?vs=72317&id=72349#toc

Repository:
  rL LLVM

https://reviews.llvm.org/D24826

Files:
  cfe/trunk/include/clang/Driver/Options.td
  cfe/trunk/lib/Driver/Tools.cpp
  cfe/trunk/test/Driver/lto-jobs.c

Index: cfe/trunk/include/clang/Driver/Options.td
===
--- cfe/trunk/include/clang/Driver/Options.td
+++ cfe/trunk/include/clang/Driver/Options.td
@@ -812,6 +812,11 @@
   HelpText<"Enable LTO in 'full' mode">;
 def fno_lto : Flag<["-"], "fno-lto">, Group,
   HelpText<"Disable LTO mode (default)">;
+def flto_jobs_EQ : Joined<["-"], "flto-jobs=">,
+  Flags<[CC1Option]>, Group,
+  HelpText<"Controls the backend parallelism of -flto=thin (default "
+   "of 0 means the number of threads will be derived from "
+   "the number of CPUs detected)">;
 def fthinlto_index_EQ : Joined<["-"], "fthinlto-index=">,
   Flags<[CC1Option]>, Group,
   HelpText<"Perform ThinLTO importing using provided function summary index">;
Index: cfe/trunk/test/Driver/lto-jobs.c
===
--- cfe/trunk/test/Driver/lto-jobs.c
+++ cfe/trunk/test/Driver/lto-jobs.c
@@ -0,0 +1,11 @@
+// Confirm that -flto-jobs=N is passed to linker
+
+// RUN: %clang -target x86_64-unknown-linux -### %s -flto=thin -flto-jobs=5 2> %t
+// RUN: FileCheck -check-prefix=CHECK-LINK-THIN-JOBS-ACTION < %t %s
+//
+// CHECK-LINK-THIN-JOBS-ACTION: "-plugin-opt=jobs=5"
+
+// RUN: %clang -target x86_64-apple-darwin13.3.0 -### %s -flto=thin -flto-jobs=5 2> %t
+// RUN: FileCheck -check-prefix=CHECK-LINK-THIN-JOBS2-ACTION < %t %s
+//
+// CHECK-LINK-THIN-JOBS2-ACTION: "-mllvm" "-threads=5"
Index: cfe/trunk/lib/Driver/Tools.cpp
===
--- cfe/trunk/lib/Driver/Tools.cpp
+++ cfe/trunk/lib/Driver/Tools.cpp
@@ -1998,8 +1998,19 @@
   }
 }
 
+static unsigned getLTOParallelism(const ArgList &Args, const Driver &D) {
+  unsigned Parallelism = 0;
+  Arg *LtoJobsArg = Args.getLastArg(options::OPT_flto_jobs_EQ);
+  if (LtoJobsArg &&
+  StringRef(LtoJobsArg->getValue()).getAsInteger(10, Parallelism))
+D.Diag(diag::err_drv_invalid_int_value) << LtoJobsArg->getAsString(Args)
+<< LtoJobsArg->getValue();
+  return Parallelism;
+}
+
 static void AddGoldPlugin(const ToolChain &ToolChain, const ArgList &Args,
-  ArgStringList &CmdArgs, bool IsThinLTO) {
+  ArgStringList &CmdArgs, bool IsThinLTO,
+  const Driver &D) {
   // Tell the linker to load the plugin. This has to come before AddLinkerInputs
   // as gold requires -plugin to come before any -plugin-opt that -Wl might
   // forward.
@@ -2032,6 +2043,10 @@
   if (IsThinLTO)
 CmdArgs.push_back("-plugin-opt=thinlto");
 
+  if (unsigned Parallelism = getLTOParallelism(Args, D))
+CmdArgs.push_back(Args.MakeArgString(Twine("-plugin-opt=jobs=") +
+ std::to_string(Parallelism)));
+
   // If an explicit debugger tuning argument appeared, pass it along.
   if (Arg *A = Args.getLastArg(options::OPT_gTune_Group,
options::OPT_ggdbN_Group)) {
@@ -7639,7 +7654,7 @@
options::OPT_t, options::OPT_Z_Flag, options::OPT_r});
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs);
 
@@ -8060,6 +8075,13 @@
 
   getMachOToolChain().addProfileRTLibs(Args, CmdArgs);
 
+  if (unsigned Parallelism =
+  getLTOParallelism(Args, getToolChain().getDriver())) {
+CmdArgs.push_back("-mllvm");
+CmdArgs.push_back(
+Args.MakeArgString(Twine("-threads=") + std::to_string(Parallelism)));
+  }
+
   if (!Args.hasArg(options::OPT_nostdlib, options::OPT_nodefaultlibs)) {
 if (getToolChain().getDriver().CCCIsCXX())
   getToolChain().AddCXXStdlibLibArgs(Args, CmdArgs);
@@ -8790,7 +8812,7 @@
   Args.AddAllArgs(CmdArgs, options::OPT_r);
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   bool NeedsSanitizerDeps = addSanitizerRuntimes(ToolChain, Args, CmdArgs);
   AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs);
@@ -9623,7 +9645,7 @@
   ToolChain.AddFilePathLibArgs(Args, CmdArgs);
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   if (Args.hasArg(options::OPT_Z_Xlinker__no_demangle

r282291 - [LTO] Add -flto-jobs=N to control backend parallelism

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Fri Sep 23 15:38:09 2016
New Revision: 282291

URL: http://llvm.org/viewvc/llvm-project?rev=282291&view=rev
Log:
[LTO] Add -flto-jobs=N to control backend parallelism

Summary:
Currently, a linker option must be used to control the backend
parallelism of ThinLTO. The linker option varies depending on the
linker (e.g. gold vs ld64). Add a new clang option -flto-jobs=N
to control this.

I've added in the wiring to pass this to the gold plugin. I also
added in the logic to pass this down in the form I understand that
ld64 uses on MacOS, for the darwin target.

Reviewers: mehdi_amini, dexonsmith

Subscribers: mehdi_amini, cfe-commits

Differential Revision: https://reviews.llvm.org/D24826

Added:
cfe/trunk/test/Driver/lto-jobs.c
Modified:
cfe/trunk/include/clang/Driver/Options.td
cfe/trunk/lib/Driver/Tools.cpp

Modified: cfe/trunk/include/clang/Driver/Options.td
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Options.td?rev=282291&r1=282290&r2=282291&view=diff
==
--- cfe/trunk/include/clang/Driver/Options.td (original)
+++ cfe/trunk/include/clang/Driver/Options.td Fri Sep 23 15:38:09 2016
@@ -812,6 +812,11 @@ def flto : Flag<["-"], "flto">, Flags<[C
   HelpText<"Enable LTO in 'full' mode">;
 def fno_lto : Flag<["-"], "fno-lto">, Group,
   HelpText<"Disable LTO mode (default)">;
+def flto_jobs_EQ : Joined<["-"], "flto-jobs=">,
+  Flags<[CC1Option]>, Group,
+  HelpText<"Controls the backend parallelism of -flto=thin (default "
+   "of 0 means the number of threads will be derived from "
+   "the number of CPUs detected)">;
 def fthinlto_index_EQ : Joined<["-"], "fthinlto-index=">,
   Flags<[CC1Option]>, Group,
   HelpText<"Perform ThinLTO importing using provided function summary index">;

Modified: cfe/trunk/lib/Driver/Tools.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Tools.cpp?rev=282291&r1=282290&r2=282291&view=diff
==
--- cfe/trunk/lib/Driver/Tools.cpp (original)
+++ cfe/trunk/lib/Driver/Tools.cpp Fri Sep 23 15:38:09 2016
@@ -1998,8 +1998,19 @@ static std::string getCPUName(const ArgL
   }
 }
 
+static unsigned getLTOParallelism(const ArgList &Args, const Driver &D) {
+  unsigned Parallelism = 0;
+  Arg *LtoJobsArg = Args.getLastArg(options::OPT_flto_jobs_EQ);
+  if (LtoJobsArg &&
+  StringRef(LtoJobsArg->getValue()).getAsInteger(10, Parallelism))
+D.Diag(diag::err_drv_invalid_int_value) << LtoJobsArg->getAsString(Args)
+<< LtoJobsArg->getValue();
+  return Parallelism;
+}
+
 static void AddGoldPlugin(const ToolChain &ToolChain, const ArgList &Args,
-  ArgStringList &CmdArgs, bool IsThinLTO) {
+  ArgStringList &CmdArgs, bool IsThinLTO,
+  const Driver &D) {
   // Tell the linker to load the plugin. This has to come before 
AddLinkerInputs
   // as gold requires -plugin to come before any -plugin-opt that -Wl might
   // forward.
@@ -2032,6 +2043,10 @@ static void AddGoldPlugin(const ToolChai
   if (IsThinLTO)
 CmdArgs.push_back("-plugin-opt=thinlto");
 
+  if (unsigned Parallelism = getLTOParallelism(Args, D))
+CmdArgs.push_back(Args.MakeArgString(Twine("-plugin-opt=jobs=") +
+ std::to_string(Parallelism)));
+
   // If an explicit debugger tuning argument appeared, pass it along.
   if (Arg *A = Args.getLastArg(options::OPT_gTune_Group,
options::OPT_ggdbN_Group)) {
@@ -7639,7 +7654,7 @@ void cloudabi::Linker::ConstructJob(Comp
options::OPT_t, options::OPT_Z_Flag, options::OPT_r});
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs);
 
@@ -8060,6 +8075,13 @@ void darwin::Linker::ConstructJob(Compil
 
   getMachOToolChain().addProfileRTLibs(Args, CmdArgs);
 
+  if (unsigned Parallelism =
+  getLTOParallelism(Args, getToolChain().getDriver())) {
+CmdArgs.push_back("-mllvm");
+CmdArgs.push_back(
+Args.MakeArgString(Twine("-threads=") + std::to_string(Parallelism)));
+  }
+
   if (!Args.hasArg(options::OPT_nostdlib, options::OPT_nodefaultlibs)) {
 if (getToolChain().getDriver().CCCIsCXX())
   getToolChain().AddCXXStdlibLibArgs(Args, CmdArgs);
@@ -8790,7 +8812,7 @@ void freebsd::Linker::ConstructJob(Compi
   Args.AddAllArgs(CmdArgs, options::OPT_r);
 
   if (D.isUsingLTO())
-AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin);
+AddGoldPlugin(ToolChain, Args, CmdArgs, D.getLTOMode() == LTOK_Thin, D);
 
   bool NeedsSanitizerDeps = addSanitizerRuntimes(ToolChain, Args, CmdArgs);
   AddLinkerInputs(ToolChain, Inputs,

r282308 - Use llvm::to_string instead of std::to_string to fix bot

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Fri Sep 23 17:25:03 2016
New Revision: 282308

URL: http://llvm.org/viewvc/llvm-project?rev=282308&view=rev
Log:
Use llvm::to_string instead of std::to_string to fix bot

This should fix the android build in this bot:
http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-buildserver/builds/11143

Modified:
cfe/trunk/lib/Driver/Tools.cpp

Modified: cfe/trunk/lib/Driver/Tools.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Tools.cpp?rev=282308&r1=282307&r2=282308&view=diff
==
--- cfe/trunk/lib/Driver/Tools.cpp (original)
+++ cfe/trunk/lib/Driver/Tools.cpp Fri Sep 23 17:25:03 2016
@@ -41,6 +41,7 @@
 #include "llvm/Support/Process.h"
 #include "llvm/Support/Program.h"
 #include "llvm/Support/raw_ostream.h"
+#include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/TargetParser.h"
 
 #ifdef LLVM_ON_UNIX
@@ -2045,7 +2046,7 @@ static void AddGoldPlugin(const ToolChai
 
   if (unsigned Parallelism = getLTOParallelism(Args, D))
 CmdArgs.push_back(Args.MakeArgString(Twine("-plugin-opt=jobs=") +
- std::to_string(Parallelism)));
+ llvm::to_string(Parallelism)));
 
   // If an explicit debugger tuning argument appeared, pass it along.
   if (Arg *A = Args.getLastArg(options::OPT_gTune_Group,
@@ -8079,7 +8080,7 @@ void darwin::Linker::ConstructJob(Compil
   getLTOParallelism(Args, getToolChain().getDriver())) {
 CmdArgs.push_back("-mllvm");
 CmdArgs.push_back(
-Args.MakeArgString(Twine("-threads=") + std::to_string(Parallelism)));
+Args.MakeArgString(Twine("-threads=") + llvm::to_string(Parallelism)));
   }
 
   if (!Args.hasArg(options::OPT_nostdlib, options::OPT_nodefaultlibs)) {


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D24826: [LTO] Add -flto-jobs=N to control backend parallelism

[Teresa Johnson via cfe-commits]

On Fri, Sep 23, 2016, 4:38 PM Andrew Ford  wrote:

> andrewford added a subscriber: andrewford.
> andrewford added a comment.
>
> This is breaking our android lldb build, because it uses std::to_string.
> Looks like there is llvm::to_string, which should be preferred. Would
> someone mind changing it?  I don't have commit access or I would myself. :)
>

Saw that and already fixed in rL282308.

Teresa


> Thanks in advance
>
>
> Repository:
>   rL LLVM
>
> https://reviews.llvm.org/D24826
>
>
>
>
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D24644: Pass -ffunction-sections/-fdata-sections along to gold-plugin

[Teresa Johnson via cfe-commits]

tejohnson added a comment.

In https://reviews.llvm.org/D24644#557774, @pcc wrote:

> Perhaps it would be better to enable -ffunction-sections/-fdata-sections 
> unconditionally at the linker level if the linker supports it? See also 
> PR22999.


My understanding is that these are not typically the default because it makes 
the resulting object files larger and the link slower. OTOH, for ThinLTO we 
definitely benefit more from these optimizations (and from PR22999 looks like 
there is a compelling reason to want this for LTO as well).

However, when you say enable them "unconditionally at the linker level" what do 
you mean - these need to go LLVM via plugin options. Are you suggesting having 
them enabled unconditionally in both gold-plugin and gold? That will require 
changes to both llvm and binutils, and the latter will have effects for other 
compilers. What about -Wl,-gc-sections, isn't that also needed to make 
effective use of these options?


https://reviews.llvm.org/D24644



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D24644: Pass -ffunction-sections/-fdata-sections along to gold-plugin

[Teresa Johnson via cfe-commits]

tejohnson added a comment.

Ping. It seems like using attributes is not feasible at this time due to the 
lack of data attributes.


https://reviews.llvm.org/D24644



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D25537: [ThinLTO] Update doc to include lld

[Teresa Johnson via cfe-commits]

tejohnson accepted this revision.
tejohnson added a comment.
This revision is now accepted and ready to land.

LGTM, thanks!


https://reviews.llvm.org/D25537



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

r284140 - Pass -ffunction-sections/-fdata-sections along to gold-plugin

[Teresa Johnson via cfe-commits]

Author: tejohnson
Date: Thu Oct 13 13:05:53 2016
New Revision: 284140

URL: http://llvm.org/viewvc/llvm-project?rev=284140&view=rev
Log:
Pass -ffunction-sections/-fdata-sections along to gold-plugin

Summary:
These options need to be passed to the plugin in order to have
an effect on LTO/ThinLTO compiles.

Reviewers: mehdi_amini, pcc

Subscribers: jfb, dschuff, mehdi_amini, cfe-commits

Differential Revision: https://reviews.llvm.org/D24644

Added:
cfe/trunk/test/Driver/gold-lto-sections.c
Modified:
cfe/trunk/lib/Driver/Tools.cpp

Modified: cfe/trunk/lib/Driver/Tools.cpp
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Tools.cpp?rev=284140&r1=284139&r2=284140&view=diff
==
--- cfe/trunk/lib/Driver/Tools.cpp (original)
+++ cfe/trunk/lib/Driver/Tools.cpp Thu Oct 13 13:05:53 2016
@@ -2002,6 +2002,14 @@ static unsigned getLTOParallelism(const
   return Parallelism;
 }
 
+// CloudABI and WebAssembly use -ffunction-sections and -fdata-sections by
+// default.
+static bool isUseSeparateSections(const llvm::Triple &Triple) {
+  return Triple.getOS() == llvm::Triple::CloudABI ||
+ Triple.getArch() == llvm::Triple::wasm32 ||
+ Triple.getArch() == llvm::Triple::wasm64;
+}
+
 static void AddGoldPlugin(const ToolChain &ToolChain, const ArgList &Args,
   ArgStringList &CmdArgs, bool IsThinLTO,
   const Driver &D) {
@@ -2051,6 +2059,19 @@ static void AddGoldPlugin(const ToolChai
 else
   CmdArgs.push_back("-plugin-opt=-debugger-tune=gdb");
   }
+
+  bool UseSeparateSections =
+  isUseSeparateSections(ToolChain.getEffectiveTriple());
+
+  if (Args.hasFlag(options::OPT_ffunction_sections,
+   options::OPT_fno_function_sections, UseSeparateSections)) {
+CmdArgs.push_back("-plugin-opt=-function-sections");
+  }
+
+  if (Args.hasFlag(options::OPT_fdata_sections, options::OPT_fno_data_sections,
+   UseSeparateSections)) {
+CmdArgs.push_back("-plugin-opt=-data-sections");
+  }
 }
 
 /// This is a helper function for validating the optional refinement step
@@ -4763,11 +4784,7 @@ void Clang::ConstructJob(Compilation &C,
 CmdArgs.push_back("-generate-type-units");
   }
 
-  // CloudABI and WebAssembly use -ffunction-sections and -fdata-sections by
-  // default.
-  bool UseSeparateSections = Triple.getOS() == llvm::Triple::CloudABI ||
- Triple.getArch() == llvm::Triple::wasm32 ||
- Triple.getArch() == llvm::Triple::wasm64;
+  bool UseSeparateSections = isUseSeparateSections(Triple);
 
   if (Args.hasFlag(options::OPT_ffunction_sections,
options::OPT_fno_function_sections, UseSeparateSections)) {

Added: cfe/trunk/test/Driver/gold-lto-sections.c
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/gold-lto-sections.c?rev=284140&view=auto
==
--- cfe/trunk/test/Driver/gold-lto-sections.c (added)
+++ cfe/trunk/test/Driver/gold-lto-sections.c Thu Oct 13 13:05:53 2016
@@ -0,0 +1,8 @@
+// RUN: touch %t.o
+//
+// RUN: %clang -target x86_64-unknown-linux -### %t.o -flto 2>&1 \
+// RUN: -Wl,-plugin-opt=foo -O3 \
+// RUN: -ffunction-sections -fdata-sections \
+// RUN: | FileCheck %s
+// CHECK: "-plugin-opt=-function-sections"
+// CHECK: "-plugin-opt=-data-sections"


___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D24644: Pass -ffunction-sections/-fdata-sections along to gold-plugin

[Teresa Johnson via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rL284140: Pass -ffunction-sections/-fdata-sections along to 
gold-plugin (authored by tejohnson).

Changed prior to commit:
  https://reviews.llvm.org/D24644?vs=71591&id=74555#toc

Repository:
  rL LLVM

https://reviews.llvm.org/D24644

Files:
  cfe/trunk/lib/Driver/Tools.cpp
  cfe/trunk/test/Driver/gold-lto-sections.c


Index: cfe/trunk/lib/Driver/Tools.cpp
===
--- cfe/trunk/lib/Driver/Tools.cpp
+++ cfe/trunk/lib/Driver/Tools.cpp
@@ -2002,6 +2002,14 @@
   return Parallelism;
 }
 
+// CloudABI and WebAssembly use -ffunction-sections and -fdata-sections by
+// default.
+static bool isUseSeparateSections(const llvm::Triple &Triple) {
+  return Triple.getOS() == llvm::Triple::CloudABI ||
+ Triple.getArch() == llvm::Triple::wasm32 ||
+ Triple.getArch() == llvm::Triple::wasm64;
+}
+
 static void AddGoldPlugin(const ToolChain &ToolChain, const ArgList &Args,
   ArgStringList &CmdArgs, bool IsThinLTO,
   const Driver &D) {
@@ -2051,6 +2059,19 @@
 else
   CmdArgs.push_back("-plugin-opt=-debugger-tune=gdb");
   }
+
+  bool UseSeparateSections =
+  isUseSeparateSections(ToolChain.getEffectiveTriple());
+
+  if (Args.hasFlag(options::OPT_ffunction_sections,
+   options::OPT_fno_function_sections, UseSeparateSections)) {
+CmdArgs.push_back("-plugin-opt=-function-sections");
+  }
+
+  if (Args.hasFlag(options::OPT_fdata_sections, options::OPT_fno_data_sections,
+   UseSeparateSections)) {
+CmdArgs.push_back("-plugin-opt=-data-sections");
+  }
 }
 
 /// This is a helper function for validating the optional refinement step
@@ -4763,11 +4784,7 @@
 CmdArgs.push_back("-generate-type-units");
   }
 
-  // CloudABI and WebAssembly use -ffunction-sections and -fdata-sections by
-  // default.
-  bool UseSeparateSections = Triple.getOS() == llvm::Triple::CloudABI ||
- Triple.getArch() == llvm::Triple::wasm32 ||
- Triple.getArch() == llvm::Triple::wasm64;
+  bool UseSeparateSections = isUseSeparateSections(Triple);
 
   if (Args.hasFlag(options::OPT_ffunction_sections,
options::OPT_fno_function_sections, UseSeparateSections)) {
Index: cfe/trunk/test/Driver/gold-lto-sections.c
===
--- cfe/trunk/test/Driver/gold-lto-sections.c
+++ cfe/trunk/test/Driver/gold-lto-sections.c
@@ -0,0 +1,8 @@
+// RUN: touch %t.o
+//
+// RUN: %clang -target x86_64-unknown-linux -### %t.o -flto 2>&1 \
+// RUN: -Wl,-plugin-opt=foo -O3 \
+// RUN: -ffunction-sections -fdata-sections \
+// RUN: | FileCheck %s
+// CHECK: "-plugin-opt=-function-sections"
+// CHECK: "-plugin-opt=-data-sections"


Index: cfe/trunk/lib/Driver/Tools.cpp
===
--- cfe/trunk/lib/Driver/Tools.cpp
+++ cfe/trunk/lib/Driver/Tools.cpp
@@ -2002,6 +2002,14 @@
   return Parallelism;
 }
 
+// CloudABI and WebAssembly use -ffunction-sections and -fdata-sections by
+// default.
+static bool isUseSeparateSections(const llvm::Triple &Triple) {
+  return Triple.getOS() == llvm::Triple::CloudABI ||
+ Triple.getArch() == llvm::Triple::wasm32 ||
+ Triple.getArch() == llvm::Triple::wasm64;
+}
+
 static void AddGoldPlugin(const ToolChain &ToolChain, const ArgList &Args,
   ArgStringList &CmdArgs, bool IsThinLTO,
   const Driver &D) {
@@ -2051,6 +2059,19 @@
 else
   CmdArgs.push_back("-plugin-opt=-debugger-tune=gdb");
   }
+
+  bool UseSeparateSections =
+  isUseSeparateSections(ToolChain.getEffectiveTriple());
+
+  if (Args.hasFlag(options::OPT_ffunction_sections,
+   options::OPT_fno_function_sections, UseSeparateSections)) {
+CmdArgs.push_back("-plugin-opt=-function-sections");
+  }
+
+  if (Args.hasFlag(options::OPT_fdata_sections, options::OPT_fno_data_sections,
+   UseSeparateSections)) {
+CmdArgs.push_back("-plugin-opt=-data-sections");
+  }
 }
 
 /// This is a helper function for validating the optional refinement step
@@ -4763,11 +4784,7 @@
 CmdArgs.push_back("-generate-type-units");
   }
 
-  // CloudABI and WebAssembly use -ffunction-sections and -fdata-sections by
-  // default.
-  bool UseSeparateSections = Triple.getOS() == llvm::Triple::CloudABI ||
- Triple.getArch() == llvm::Triple::wasm32 ||
- Triple.getArch() == llvm::Triple::wasm64;
+  bool UseSeparateSections = isUseSeparateSections(Triple);
 
   if (Args.hasFlag(options::OPT_ffunction_sections,
options::OPT_fno_function_sections, UseSeparateSections)) {
Index: cfe/trunk/test/Driver/gold-lto-sections.c
===

Re: r284137 - [ThinLTO] Update doc to include lld (now supported).

[Teresa Johnson via cfe-commits]

On Mon, Oct 17, 2016 at 5:13 AM, Rafael Espíndola <
rafael.espind...@gmail.com> wrote:

> On 16 October 2016 at 22:13, Davide Italiano  wrote:
> > On Sun, Oct 16, 2016 at 6:43 PM, Sean Silva 
> wrote:
> >> Nice to see this land!
> >>
> >> One nit:
> >> Currently, doesn't LLD/ELF ignore -plugin-opt? That will mean that if a
> user
> >> uses the "gold syntax" then LLD will silently ignore it, which isn't
> good.
> >> At the very least, can we issue an error if we see `-plugin-opt jobs=N`
> and
> >> suggest the LLD spelling?
> >>
> >> Or maybe just accept the gold syntax? Our current handling of `-plugin`
> and
> >> `-plugin-opt` is intended to make LLD transparently Do The Right Thing
> when
> >> LLD is invoked as if it were gold, so clearly gold compatibility is
> >> important enough for that. This suggests it is important enough to be
> >> compatible from a ThinLTO perspective too.
> >>
> >
> > I agree with what you're suggesting.  My initial vote would be for
> > error'ing out on anything we can't understand that's passed via
> > `-plugin-opt` and see what breaks (and add incremental support for
> > every feature needed).
> > Teresa, Rafael, any opinions about it?
>
> I agree. Having clang known if it is using gold or lld is probably not
> worth it.
>

Sure, that seems reasonable to me as well. For example, there is now a
clang option-flto-jobs=N that hooks up to the gold plugin option jobs=N
option, and you would get that automatically without having to wire it in.
Erroring on any unrecognized options would be good too.

Teresa


> Cheers,
> Rafael
>



-- 
Teresa Johnson |  Software Engineer |  tejohn...@google.com |  408-460-2413
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: r284137 - [ThinLTO] Update doc to include lld (now supported).

[Teresa Johnson via cfe-commits]

On Mon, Oct 17, 2016 at 6:15 AM, Teresa Johnson 
wrote:

>
>
> On Mon, Oct 17, 2016 at 5:13 AM, Rafael Espíndola <
> rafael.espind...@gmail.com> wrote:
>
>> On 16 October 2016 at 22:13, Davide Italiano  wrote:
>> > On Sun, Oct 16, 2016 at 6:43 PM, Sean Silva 
>> wrote:
>> >> Nice to see this land!
>> >>
>> >> One nit:
>> >> Currently, doesn't LLD/ELF ignore -plugin-opt? That will mean that if
>> a user
>> >> uses the "gold syntax" then LLD will silently ignore it, which isn't
>> good.
>> >> At the very least, can we issue an error if we see `-plugin-opt
>> jobs=N` and
>> >> suggest the LLD spelling?
>> >>
>> >> Or maybe just accept the gold syntax? Our current handling of
>> `-plugin` and
>> >> `-plugin-opt` is intended to make LLD transparently Do The Right Thing
>> when
>> >> LLD is invoked as if it were gold, so clearly gold compatibility is
>> >> important enough for that. This suggests it is important enough to be
>> >> compatible from a ThinLTO perspective too.
>> >>
>> >
>> > I agree with what you're suggesting.  My initial vote would be for
>> > error'ing out on anything we can't understand that's passed via
>> > `-plugin-opt` and see what breaks (and add incremental support for
>> > every feature needed).
>> > Teresa, Rafael, any opinions about it?
>>
>> I agree. Having clang known if it is using gold or lld is probably not
>> worth it.
>>
>
> Sure, that seems reasonable to me as well. For example, there is now a
> clang option-flto-jobs=N that hooks up to the gold plugin option jobs=N
> option, and you would get that automatically without having to wire it in.
> Erroring on any unrecognized options would be good too.
>

(In fact, since -flto-jobs=N is already hooked up to ld64's -mllvm -threads
option, once lld accepts the -plugin-opt version I can update this
documentation just to list the single clang option.

Teresa


> Teresa
>
>
>> Cheers,
>> Rafael
>>
>
>
>
> --
> Teresa Johnson |  Software Engineer |  tejohn...@google.com |
> 408-460-2413
>



-- 
Teresa Johnson |  Software Engineer |  tejohn...@google.com |  408-460-2413
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Support VFE in thinLTO (PR #69735)

[Teresa Johnson via cfe-commits]

https://github.com/teresajohnson edited 
https://github.com/llvm/llvm-project/pull/69735
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Support VFE in thinLTO (PR #69735)

[Teresa Johnson via cfe-commits]

https://github.com/teresajohnson commented:

Thanks for the patch! I didn't go through in too much detail yet, but have some 
mostly higher level comments and suggestions.

https://github.com/llvm/llvm-project/pull/69735
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Support VFE in thinLTO (PR #69735)

[Teresa Johnson via cfe-commits]

@@ -1362,6 +1362,8 @@ class ModuleSummaryIndex {
   // Temporary map while building StackIds list. Clear when index is completely
   // built via releaseTemporaryMemory.
   std::map StackIdToIndex;
+  std::set FuncsWithNonVtableRef;
+  std::set VFuncsToBeRemoved; // no need to serialize

teresajohnson wrote:

It would need to be serialized for distributed thinlto. A better thing would be 
to describe which of these are produced by the per-module analysis and used by 
the thin link, vs produced by the thin link and consumed by the ThinLTO 
backends. However, I suggest making these flags as noted above (but those 
should be documented with this information too)

https://github.com/llvm/llvm-project/pull/69735
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] Support VFE in thinLTO (PR #69735)

[Teresa Johnson via cfe-commits]

@@ -34,12 +40,223 @@ static cl::opt
 ClEnableVFE("enable-vfe", cl::Hidden, cl::init(true),
 cl::desc("Enable virtual function elimination"));
 
+static cl::opt ClReadSummary(
+"globaldce-read-summary",
+cl::desc("Read summary from given bitcode before running pass"),
+cl::Hidden);
+
 STATISTIC(NumAliases  , "Number of global aliases removed");
 STATISTIC(NumFunctions, "Number of functions removed");
 STATISTIC(NumIFuncs,"Number of indirect functions removed");
 STATISTIC(NumVariables, "Number of global variables removed");
 STATISTIC(NumVFuncs,"Number of virtual functions removed");
 
+namespace llvm {
+
+// Returning a representative summary for the vtable, also set isSafe.
+static const GlobalVarSummary *
+getVTableFuncsForTId(const TypeIdOffsetVtableInfo &P, bool &isSafe) {
+  // Find a representative copy of the vtable initializer.
+  const GlobalVarSummary *VS = nullptr;
+  bool LocalFound = false;
+  for (auto &S : P.VTableVI.getSummaryList()) {
+if (GlobalValue::isLocalLinkage(S->linkage())) {
+  if (LocalFound) {
+isSafe = false;
+return nullptr;
+  }
+  LocalFound = true;
+}
+auto *CurVS = cast(S->getBaseObject());
+// Ignore if vTableFuncs is empty and vtable is available_externally.
+if (!CurVS->vTableFuncs().empty() ||
+!GlobalValue::isAvailableExternallyLinkage(S->linkage())) {
+  VS = CurVS;
+  if (VS->getVCallVisibility() == GlobalObject::VCallVisibilityPublic) {
+isSafe = false;
+return VS;
+  }
+}
+  }
+
+  if (!VS) {
+isSafe = false;
+return nullptr;
+  }
+  if (!VS->isLive()) {
+isSafe = true;
+return nullptr;
+  }
+  isSafe = true;
+  return VS;
+}
+
+static void collectSafeVTables(
+ModuleSummaryIndex &Summary,
+DenseMap> &NameByGUID,
+std::map> &VFESafeVTablesAndFns) {
+  // Update VFESafeVTablesAndFns with information from summary.
+  for (auto &P : Summary.typeIdCompatibleVtableMap()) {
+NameByGUID[GlobalValue::getGUID(P.first)].push_back(P.first);
+LLVM_DEBUG(dbgs() << "TId " << GlobalValue::getGUID(P.first) << " "
+  << P.first << "\n");
+  }
+  llvm::errs() << "VFEThinLTO number of TIds: " << NameByGUID.size() << "\n";
+
+  // VFESafeVTablesAndFns: map from VI for vTable to VI for vfunc
+  std::map> vFuncSet;
+  unsigned numSafeVFuncs = 0;
+  // Collect stats for VTables (safe, public-visibility, other).
+  std::set vTablePublicVis;
+  std::set vTableOther;
+  for (auto &TidSummary : Summary.typeIdCompatibleVtableMap()) {
+for (const TypeIdOffsetVtableInfo &P : TidSummary.second) {
+  LLVM_DEBUG(dbgs() << "TId-vTable " << TidSummary.first << " "
+<< P.VTableVI.name() << " " << P.AddressPointOffset
+<< "\n");
+  bool isSafe = false;
+  const GlobalVarSummary *VS = getVTableFuncsForTId(P, isSafe);
+  if (!isSafe && VS)
+vTablePublicVis.insert(P.VTableVI);
+  if ((isSafe && !VS) || (!isSafe && !VS))
+vTableOther.insert(P.VTableVI);
+  if (!isSafe || !VS) {
+continue;
+  }
+
+  // Go through VS->vTableFuncs
+  for (auto VTP : VS->vTableFuncs()) {
+if (vFuncSet.find(P.VTableVI) == vFuncSet.end() ||
+!vFuncSet[P.VTableVI].count(VTP.FuncVI.getGUID())) {
+  VFESafeVTablesAndFns[P.VTableVI].push_back(VTP);
+  LLVM_DEBUG(dbgs()
+ << "vTable " << P.VTableVI.name() << " "
+ << VTP.FuncVI.name() << " " << VTP.VTableOffset << "\n");
+  ++numSafeVFuncs;
+}
+vFuncSet[P.VTableVI].insert(VTP.FuncVI.getGUID());
+  }
+}
+  }
+  llvm::errs() << "VFEThinLTO number of vTables: " << vFuncSet.size() << " "

teresajohnson wrote:

Should these be optimization remarks or debug messages?

https://github.com/llvm/llvm-project/pull/69735
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Support VFE in thinLTO (PR #69735)

[Teresa Johnson via cfe-commits]

@@ -34,12 +40,223 @@ static cl::opt
 ClEnableVFE("enable-vfe", cl::Hidden, cl::init(true),
 cl::desc("Enable virtual function elimination"));
 
+static cl::opt ClReadSummary(
+"globaldce-read-summary",
+cl::desc("Read summary from given bitcode before running pass"),
+cl::Hidden);
+
 STATISTIC(NumAliases  , "Number of global aliases removed");
 STATISTIC(NumFunctions, "Number of functions removed");
 STATISTIC(NumIFuncs,"Number of indirect functions removed");
 STATISTIC(NumVariables, "Number of global variables removed");
 STATISTIC(NumVFuncs,"Number of virtual functions removed");
 
+namespace llvm {
+
+// Returning a representative summary for the vtable, also set isSafe.
+static const GlobalVarSummary *
+getVTableFuncsForTId(const TypeIdOffsetVtableInfo &P, bool &isSafe) {
+  // Find a representative copy of the vtable initializer.
+  const GlobalVarSummary *VS = nullptr;
+  bool LocalFound = false;
+  for (auto &S : P.VTableVI.getSummaryList()) {
+if (GlobalValue::isLocalLinkage(S->linkage())) {
+  if (LocalFound) {
+isSafe = false;
+return nullptr;
+  }
+  LocalFound = true;
+}
+auto *CurVS = cast(S->getBaseObject());
+// Ignore if vTableFuncs is empty and vtable is available_externally.
+if (!CurVS->vTableFuncs().empty() ||
+!GlobalValue::isAvailableExternallyLinkage(S->linkage())) {
+  VS = CurVS;
+  if (VS->getVCallVisibility() == GlobalObject::VCallVisibilityPublic) {
+isSafe = false;
+return VS;
+  }
+}
+  }
+
+  if (!VS) {
+isSafe = false;
+return nullptr;
+  }
+  if (!VS->isLive()) {
+isSafe = true;
+return nullptr;
+  }
+  isSafe = true;
+  return VS;
+}
+
+static void collectSafeVTables(
+ModuleSummaryIndex &Summary,
+DenseMap> &NameByGUID,
+std::map> &VFESafeVTablesAndFns) {
+  // Update VFESafeVTablesAndFns with information from summary.
+  for (auto &P : Summary.typeIdCompatibleVtableMap()) {
+NameByGUID[GlobalValue::getGUID(P.first)].push_back(P.first);
+LLVM_DEBUG(dbgs() << "TId " << GlobalValue::getGUID(P.first) << " "
+  << P.first << "\n");
+  }
+  llvm::errs() << "VFEThinLTO number of TIds: " << NameByGUID.size() << "\n";
+
+  // VFESafeVTablesAndFns: map from VI for vTable to VI for vfunc
+  std::map> vFuncSet;
+  unsigned numSafeVFuncs = 0;
+  // Collect stats for VTables (safe, public-visibility, other).
+  std::set vTablePublicVis;
+  std::set vTableOther;
+  for (auto &TidSummary : Summary.typeIdCompatibleVtableMap()) {
+for (const TypeIdOffsetVtableInfo &P : TidSummary.second) {
+  LLVM_DEBUG(dbgs() << "TId-vTable " << TidSummary.first << " "
+<< P.VTableVI.name() << " " << P.AddressPointOffset
+<< "\n");
+  bool isSafe = false;
+  const GlobalVarSummary *VS = getVTableFuncsForTId(P, isSafe);
+  if (!isSafe && VS)
+vTablePublicVis.insert(P.VTableVI);
+  if ((isSafe && !VS) || (!isSafe && !VS))
+vTableOther.insert(P.VTableVI);
+  if (!isSafe || !VS) {
+continue;
+  }
+
+  // Go through VS->vTableFuncs
+  for (auto VTP : VS->vTableFuncs()) {
+if (vFuncSet.find(P.VTableVI) == vFuncSet.end() ||
+!vFuncSet[P.VTableVI].count(VTP.FuncVI.getGUID())) {
+  VFESafeVTablesAndFns[P.VTableVI].push_back(VTP);
+  LLVM_DEBUG(dbgs()
+ << "vTable " << P.VTableVI.name() << " "
+ << VTP.FuncVI.name() << " " << VTP.VTableOffset << "\n");
+  ++numSafeVFuncs;
+}
+vFuncSet[P.VTableVI].insert(VTP.FuncVI.getGUID());
+  }
+}
+  }
+  llvm::errs() << "VFEThinLTO number of vTables: " << vFuncSet.size() << " "
+   << vTablePublicVis.size() << " " << vTableOther.size() << "\n";
+  llvm::errs() << "VFEThinLTO numSafeVFuncs: " << numSafeVFuncs << "\n";
+}
+
+static void checkVTableLoadsIndex(
+ModuleSummaryIndex &Summary,
+DenseMap> &NameByGUID,
+std::map> &VFESafeVTablesAndFns,
+std::map> &VFuncsAndCallers) {
+  // Go through Function summarys for intrinsics, also funcHasNonVTableRef to
+  // erase entries from VFESafeVTableAndFns.
+  for (auto &PI : Summary) {
+for (auto &S : PI.second.SummaryList) {
+  auto *FS = dyn_cast(S.get());
+  if (!FS)
+continue;
+  // We should ignore Tid if there is a type.checked.load with Offset not
+  // ConstantInt. Currently ModuleSummaryAnalysis will update TypeTests.
+  for (GlobalValue::GUID G : FS->type_tests()) {
+if (NameByGUID.find(G) == NameByGUID.end())
+  continue;
+auto TidSummary =
+Summary.getTypeIdCompatibleVtableSummary(NameByGUID[G][0]);
+for (const TypeIdOffsetVtableInfo &P : *TidSummary) {
+  LLVM_DEBUG(dbgs() << "unsafe-vtable due to type_tests: "
+<< P.VTableVI.name() << "\n");
+  VFESafeVTablesAnd

[llvm] [clang] Support VFE in thinLTO (PR #69735)

[Teresa Johnson via cfe-commits]

@@ -775,6 +783,58 @@ static void computeVariableSummary(ModuleSummaryIndex 
&Index,
   Index.addGlobalValueSummary(V, std::move(GVarSummary));
 }
 
+static void ComputeDependencies(

teresajohnson wrote:

nit: function should be lowerCamelCase

https://github.com/llvm/llvm-project/pull/69735
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Support VFE in thinLTO (PR #69735)

[Teresa Johnson via cfe-commits]

@@ -34,12 +40,223 @@ static cl::opt
 ClEnableVFE("enable-vfe", cl::Hidden, cl::init(true),
 cl::desc("Enable virtual function elimination"));
 
+static cl::opt ClReadSummary(
+"globaldce-read-summary",
+cl::desc("Read summary from given bitcode before running pass"),
+cl::Hidden);
+
 STATISTIC(NumAliases  , "Number of global aliases removed");
 STATISTIC(NumFunctions, "Number of functions removed");
 STATISTIC(NumIFuncs,"Number of indirect functions removed");
 STATISTIC(NumVariables, "Number of global variables removed");
 STATISTIC(NumVFuncs,"Number of virtual functions removed");
 
+namespace llvm {
+
+// Returning a representative summary for the vtable, also set isSafe.
+static const GlobalVarSummary *
+getVTableFuncsForTId(const TypeIdOffsetVtableInfo &P, bool &isSafe) {
+  // Find a representative copy of the vtable initializer.
+  const GlobalVarSummary *VS = nullptr;
+  bool LocalFound = false;
+  for (auto &S : P.VTableVI.getSummaryList()) {
+if (GlobalValue::isLocalLinkage(S->linkage())) {
+  if (LocalFound) {
+isSafe = false;
+return nullptr;
+  }
+  LocalFound = true;
+}
+auto *CurVS = cast(S->getBaseObject());
+// Ignore if vTableFuncs is empty and vtable is available_externally.
+if (!CurVS->vTableFuncs().empty() ||
+!GlobalValue::isAvailableExternallyLinkage(S->linkage())) {
+  VS = CurVS;
+  if (VS->getVCallVisibility() == GlobalObject::VCallVisibilityPublic) {
+isSafe = false;
+return VS;
+  }
+}
+  }
+
+  if (!VS) {
+isSafe = false;
+return nullptr;
+  }
+  if (!VS->isLive()) {
+isSafe = true;
+return nullptr;
+  }
+  isSafe = true;
+  return VS;
+}
+
+static void collectSafeVTables(
+ModuleSummaryIndex &Summary,
+DenseMap> &NameByGUID,
+std::map> &VFESafeVTablesAndFns) {
+  // Update VFESafeVTablesAndFns with information from summary.
+  for (auto &P : Summary.typeIdCompatibleVtableMap()) {
+NameByGUID[GlobalValue::getGUID(P.first)].push_back(P.first);
+LLVM_DEBUG(dbgs() << "TId " << GlobalValue::getGUID(P.first) << " "
+  << P.first << "\n");
+  }
+  llvm::errs() << "VFEThinLTO number of TIds: " << NameByGUID.size() << "\n";
+
+  // VFESafeVTablesAndFns: map from VI for vTable to VI for vfunc

teresajohnson wrote:

This comment seems to be in the wrong place, can you add a description of the 
below data structure (and describe VFESafeVTablesAndFns somewhere too, maybe 
where it is defined?)

https://github.com/llvm/llvm-project/pull/69735
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] Support VFE in thinLTO (PR #69735)

[Teresa Johnson via cfe-commits]

@@ -34,12 +40,223 @@ static cl::opt
 ClEnableVFE("enable-vfe", cl::Hidden, cl::init(true),
 cl::desc("Enable virtual function elimination"));
 
+static cl::opt ClReadSummary(
+"globaldce-read-summary",
+cl::desc("Read summary from given bitcode before running pass"),
+cl::Hidden);
+
 STATISTIC(NumAliases  , "Number of global aliases removed");
 STATISTIC(NumFunctions, "Number of functions removed");
 STATISTIC(NumIFuncs,"Number of indirect functions removed");
 STATISTIC(NumVariables, "Number of global variables removed");
 STATISTIC(NumVFuncs,"Number of virtual functions removed");
 
+namespace llvm {
+
+// Returning a representative summary for the vtable, also set isSafe.
+static const GlobalVarSummary *
+getVTableFuncsForTId(const TypeIdOffsetVtableInfo &P, bool &isSafe) {
+  // Find a representative copy of the vtable initializer.
+  const GlobalVarSummary *VS = nullptr;
+  bool LocalFound = false;
+  for (auto &S : P.VTableVI.getSummaryList()) {
+if (GlobalValue::isLocalLinkage(S->linkage())) {
+  if (LocalFound) {
+isSafe = false;
+return nullptr;
+  }
+  LocalFound = true;
+}
+auto *CurVS = cast(S->getBaseObject());
+// Ignore if vTableFuncs is empty and vtable is available_externally.
+if (!CurVS->vTableFuncs().empty() ||
+!GlobalValue::isAvailableExternallyLinkage(S->linkage())) {
+  VS = CurVS;
+  if (VS->getVCallVisibility() == GlobalObject::VCallVisibilityPublic) {
+isSafe = false;
+return VS;
+  }
+}
+  }
+
+  if (!VS) {
+isSafe = false;
+return nullptr;
+  }
+  if (!VS->isLive()) {
+isSafe = true;
+return nullptr;
+  }
+  isSafe = true;
+  return VS;
+}
+
+static void collectSafeVTables(
+ModuleSummaryIndex &Summary,
+DenseMap> &NameByGUID,
+std::map> &VFESafeVTablesAndFns) {
+  // Update VFESafeVTablesAndFns with information from summary.
+  for (auto &P : Summary.typeIdCompatibleVtableMap()) {
+NameByGUID[GlobalValue::getGUID(P.first)].push_back(P.first);
+LLVM_DEBUG(dbgs() << "TId " << GlobalValue::getGUID(P.first) << " "
+  << P.first << "\n");
+  }
+  llvm::errs() << "VFEThinLTO number of TIds: " << NameByGUID.size() << "\n";
+
+  // VFESafeVTablesAndFns: map from VI for vTable to VI for vfunc
+  std::map> vFuncSet;
+  unsigned numSafeVFuncs = 0;
+  // Collect stats for VTables (safe, public-visibility, other).
+  std::set vTablePublicVis;
+  std::set vTableOther;
+  for (auto &TidSummary : Summary.typeIdCompatibleVtableMap()) {
+for (const TypeIdOffsetVtableInfo &P : TidSummary.second) {
+  LLVM_DEBUG(dbgs() << "TId-vTable " << TidSummary.first << " "
+<< P.VTableVI.name() << " " << P.AddressPointOffset
+<< "\n");
+  bool isSafe = false;
+  const GlobalVarSummary *VS = getVTableFuncsForTId(P, isSafe);
+  if (!isSafe && VS)
+vTablePublicVis.insert(P.VTableVI);
+  if ((isSafe && !VS) || (!isSafe && !VS))
+vTableOther.insert(P.VTableVI);
+  if (!isSafe || !VS) {
+continue;
+  }
+
+  // Go through VS->vTableFuncs

teresajohnson wrote:

Can you expand on the comment - what is this code doing and why?

https://github.com/llvm/llvm-project/pull/69735
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Support VFE in thinLTO (PR #69735)

[Teresa Johnson via cfe-commits]

@@ -1362,6 +1362,8 @@ class ModuleSummaryIndex {
   // Temporary map while building StackIds list. Clear when index is completely
   // built via releaseTemporaryMemory.
   std::map StackIdToIndex;
+  std::set FuncsWithNonVtableRef;

teresajohnson wrote:

Instead of sets of GUIDs, it would probably be more efficient to represent, and 
to access, to just add new function flags (FFlags in the FunctionSummary).

https://github.com/llvm/llvm-project/pull/69735
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] Support VFE in thinLTO (PR #69735)

[Teresa Johnson via cfe-commits]

@@ -34,12 +40,223 @@ static cl::opt
 ClEnableVFE("enable-vfe", cl::Hidden, cl::init(true),
 cl::desc("Enable virtual function elimination"));
 
+static cl::opt ClReadSummary(
+"globaldce-read-summary",
+cl::desc("Read summary from given bitcode before running pass"),
+cl::Hidden);
+
 STATISTIC(NumAliases  , "Number of global aliases removed");
 STATISTIC(NumFunctions, "Number of functions removed");
 STATISTIC(NumIFuncs,"Number of indirect functions removed");
 STATISTIC(NumVariables, "Number of global variables removed");
 STATISTIC(NumVFuncs,"Number of virtual functions removed");
 
+namespace llvm {
+
+// Returning a representative summary for the vtable, also set isSafe.
+static const GlobalVarSummary *
+getVTableFuncsForTId(const TypeIdOffsetVtableInfo &P, bool &isSafe) {
+  // Find a representative copy of the vtable initializer.
+  const GlobalVarSummary *VS = nullptr;
+  bool LocalFound = false;
+  for (auto &S : P.VTableVI.getSummaryList()) {
+if (GlobalValue::isLocalLinkage(S->linkage())) {
+  if (LocalFound) {
+isSafe = false;
+return nullptr;
+  }
+  LocalFound = true;
+}
+auto *CurVS = cast(S->getBaseObject());
+// Ignore if vTableFuncs is empty and vtable is available_externally.
+if (!CurVS->vTableFuncs().empty() ||
+!GlobalValue::isAvailableExternallyLinkage(S->linkage())) {
+  VS = CurVS;
+  if (VS->getVCallVisibility() == GlobalObject::VCallVisibilityPublic) {
+isSafe = false;
+return VS;
+  }
+}
+  }
+
+  if (!VS) {
+isSafe = false;
+return nullptr;
+  }
+  if (!VS->isLive()) {
+isSafe = true;
+return nullptr;
+  }
+  isSafe = true;
+  return VS;
+}
+
+static void collectSafeVTables(
+ModuleSummaryIndex &Summary,
+DenseMap> &NameByGUID,
+std::map> &VFESafeVTablesAndFns) {
+  // Update VFESafeVTablesAndFns with information from summary.
+  for (auto &P : Summary.typeIdCompatibleVtableMap()) {
+NameByGUID[GlobalValue::getGUID(P.first)].push_back(P.first);
+LLVM_DEBUG(dbgs() << "TId " << GlobalValue::getGUID(P.first) << " "
+  << P.first << "\n");
+  }
+  llvm::errs() << "VFEThinLTO number of TIds: " << NameByGUID.size() << "\n";
+
+  // VFESafeVTablesAndFns: map from VI for vTable to VI for vfunc
+  std::map> vFuncSet;
+  unsigned numSafeVFuncs = 0;
+  // Collect stats for VTables (safe, public-visibility, other).
+  std::set vTablePublicVis;
+  std::set vTableOther;
+  for (auto &TidSummary : Summary.typeIdCompatibleVtableMap()) {
+for (const TypeIdOffsetVtableInfo &P : TidSummary.second) {
+  LLVM_DEBUG(dbgs() << "TId-vTable " << TidSummary.first << " "
+<< P.VTableVI.name() << " " << P.AddressPointOffset
+<< "\n");
+  bool isSafe = false;
+  const GlobalVarSummary *VS = getVTableFuncsForTId(P, isSafe);
+  if (!isSafe && VS)
+vTablePublicVis.insert(P.VTableVI);
+  if ((isSafe && !VS) || (!isSafe && !VS))
+vTableOther.insert(P.VTableVI);
+  if (!isSafe || !VS) {
+continue;
+  }
+
+  // Go through VS->vTableFuncs
+  for (auto VTP : VS->vTableFuncs()) {
+if (vFuncSet.find(P.VTableVI) == vFuncSet.end() ||
+!vFuncSet[P.VTableVI].count(VTP.FuncVI.getGUID())) {
+  VFESafeVTablesAndFns[P.VTableVI].push_back(VTP);
+  LLVM_DEBUG(dbgs()
+ << "vTable " << P.VTableVI.name() << " "
+ << VTP.FuncVI.name() << " " << VTP.VTableOffset << "\n");
+  ++numSafeVFuncs;
+}
+vFuncSet[P.VTableVI].insert(VTP.FuncVI.getGUID());
+  }
+}
+  }
+  llvm::errs() << "VFEThinLTO number of vTables: " << vFuncSet.size() << " "
+   << vTablePublicVis.size() << " " << vTableOther.size() << "\n";
+  llvm::errs() << "VFEThinLTO numSafeVFuncs: " << numSafeVFuncs << "\n";
+}
+
+static void checkVTableLoadsIndex(
+ModuleSummaryIndex &Summary,
+DenseMap> &NameByGUID,
+std::map> &VFESafeVTablesAndFns,
+std::map> &VFuncsAndCallers) {
+  // Go through Function summarys for intrinsics, also funcHasNonVTableRef to
+  // erase entries from VFESafeVTableAndFns.

teresajohnson wrote:

why and when are entries being erased? More qualitative comments about what 
this algorithm is doing would be helpful.

https://github.com/llvm/llvm-project/pull/69735
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] Support VFE in thinLTO (PR #69735)

[Teresa Johnson via cfe-commits]

@@ -338,12 +574,33 @@ PreservedAnalyses GlobalDCEPass::run(Module &M, 
ModuleAnalysisManager &MAM) {
 
   // The second pass drops the bodies of functions which are dead...
   std::vector DeadFunctions;
-  for (Function &F : M)
+  std::set DeadFunctionsSet;
+  auto funcRemovedInIndex = [&](Function *F) -> bool {
+if (!ImportSummary)
+  return false;
+auto &vfuncsRemoved = ImportSummary->funcsToBeRemoved();
+// If function is internalized, its current GUID will be different
+// from the GUID in funcsToBeRemoved. Treat it as a global and search
+// again.
+if (vfuncsRemoved.count(F->getGUID()))
+  return true;
+if (vfuncsRemoved.count(GlobalValue::getGUID(F->getName(

teresajohnson wrote:

These lookups are a little dangerous as I think there are cases where we might 
rename and it wouldn't find the right function. For internalization we tag the 
function with an attribute before any thinlto renaming, linkage changes, or 
other optimization passes. See 
https://github.com/llvm/llvm-project/blob/0936a718492d248a726b8e8d4529e4aa194bc8e9/llvm/lib/Transforms/Utils/FunctionImportUtils.cpp#L271.

https://github.com/llvm/llvm-project/pull/69735
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

[Teresa Johnson via cfe-commits]

@@ -300,12 +316,8 @@ getIRPGONameForGlobalObject(const GlobalObject &GO,
 GlobalValue::LinkageTypes Linkage,
 StringRef FileName) {
   SmallString<64> Name;
-  if (llvm::GlobalValue::isLocalLinkage(Linkage)) {
-Name.append(FileName.empty() ? "" : FileName);
-Name.append(";");
-  }
   Mangler().getNameWithPrefix(Name, &GO, /*CannotUsePrivateLabel=*/true);

teresajohnson wrote:

> It wasn't broken in general, but it's needed to get -order_file working 
> correctly.

Unfortunately this change broke aspects of ThinLTO ICP, however. Is it possible 
to change the handling of -order_file in the linker to modify the symbol names 
appropriately?

> To avoid subtle issues when linkage-name is different from mangled names,,I'm 
> wondering if it warrants a change to use linkage-names (as opposed to mangled 
> name) in GlobalValue::getGlobalIdentifier in this PR.

If the -order_name handling cannot be fixed as I suggested above, then yes, we 
need some solution to ensure that the hashed symbol names are consistent across 
PGO and ThinLTO.

https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

[Teresa Johnson via cfe-commits]

teresajohnson wrote:

> Using the same added ICP test, profile matching on local-linkage 
> `_ZL7callee0v` using `clang++ -v -fuse-ld=lld -O2 
> -fprofile-use=thinlto_icall_prom.profdata ` , as 
> [this](https://gist.github.com/minglotus-6/11817ba645c6b12cd7116f41bfb1185e) 
> pgo-instr-use output shows.

Can you clarify what you are saying here - is it working or not working?

https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [clang] [compiler-rt] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

[Teresa Johnson via cfe-commits]

teresajohnson wrote:

> David says the itanium remapper file was only used once during gcc to llvm 
> transition, so not relevant here.

I believe it was actually for the libstdc++ to libc++ transition (see 
https://reviews.llvm.org/D51247 and https://reviews.llvm.org/D51240).

If it is broken we'll at least want to add a FIXME there. 

https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[compiler-rt] [clang-tools-extra] [clang] [llvm] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

[Teresa Johnson via cfe-commits]

@@ -1,39 +1,45 @@
-; Do setup work for all below tests: generate bitcode and combined index
-; RUN: opt -module-summary %s -o %t.bc
-; RUN: opt -module-summary %p/Inputs/thinlto_indirect_call_promotion.ll -o 
%t2.bc
+; The raw profiles and reduced IR inputs are generated from 
Inputs/update_icall_promotion_inputs.sh

teresajohnson wrote:

No because the point of this test is to catch regressions if the separator 
character (which is hardcoded in the proftext) changes again in the future.

https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[compiler-rt] [clang-tools-extra] [llvm] [clang] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

[Teresa Johnson via cfe-commits]

@@ -1,39 +1,45 @@
-; Do setup work for all below tests: generate bitcode and combined index
-; RUN: opt -module-summary %s -o %t.bc
-; RUN: opt -module-summary %p/Inputs/thinlto_indirect_call_promotion.ll -o 
%t2.bc
+; The raw profiles and reduced IR inputs are generated from 
Inputs/update_icall_promotion_inputs.sh

teresajohnson wrote:

The reason for having this in a single test this way is specifically to detect 
mismatches in the separator used in the PGO name and in the ThinLTO index. For 
this we need to convert from raw profile and feed it through ThinLTO+ICP. 

Having separate tests would not do this. I.e. with the change from ':' to ';', 
the compiler-rt test would presumably have failed and would be updated, but the 
test using the proftext input would not fail. The lack of testing these pieces 
together led to the prior separator rename not updating all the necessary 
compiler bits.

https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][fatlto] Don't set ThinLTO module flag with FatLTO (PR #75079)

[Teresa Johnson via cfe-commits]

teresajohnson wrote:

Added a comment to that issue, I think it would be good to understand why 
unified LTO is not expected in that case (for the assertion).

https://github.com/llvm/llvm-project/pull/75079
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[compiler-rt] [clang-tools-extra] [clang] [llvm] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

[Teresa Johnson via cfe-commits]

@@ -1,39 +0,0 @@
-; Do setup work for all below tests: generate bitcode and combined index

teresajohnson wrote:

Without use of the raw profile, this test would not have caught the regression. 
If we think the new compiler-rt test is enough to catch this case in the 
future, then perhaps we don't need this LLVM test at all (there should already 
be some ICP tests). But if the compiler-rt test is not enough, because some 
people don't run those, then this should stay and use a raw profile as input.

https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[compiler-rt] [clang-tools-extra] [clang] [llvm] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

[Teresa Johnson via cfe-commits]

@@ -0,0 +1,115 @@
+// This is a regression test for ThinLTO indirect-call-promotion when candidate
+// callees need to be imported from another IR module.  In the C++ test case,
+// `main` calls `global_func` which is defined in another module. `global_func`
+// has two indirect callees, one has external linkage and one has local 
linkage.
+// All three functions should be imported into the IR module of main.
+
+// What the test does:
+// - Generate raw profiles from executables and convert it to indexed profiles.
+//   During the conversion, a profiled callee address in raw profiles will be
+//   converted to function hash in indexed profiles.
+// - Run IRPGO profile use and ThinTLO prelink pipeline and get LLVM bitcodes
+//   for both cpp files in the C++ test case.
+// - Generate ThinLTO summary file with LLVM bitcodes, and run 
`function-import` pass.
+// - Run `pgo-icall-prom` pass for the IR module which needs to import callees.
+
+// Use lld as linker for more robust test. We need to REQUIRE LLVMgold.so for
+// LTO if default linker is GNU ld or gold anyway.
+// REQUIRES: lld-available
+
+// Test should fail where linkage-name and mangled-name diverges, see issue 
https://github.com/llvm/llvm-project/issues/74565).
+// Currently, this name divergence happens on Mach-O object file format, or on
+// many (but not all) 32-bit Windows systems.
+//
+// XFAIL: system-darwin
+//
+// Mark 32-bit Windows as UNSUPPORTED for now as opposed to XFAIL. This test
+// should fail on many (but not all) 32-bit Windows systems and succeed on the
+// rest. The flexibility in triple string parsing makes it tricky to capture
+// both sets accurately. i[3-9]86 specifies arch as Triple::ArchType::x86, 
(win32|windows)
+// specifies OS as Triple::OS::Win32
+//
+// UNSUPPORTED: target={{i[3-9]86-.*-(win32|windows).*}}
+
+// RUN: rm -rf %t && split-file %s %t && cd %t
+
+// Do setup work for all below tests.
+// Generate raw profiles from real programs and convert it into indexed 
profiles.
+// Use clangxx_pgogen for IR level instrumentation for C++.
+// RUN: %clangxx_pgogen -fuse-ld=lld -O2 lib.cpp main.cpp -o main
+// RUN: env LLVM_PROFILE_FILE=main.profraw %run ./main
+// RUN: llvm-profdata merge main.profraw -o main.profdata
+
+// Use profile on lib and get bitcode, test that local function callee0 has
+// expected !PGOFuncName metadata and external function callee1 doesn't have
+// !PGOFuncName metadata. Explicitly skip ICP pass to test ICP happens as
+// expected in the IR module that imports functions from lib.
+// RUN: %clang -mllvm -disable-icp -fprofile-use=main.profdata -flto=thin -O2 
-c lib.cpp -o lib.bc
+// RUN: llvm-dis lib.bc -o - | FileCheck %s --check-prefix=PGOName
+
+// Use profile on main and get bitcode.
+// RUN: %clang -fprofile-use=main.profdata -flto=thin -O2 -c main.cpp -o 
main.bc
+
+// Run llvm-lto to get summary file.
+// RUN: llvm-lto -thinlto -o summary main.bc lib.bc
+
+// Test the imports of functions. Default import thresholds would work but do
+// explicit override to be more futureproof. Note all functions have one basic
+// block with a function-entry-count of one, so they are actually hot functions
+// per default profile summary hotness cutoff.
+// RUN: opt -passes=function-import -import-instr-limit=100 
-import-cold-multiplier=1 -summary-file summary.thinlto.bc main.bc -o 
main.import.bc -print-imports 2>&1 | FileCheck %s --check-prefix=IMPORTS
+// Test that '_Z11global_funcv' has indirect calls annotated with value 
profiles.
+// RUN: llvm-dis main.import.bc -o - | FileCheck %s --check-prefix=IR
+
+// Test that both candidates are ICP'ed and there is no `!VP` in the IR.
+// RUN: opt main.import.bc -icp-lto -passes=pgo-icall-prom -S 
-pass-remarks=pgo-icall-prom 2>&1 | FileCheck %s 
--check-prefixes=ICP-IR,ICP-REMARK --implicit-check-not="!VP"
+
+// IMPORTS: main.cpp: Import _Z7callee1v
+// IMPORTS: main.cpp: Import _ZL7callee0v.llvm.[[#]]
+// IMPORTS: main.cpp: Import _Z11global_funcv
+
+// PGOName: define dso_local void @_Z7callee1v() {{.*}} !prof ![[#]] {
+// PGOName: define internal void @_ZL7callee0v() {{.*}} !prof ![[#]] 
!PGOFuncName ![[#MD:]] {
+// PGOName: ![[#MD]] = !{!"{{.*}}lib.cpp;_ZL7callee0v"}
+
+// IR-LABEL: define available_externally {{.*}} void @_Z11global_funcv() 
{{.*}} !prof ![[#]] {
+// IR-NEXT: entry:
+// IR-NEXT:  %0 = load ptr, ptr @calleeAddrs
+// IR-NEXT:  tail call void %0(), !prof ![[#PROF1:]]
+// IR-NEXT:  %1 = load ptr, ptr getelementptr inbounds ([2 x ptr], ptr 
@calleeAddrs,
+// IR-NEXT:  tail call void %1(), !prof ![[#PROF2:]]
+
+// The GUID of indirect callee is the MD5 hash of 
`/path/to/lib.cpp:_ZL7callee0v`

teresajohnson wrote:

should this have the semicolon separator not a colon?

https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[compiler-rt] [clang] [clang-tools-extra] [llvm] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

[Teresa Johnson via cfe-commits]

@@ -352,6 +366,8 @@ std::string getIRPGOFuncName(const Function &F, bool InLTO) 
{
   return getIRPGOObjectName(F, InLTO, getPGOFuncNameMetadata(F));
 }
 
+// DEPRECATED. Use `getIRPGOFuncName`for new code. See that function for

teresajohnson wrote:

In the header it is described as for FE instrumentation. Probably want the 
comments to be consistent?

https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[compiler-rt] [clang] [clang-tools-extra] [llvm] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

[Teresa Johnson via cfe-commits]

https://github.com/teresajohnson approved this pull request.

LGTM other than a couple of minor comments and pending resolution of the LLVM 
IR test. Thanks!

https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang] Emit type metadata on vtables when IRPGO instrumentation option is on. (PR #70841)

[Teresa Johnson via cfe-commits]

@@ -0,0 +1,145 @@
+// Test that type metadata are emitted with -fprofile-generate
+//
+// RUN: %clang -fprofile-generate -fno-lto -target x86_64-unknown-linux 
-emit-llvm -S %s -o - | FileCheck %s --check-prefix=ITANIUM
+// RUN: %clang -fprofile-generate -fno-lto -target x86_64-pc-windows-msvc 
-emit-llvm -S %s -o - | FileCheck %s --check-prefix=MS

teresajohnson wrote:

Yes I think that would be more typical and better imo (i.e. use %clang_cc to 
test)

https://github.com/llvm/llvm-project/pull/70841
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang] Emit type metadata on vtables when IRPGO instrumentation option is on. (PR #70841)

[Teresa Johnson via cfe-commits]

@@ -1,98 +1,116 @@
+
 // Tests for the cfi-vcall feature:
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm 
-o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT-NO-RV 
--check-prefix=ITANIUM --check-prefix=ITANIUM-MD 
--check-prefix=TT-ITANIUM-HIDDEN --check-prefix=NDIAG %s
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -emit-llvm -o - %s | FileCheck 
--check-prefix=CFI --check-prefix=CFI-NVT-NO-RV --check-prefix=ITANIUM 
--check-prefix=ITANIUM-MD --check-prefix=TT-ITANIUM-HIDDEN 
--check-prefix=ITANIUM-MD-DIAG --check-prefix=ITANIUM-DIAG --check-prefix=DIAG 
--check-prefix=DIAG-ABORT %s
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -fsanitize-recover=cfi-vcall 
-emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT-NO-RV 
--check-prefix=ITANIUM --check-prefix=ITANIUM-MD 
--check-prefix=TT-ITANIUM-HIDDEN --check-prefix=ITANIUM-MD-DIAG 
--check-prefix=ITANIUM-DIAG --check-prefix=DIAG --check-prefix=DIAG-RECOVER %s
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc 
-fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm -o - %s | FileCheck 
--check-prefix=CFI --check-prefix=CFI-NVT-NO-RV --check-prefix=MS 
--check-prefix=TT-MS --check-prefix=NDIAG %s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm 
-o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT-NO-RV 
--check-prefix=ITANIUM --check-prefix=ITANIUM-TYPEMETADATA 
--check-prefix=ITANIUM-MD --check-prefix=TT-ITANIUM-HIDDEN --check-prefix=NDIAG 
%s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -emit-llvm -o - %s | FileCheck 
--check-prefix=CFI --check-prefix=CFI-NVT-NO-RV --check-prefix=ITANIUM 
--check-prefix=ITANIUM-TYPEMETADATA --check-prefix=ITANIUM-MD 
--check-prefix=TT-ITANIUM-HIDDEN --check-prefix=ITANIUM-MD-DIAG 
--check-prefix=ITANIUM-DIAG --check-prefix=DIAG --check-prefix=DIAG-ABORT %s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -fsanitize-recover=cfi-vcall 
-emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT-NO-RV 
--check-prefix=ITANIUM --check-prefix=ITANIUM-TYPEMETADATA 
--check-prefix=ITANIUM-MD --check-prefix=TT-ITANIUM-HIDDEN 
--check-prefix=ITANIUM-MD-DIAG --check-prefix=ITANIUM-DIAG --check-prefix=DIAG 
--check-prefix=DIAG-RECOVER %s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc 
-fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm -o - %s | FileCheck 
--check-prefix=CFI --check-prefix=CFI-NVT-NO-RV --check-prefix=MS 
--check-prefix=MS-TYPEMETADATA --check-prefix=TT-MS --check-prefix=NDIAG %s
 
 // Tests for the whole-program-vtables feature:
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=VTABLE-OPT --check-prefix=ITANIUM --check-prefix=ITANIUM-MD 
--check-prefix=TT-ITANIUM-HIDDEN %s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=VTABLE-OPT --check-prefix=ITANIUM 
--check-prefix=ITANIUM-TYPEMETADATA --check-prefix=ITANIUM-MD 
--check-prefix=TT-ITANIUM-HIDDEN %s
 // RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=VTABLE-OPT --check-prefix=ITANIUM-DEFAULTVIS 
--check-prefix=TT-ITANIUM-DEFAULT %s
 // RUN: %clang_cc1 -O2 -flto -flto-unit -triple x86_64-unknown-linux 
-fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=ITANIUM-OPT --check-prefix=ITANIUM-OPT-LAYOUT %s
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc 
-fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=VTABLE-OPT --check-prefix=MS --check-prefix=TT-MS %s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc 
-fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=VTABLE-OPT --check-prefix=MS --check-prefix=MS-TYPEMETADATA 
--check-prefix=TT-MS %s
 
 // Tests for cfi + whole-program-vtables:
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall 
-fwhole-program-vtables -emit-llvm -o - %s | FileCheck --check-prefix=CFI 
--check-prefix=CFI-VT --check-prefix=ITANIUM --check-prefix=TC-ITANIUM 
--check-prefix=ITANIUM-MD %s
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc 
-fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -fwhole-program-vtables 
-emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-VT 
--check-prefix=MS --check-prefix=T

[clang] [Clang] Emit type metadata on vtables when IRPGO instrumentation option is on. (PR #70841)

[Teresa Johnson via cfe-commits]

@@ -1312,7 +1312,7 @@ llvm::GlobalObject::VCallVisibility 
CodeGenModule::GetVCallVisibilityLevel(
 void CodeGenModule::EmitVTableTypeMetadata(const CXXRecordDecl *RD,
llvm::GlobalVariable *VTable,
const VTableLayout &VTLayout) {
-  if (!getCodeGenOpts().LTOUnit)
+  if (!getCodeGenOpts().LTOUnit && !getCodeGenOpts().hasProfileIRInstr())

teresajohnson wrote:

Add a comment here and in the other file noting why it is added for IR 
instrumentation.

https://github.com/llvm/llvm-project/pull/70841
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang] Emit type metadata on vtables when IRPGO instrumentation option is on. (PR #70841)

[Teresa Johnson via cfe-commits]

@@ -1,98 +1,116 @@
+
 // Tests for the cfi-vcall feature:
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm 
-o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT-NO-RV 
--check-prefix=ITANIUM --check-prefix=ITANIUM-MD 
--check-prefix=TT-ITANIUM-HIDDEN --check-prefix=NDIAG %s
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -emit-llvm -o - %s | FileCheck 
--check-prefix=CFI --check-prefix=CFI-NVT-NO-RV --check-prefix=ITANIUM 
--check-prefix=ITANIUM-MD --check-prefix=TT-ITANIUM-HIDDEN 
--check-prefix=ITANIUM-MD-DIAG --check-prefix=ITANIUM-DIAG --check-prefix=DIAG 
--check-prefix=DIAG-ABORT %s
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -fsanitize-recover=cfi-vcall 
-emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT-NO-RV 
--check-prefix=ITANIUM --check-prefix=ITANIUM-MD 
--check-prefix=TT-ITANIUM-HIDDEN --check-prefix=ITANIUM-MD-DIAG 
--check-prefix=ITANIUM-DIAG --check-prefix=DIAG --check-prefix=DIAG-RECOVER %s
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc 
-fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm -o - %s | FileCheck 
--check-prefix=CFI --check-prefix=CFI-NVT-NO-RV --check-prefix=MS 
--check-prefix=TT-MS --check-prefix=NDIAG %s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm 
-o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT-NO-RV 
--check-prefix=ITANIUM --check-prefix=ITANIUM-TYPEMETADATA 
--check-prefix=ITANIUM-MD --check-prefix=TT-ITANIUM-HIDDEN --check-prefix=NDIAG 
%s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -emit-llvm -o - %s | FileCheck 
--check-prefix=CFI --check-prefix=CFI-NVT-NO-RV --check-prefix=ITANIUM 
--check-prefix=ITANIUM-TYPEMETADATA --check-prefix=ITANIUM-MD 
--check-prefix=TT-ITANIUM-HIDDEN --check-prefix=ITANIUM-MD-DIAG 
--check-prefix=ITANIUM-DIAG --check-prefix=DIAG --check-prefix=DIAG-ABORT %s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -fsanitize-recover=cfi-vcall 
-emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT-NO-RV 
--check-prefix=ITANIUM --check-prefix=ITANIUM-TYPEMETADATA 
--check-prefix=ITANIUM-MD --check-prefix=TT-ITANIUM-HIDDEN 
--check-prefix=ITANIUM-MD-DIAG --check-prefix=ITANIUM-DIAG --check-prefix=DIAG 
--check-prefix=DIAG-RECOVER %s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc 
-fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm -o - %s | FileCheck 
--check-prefix=CFI --check-prefix=CFI-NVT-NO-RV --check-prefix=MS 
--check-prefix=MS-TYPEMETADATA --check-prefix=TT-MS --check-prefix=NDIAG %s
 
 // Tests for the whole-program-vtables feature:
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=VTABLE-OPT --check-prefix=ITANIUM --check-prefix=ITANIUM-MD 
--check-prefix=TT-ITANIUM-HIDDEN %s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=VTABLE-OPT --check-prefix=ITANIUM 
--check-prefix=ITANIUM-TYPEMETADATA --check-prefix=ITANIUM-MD 
--check-prefix=TT-ITANIUM-HIDDEN %s
 // RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=VTABLE-OPT --check-prefix=ITANIUM-DEFAULTVIS 
--check-prefix=TT-ITANIUM-DEFAULT %s
 // RUN: %clang_cc1 -O2 -flto -flto-unit -triple x86_64-unknown-linux 
-fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=ITANIUM-OPT --check-prefix=ITANIUM-OPT-LAYOUT %s
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc 
-fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=VTABLE-OPT --check-prefix=MS --check-prefix=TT-MS %s
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc 
-fwhole-program-vtables -emit-llvm -o - %s | FileCheck 
--check-prefix=VTABLE-OPT --check-prefix=MS --check-prefix=MS-TYPEMETADATA 
--check-prefix=TT-MS %s
 
 // Tests for cfi + whole-program-vtables:
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux 
-fvisibility=hidden -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall 
-fwhole-program-vtables -emit-llvm -o - %s | FileCheck --check-prefix=CFI 
--check-prefix=CFI-VT --check-prefix=ITANIUM --check-prefix=TC-ITANIUM 
--check-prefix=ITANIUM-MD %s
-// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc 
-fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -fwhole-program-vtables 
-emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-VT 
--check-prefix=MS --check-prefix=T

[clang] [Clang] Emit type metadata on vtables when IRPGO instrumentation option is on. (PR #70841)

[Teresa Johnson via cfe-commits]

@@ -101,14 +119,16 @@
 // ITANIUM-OPT-SAME: !type [[EF16:![0-9]+]]
 // ITANIUM-OPT: @llvm.compiler.used = appending global [1 x ptr] [ptr 
@_ZTVN5test31EE]
 
-// MS: comdat($"??_7A@@6B@"), !type [[A8:![0-9]+]]
-// MS: comdat($"??_7B@@6B0@@"), !type [[B8:![0-9]+]]
-// MS: comdat($"??_7B@@6BA@@@"), !type [[A8]]
-// MS: comdat($"??_7C@@6B@"), !type [[A8]]
-// MS: comdat($"??_7D@?A0x{{[^@]*}}@@6BB@@@"), !type [[B8]], !type 
[[D8:![0-9]+]]
-// MS: comdat($"??_7D@?A0x{{[^@]*}}@@6BA@@@"), !type [[A8]]
-// MS: comdat($"??_7FA@?1??foo@@YAXXZ@6B@"), !type [[A8]], !type 
[[FA8:![0-9]+]]
+// MS-TYPEMETADATA: comdat($"??_7A@@6B@"), !type [[A8:![0-9]+]]
+// MS-TYPEMETADATA: comdat($"??_7B@@6B0@@"), !type [[B8:![0-9]+]]
+// MS-TYPEMETADATA: comdat($"??_7B@@6BA@@@"), !type [[A8]]
+// MS-TYPEMETADATA: comdat($"??_7C@@6B@"), !type [[A8]]
+// MS-TYPEMETADATA: comdat($"??_7D@?A0x{{[^@]*}}@@6BB@@@"), !type [[B8]], 
!type [[D8:![0-9]+]]
+// MS-TYPEMETADATA: comdat($"??_7D@?A0x{{[^@]*}}@@6BA@@@"), !type [[A8]]
+// MS-TYPEMETADATA: comdat($"??_7FA@?1??foo@@YAXXZ@6B@"), !type [[A8]], !type 
[[FA8:![0-9]+]]
 
+// Test !type doesn't exist in the generated IR.
+// NOTYPEMD-NOT: !type

teresajohnson wrote:

Instead of this do an --implicit-check-not='!type' on the FileCheck invocations

https://github.com/llvm/llvm-project/pull/70841
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang] Emit type metadata on vtables when IRPGO instrumentation option is on. (PR #70841)

[Teresa Johnson via cfe-commits]

https://github.com/teresajohnson approved this pull request.

Somehow I missed those other uses of the ITANIUM prefix. Thanks for cleaning it 
up though I think the new names are more descriptive.

https://github.com/llvm/llvm-project/pull/70841
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] Disable PGO instrumentation on naked function (PR #75224)

[Teresa Johnson via cfe-commits]

@@ -892,6 +892,10 @@ void CodeGenFunction::StartFunction(GlobalDecl GD, 
QualType RetTy,
 }
   }
 
+  if (FD->hasAttr()) {

teresajohnson wrote:

Is this change needed given the separate change in LLVM skipPGOGen?

https://github.com/llvm/llvm-project/pull/75224
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] Disable PGO instrumentation on naked function (PR #75224)

[Teresa Johnson via cfe-commits]

https://github.com/teresajohnson approved this pull request.


https://github.com/llvm/llvm-project/pull/75224
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[lld] [llvm] [clang] [LTO] Improve diagnostics handling when parsing module-level inline assembly (PR #75726)

[Teresa Johnson via cfe-commits]

https://github.com/teresajohnson approved this pull request.


https://github.com/llvm/llvm-project/pull/75726
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] DiagnosticHandler: refactor error checking (PR #75889)

[Teresa Johnson via cfe-commits]

@@ -256,10 +256,13 @@ void LLVMContext::diagnose(const DiagnosticInfo &DI) {
   RS->emit(*OptDiagBase);
 
   // If there is a report handler, use it.
-  if (pImpl->DiagHandler &&
-  (!pImpl->RespectDiagnosticFilters || isDiagnosticEnabled(DI)) &&
-  pImpl->DiagHandler->handleDiagnostics(DI))
-return;
+  if (pImpl->DiagHandler) {
+if (DI.getSeverity() == DS_Error)
+  pImpl->DiagHandler->HasErrors = true;

teresajohnson wrote:

Perhaps move the set of HasErrors into a new non-virtual method that we call 
below just before calling handleDiagnostics.

Also, should this be set when handleDiagnostics is not called?

https://github.com/llvm/llvm-project/pull/75889
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] DiagnosticHandler: refactor error checking (PR #75889)

[Teresa Johnson via cfe-commits]

@@ -256,10 +256,13 @@ void LLVMContext::diagnose(const DiagnosticInfo &DI) {
   RS->emit(*OptDiagBase);
 
   // If there is a report handler, use it.
-  if (pImpl->DiagHandler &&
-  (!pImpl->RespectDiagnosticFilters || isDiagnosticEnabled(DI)) &&
-  pImpl->DiagHandler->handleDiagnostics(DI))
-return;
+  if (pImpl->DiagHandler) {
+if (DI.getSeverity() == DS_Error)
+  pImpl->DiagHandler->HasErrors = true;

teresajohnson wrote:

I didn't mean calling handle Diagnostics in more places. I just meant rather 
than directly setting the HasErrors flags here, do that in a new non-virtual 
method (e.g. prepareToHandleDiagnostics() or whatever), and call it here just 
before calling handle Diagnostics. To abstract away what it is actually doing 
and leave the setting of the flag to the DiagnosticHandler class.

https://github.com/llvm/llvm-project/pull/75889
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] DiagnosticHandler: refactor error checking (PR #75889)

[Teresa Johnson via cfe-commits]

https://github.com/teresajohnson approved this pull request.

lgtm

https://github.com/llvm/llvm-project/pull/75889
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [MemProf] Expand optimization scope to internal linakge function (PR #73236)

[Teresa Johnson via cfe-commits]

teresajohnson wrote:

@snehasish can you take a look at the issue described here? Should we be doing 
something different in llvm-profdata?

https://github.com/llvm/llvm-project/pull/73236
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][llvm][fatlto] Avoid cloning modules in FatLTO (PR #72180)

[Teresa Johnson via cfe-commits]

@@ -810,7 +810,7 @@ void EmitAssemblyHelper::RunOptimizationPipeline(
   // Only enable CGProfilePass when using integrated assembler, since
   // non-integrated assemblers don't recognize .cgprofile section.
   PTO.CallGraphProfile = !CodeGenOpts.DisableIntegratedAS;
-  PTO.UnifiedLTO = CodeGenOpts.UnifiedLTO;
+  PTO.UnifiedLTO = CodeGenOpts.UnifiedLTO || CodeGenOpts.FatLTO;

teresajohnson wrote:

Won't your change to lib/Driver/ToolChains/Clang.cpp below mean that 
CodeGenOpts.UnifiedLTO will always be set with FatLTO? Do we thus need this 
change or the one further below for the module flag (and maybe add an assert 
that UnifiedLTO is set of FatLTO is set)?

https://github.com/llvm/llvm-project/pull/72180
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][llvm][fatlto] Avoid cloning modules in FatLTO (PR #72180)

[Teresa Johnson via cfe-commits]

@@ -810,7 +810,7 @@ void EmitAssemblyHelper::RunOptimizationPipeline(
   // Only enable CGProfilePass when using integrated assembler, since
   // non-integrated assemblers don't recognize .cgprofile section.
   PTO.CallGraphProfile = !CodeGenOpts.DisableIntegratedAS;
-  PTO.UnifiedLTO = CodeGenOpts.UnifiedLTO;
+  PTO.UnifiedLTO = CodeGenOpts.UnifiedLTO || CodeGenOpts.FatLTO;

teresajohnson wrote:

Not sure I follow. Isn't the change in Driver/Toolchains/Clang.cpp going to 
ensure that -funified-lto is also passed to the cc1 invocation, and thus both 
options should be set in CodeGenOpts during the cc1 invocation?

https://github.com/llvm/llvm-project/pull/72180
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [clang][llvm][fatlto] Avoid cloning modules in FatLTO (PR #72180)

[Teresa Johnson via cfe-commits]

@@ -810,7 +810,7 @@ void EmitAssemblyHelper::RunOptimizationPipeline(
   // Only enable CGProfilePass when using integrated assembler, since
   // non-integrated assemblers don't recognize .cgprofile section.
   PTO.CallGraphProfile = !CodeGenOpts.DisableIntegratedAS;
-  PTO.UnifiedLTO = CodeGenOpts.UnifiedLTO;
+  PTO.UnifiedLTO = CodeGenOpts.UnifiedLTO || CodeGenOpts.FatLTO;

teresajohnson wrote:

Ah ok. I guess that would be expected. Whether it is test friendly is another 
question, but in general the driver does a lot of option set up for the cc1 
invocation.

https://github.com/llvm/llvm-project/pull/72180
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][llvm][fatlto] Avoid cloning modules in FatLTO (PR #72180)

[Teresa Johnson via cfe-commits]

@@ -810,7 +810,7 @@ void EmitAssemblyHelper::RunOptimizationPipeline(
   // Only enable CGProfilePass when using integrated assembler, since
   // non-integrated assemblers don't recognize .cgprofile section.
   PTO.CallGraphProfile = !CodeGenOpts.DisableIntegratedAS;
-  PTO.UnifiedLTO = CodeGenOpts.UnifiedLTO;
+  PTO.UnifiedLTO = CodeGenOpts.UnifiedLTO || CodeGenOpts.FatLTO;

teresajohnson wrote:

Actually, I believe the cc1 parsing is handled by ParseCodeGenArgs in 
clang/lib/Frontend/CompilerInvocation.cpp. You can potentially add something in 
there to ensure UnifiedLTO is set with FatLTO? Or give your error there that 
UnifiedLTO must be specified with Fat LTO.

https://github.com/llvm/llvm-project/pull/72180
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [MemProf] Expand optimization scope to internal linakge function (PR #73236)

[Teresa Johnson via cfe-commits]

teresajohnson wrote:

> @lifengxiang1025 thanks for flagging this issue. I think it's best to not 
> rely on unique-internal-linkage-name here. Instead we should extend the logic 
> in RawMemProfReader.cpp to include "filename;" if the function is internal 
> linkage as expected by IRPGOFuncName. Can you try this approach?

I don't think we want to change the name in the frames themselves, as the names 
in the debug info when matching don't contain this prefix. Although I suppose 
we could modify the matcher to add the prefixes when matching frames too. I.e. 
here: 
https://github.com/llvm/llvm-project/blob/c0fe0719df457b0992606b0f2a235719a5bbfd54/llvm/lib/Transforms/Instrumentation/MemProfiler.cpp#L806

It sounds like the issue is that the names used to index the memprof records do 
not use the PGOFuncName scheme and therefore we never find any memprof records 
for an internal linkage function during matching. I.e. the GUID passed to 
InstrProfWriter::addMemProfRecord. Unfortunately, looking at 
RawMemProfReader.cpp it does look like that eventually gets populated out of 
the Function (GUID) field from same Frame setup code. So we'd either need to do 
some extra handling in that code so that the prefix is only used for the record 
entry, or change the matcher (the latter may be the easiest).

https://github.com/llvm/llvm-project/pull/73236
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][llvm][fatlto] Avoid cloning modules in FatLTO (PR #72180)

[Teresa Johnson via cfe-commits]

@@ -1861,6 +1861,13 @@ bool CompilerInvocation::ParseCodeGenArgs(CodeGenOptions 
&Opts, ArgList &Args,
 if (Args.hasArg(OPT_funified_lto))
   Opts.PrepareForThinLTO = true;
   }
+  if (Arg *A = Args.getLastArg(options::OPT_ffat_lto_objects,
+   options::OPT_fno_fat_lto_objects)) {
+if (!Args.hasArg(OPT_funified_lto))
+  Diags.Report(diag::err_drv_incompatible_options)

teresajohnson wrote:

It might be less confusing to users if this error message is only given upon an 
explicit -fno-unified-lto, and diag::err_drv_argument_only_allowed_with is used 
for the lack of -funified-lto.

Also can you add driver tests to check that we get the expected error(s) in the 
expected option combinations?

https://github.com/llvm/llvm-project/pull/72180
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][llvm][fatlto] Avoid cloning modules in FatLTO (PR #72180)

[Teresa Johnson via cfe-commits]

https://github.com/teresajohnson approved this pull request.

lgtm

https://github.com/llvm/llvm-project/pull/72180
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [clang][llvm][fatlto] Avoid cloning modules in FatLTO (PR #72180)

[Teresa Johnson via cfe-commits]

@@ -1530,14 +1530,11 @@ 
PassBuilder::buildPerModuleDefaultPipeline(OptimizationLevel Level,
 }
 
 ModulePassManager
-PassBuilder::buildFatLTODefaultPipeline(OptimizationLevel Level, bool ThinLTO,
-bool EmitSummary) {
+PassBuilder::buildFatLTODefaultPipeline(OptimizationLevel Level) {
   ModulePassManager MPM;
-  MPM.addPass(EmbedBitcodePass(ThinLTO, EmitSummary,
-   ThinLTO
-   ? buildThinLTOPreLinkDefaultPipeline(Level)
-   : buildLTOPreLinkDefaultPipeline(Level)));
-  MPM.addPass(buildPerModuleDefaultPipeline(Level));
+  MPM.addPass(buildThinLTOPreLinkDefaultPipeline(Level));
+  MPM.addPass(EmbedBitcodePass());
+  MPM.addPass(buildThinLTODefaultPipeline(Level, /*ImportSummary=*/nullptr));

teresajohnson wrote:

What was the downside of using ThinLTODefaultPipeline always? I guess it was 
essentially over-optimizing in the non-LTO case? I guess using the 
ThinLTODefaultPipeline only under SamplePGO is ok with me, although it seems 
like over time as the pipelines get modified it will be difficult to ensure 
that is the only case where the pipelines get out of sync. I think in either 
case we are essentially saying: if you use Fat LTO then don't expect the 
resulting non-LTO native code to be the same as that of a fully non-LTO 
compile. In one case there is more optimization, in the other there is the risk 
of future deoptimization if things don't stay in sync.

https://github.com/llvm/llvm-project/pull/72180
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [MemProf] Expand optimization scope to internal linakge function (PR #73236)

[Teresa Johnson via cfe-commits]

teresajohnson wrote:

> > Yes, you're right. As an alternative can we use the symbol table and find 
> > Bind = LOCAL to add the prefix before hashing?
> 
> If we choose this method. I think we can't deal with the situation which one 
> symbol is not local linkage type in thin compile, but will be converted to 
> local linkage after thin backend by thinlto's internalization. In this 
> situation function name in llvm-profdata will have prefix with file name. But 
> when llvm consumes memory profile, PGOFuncName won't return function name 
> with prefix.

If I understand the issue you are describing, that would only occur if the 
instrumentation build also used ThinLTO. Is that a typical use case for you?

https://github.com/llvm/llvm-project/pull/73236
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [MemProf] Expand optimization scope to internal linakge function (PR #73236)

[Teresa Johnson via cfe-commits]

teresajohnson wrote:

@snehasish and I chatted more about this offline. Using the dwarf info to 
figure out the right source name prefix during indexing is not straightforward. 
The source file name used as the prefix in the compiler for the IRPGOName is 
that of the TU. The source file of the line in the dwarf info may be a header. 
It could be possible to walk up the profiled inline frames to find a source 
file, but that is error prone in certain circumstances (e.g. if the profiled 
binary had thinlto cross module inlining, and if it isn't obvious which source 
file name is a non-included TU).

I think your change to the matching is almost strictly better than the complete 
non-matching we get currently for local functions. It would even work in many 
cases without uniq suffixes, but uniq suffixes will make that work even better. 
So let's go ahead with that change.

However, we'd like to request that you split the __cxx_global_var_init change 
into a separate patch, as it is somewhat orthogonal to the matching change, and 
in case there are unexpected issues with that change. Can you split that out 
and make the test case here for matching of a non __cxx_global_var_init 
function?

Thanks!

https://github.com/llvm/llvm-project/pull/73236
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

[Teresa Johnson via cfe-commits]

@@ -300,12 +316,8 @@ getIRPGONameForGlobalObject(const GlobalObject &GO,
 GlobalValue::LinkageTypes Linkage,
 StringRef FileName) {
   SmallString<64> Name;
-  if (llvm::GlobalValue::isLocalLinkage(Linkage)) {
-Name.append(FileName.empty() ? "" : FileName);
-Name.append(";");
-  }
   Mangler().getNameWithPrefix(Name, &GO, /*CannotUsePrivateLabel=*/true);

teresajohnson wrote:

@ellishg How much of D156569 relied on the invocation of Mangler? It is not 
mentioned in the patch description, only the rationale for changing ":" to ";". 
The problem is if these are out of sync, then cross-module importing of 
indirectly called local functions will continue to be broken in whatever cases 
Mangler().getNameWithPrefix affects.

https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits