Hi Richard, This patch broke ARM bots which don't have x86_64 target available. Logs are available here:
http://lab.llvm.org:8011/builders/clang-cmake-armv7-quick/builds/4326/steps/ninja%20check%201/logs/FAIL%3A%20Clang%3A%3Aprofile-remap.cpp Cheers, Yvan On Thu, 11 Oct 2018 at 01:15, Richard Smith via cfe-commits <cfe-commits@lists.llvm.org> wrote: > > Author: rsmith > Date: Wed Oct 10 16:13:35 2018 > New Revision: 344199 > > URL: http://llvm.org/viewvc/llvm-project?rev=344199&view=rev > Log: > Add a flag to remap manglings when reading profile data information. > > This can be used to preserve profiling information across codebase > changes that have widespread impact on mangled names, but across which > most profiling data should still be usable. For example, when switching > from libstdc++ to libc++, or from the old libstdc++ ABI to the new ABI, > or even from a 32-bit to a 64-bit build. > > The user can provide a remapping file specifying parts of mangled names > that should be treated as equivalent (eg, std::__1 should be treated as > equivalent to std::__cxx11), and profile data will be treated as > applying to a particular function if its name is equivalent to the name > of a function in the profile data under the provided equivalences. See > the documentation change for a description of how this is configured. > > Remapping is supported for both sample-based profiling and instruction > profiling. We do not support remapping indirect branch target > information, but all other profile data should be remapped > appropriately. > > Support is only added for the new pass manager. If someone wants to also > add support for this for the old pass manager, doing so should be > straightforward. > > Added: > cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map > cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext > cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples > cfe/trunk/test/CodeGenCXX/profile-remap.cpp > Modified: > cfe/trunk/docs/ReleaseNotes.rst > cfe/trunk/docs/UsersManual.rst > cfe/trunk/include/clang/Driver/Options.td > cfe/trunk/include/clang/Frontend/CodeGenOptions.h > cfe/trunk/lib/CodeGen/BackendUtil.cpp > cfe/trunk/lib/CodeGen/CodeGenModule.cpp > cfe/trunk/lib/Driver/ToolChains/Clang.cpp > cfe/trunk/lib/Frontend/CompilerInvocation.cpp > cfe/trunk/test/Driver/clang_f_opts.c > > Modified: cfe/trunk/docs/ReleaseNotes.rst > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/ReleaseNotes.rst?rev=344199&r1=344198&r2=344199&view=diff > ============================================================================== > --- cfe/trunk/docs/ReleaseNotes.rst (original) > +++ cfe/trunk/docs/ReleaseNotes.rst Wed Oct 10 16:13:35 2018 > @@ -46,6 +46,11 @@ sections with improvements to Clang's su > Major New Features > ------------------ > > +- Clang supports use of a profile remapping file, which permits > + profile data captured for one version of a program to be applied > + when building another version where symbols have changed (for > + example, due to renaming a class or namespace). > + See the :doc:`UsersManual` for details. > > Improvements to Clang's diagnostics > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > Modified: cfe/trunk/docs/UsersManual.rst > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/UsersManual.rst?rev=344199&r1=344198&r2=344199&view=diff > ============================================================================== > --- cfe/trunk/docs/UsersManual.rst (original) > +++ cfe/trunk/docs/UsersManual.rst Wed Oct 10 16:13:35 2018 > @@ -1799,6 +1799,69 @@ In these cases, you can use the flag ``- > Note that these flags should appear after the corresponding profile > flags to have an effect. > > +Profile remapping > +^^^^^^^^^^^^^^^^^ > + > +When the program is compiled after a change that affects many symbol names, > +pre-existing profile data may no longer match the program. For example: > + > + * switching from libstdc++ to libc++ will result in the mangled names of all > + functions taking standard library types to change > + * renaming a widely-used type in C++ will result in the mangled names of all > + functions that have parameters involving that type to change > + * moving from a 32-bit compilation to a 64-bit compilation may change the > + underlying type of ``size_t`` and similar types, resulting in changes to > + manglings > + > +Clang allows use of a profile remapping file to specify that such differences > +in mangled names should be ignored when matching the profile data against the > +program. > + > +.. option:: -fprofile-remapping-file=<file> > + > + Specifies a file containing profile remapping information, that will be > + used to match mangled names in the profile data to mangled names in the > + program. > + > +The profile remapping file is a text file containing lines of the form > + > +.. code-block:: > + > + fragmentkind fragment1 fragment2 > + > +where ``fragmentkind`` is one of ``name``, ``type``, or ``encoding``, > +indicating whether the following mangled name fragments are > +<`name <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.name>`>s, > +<`type <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.type>`>s, or > +<`encoding > <http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangle.encoding>`>s, > +respectively. > + > +Blank lines and lines starting with ``#`` are ignored. > + > +For example, to specify that ``absl::string_view`` and ``std::string_view`` > +should be treated as equivalent when matching profile data, the following > +remapping file could be used: > + > +.. code-block:: > + > + # absl::string_view is considered equivalent to std::string_view > + type N4absl11string_viewE St17basic_string_viewIcSt11char_traitsIcEE > + > + # std:: might be std::__1:: in libc++ or std::__cxx11:: in libstdc++ > + name 3std St3__1 > + name 3std St7__cxx11 > + > +Matching profile data using a profile remapping file is supported on a > +best-effort basis. For example, information regarding indirect call targets > is > +currently not remapped. For best results, you are encouraged to generate new > +profile data matching the updated program. > + > +.. note:: > + > + Profile data remapping is currently only supported for C++ mangled names > + following the Itanium C++ ABI mangling scheme. This covers all C++ targets > + supported by Clang other than Windows. > + > Controlling Debug Information > ----------------------------- > > > Modified: cfe/trunk/include/clang/Driver/Options.td > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Options.td?rev=344199&r1=344198&r2=344199&view=diff > ============================================================================== > --- cfe/trunk/include/clang/Driver/Options.td (original) > +++ cfe/trunk/include/clang/Driver/Options.td Wed Oct 10 16:13:35 2018 > @@ -735,6 +735,11 @@ def fprofile_instr_use : Flag<["-"], "fp > def fprofile_instr_use_EQ : Joined<["-"], "fprofile-instr-use=">, > Group<f_Group>, Flags<[CoreOption]>, > HelpText<"Use instrumentation data for profile-guided optimization">; > +def fprofile_remapping_file_EQ : Joined<["-"], "fprofile-remapping-file=">, > + Group<f_Group>, Flags<[CC1Option, CoreOption]>, MetaVarName<"<file>">, > + HelpText<"Use the remappings described in <file> to match the profile > data against names in the program">; > +def fprofile_remapping_file : Separate<["-"], "fprofile-remapping-file">, > + Group<f_Group>, Flags<[CoreOption]>, Alias<fprofile_remapping_file_EQ>; > def fcoverage_mapping : Flag<["-"], "fcoverage-mapping">, > Group<f_Group>, Flags<[CC1Option, CoreOption]>, > HelpText<"Generate coverage mapping to enable code coverage analysis">; > > Modified: cfe/trunk/include/clang/Frontend/CodeGenOptions.h > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/CodeGenOptions.h?rev=344199&r1=344198&r2=344199&view=diff > ============================================================================== > --- cfe/trunk/include/clang/Frontend/CodeGenOptions.h (original) > +++ cfe/trunk/include/clang/Frontend/CodeGenOptions.h Wed Oct 10 16:13:35 2018 > @@ -200,6 +200,10 @@ public: > /// Name of the profile file to use as input for -fprofile-instr-use > std::string ProfileInstrumentUsePath; > > + /// Name of the profile remapping file to apply to the profile data > supplied > + /// by -fprofile-sample-use or -fprofile-instr-use. > + std::string ProfileRemappingFile; > + > /// Name of the function summary index file to use for ThinLTO function > /// importing. > std::string ThinLTOIndexFile; > > Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=344199&r1=344198&r2=344199&view=diff > ============================================================================== > --- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original) > +++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Wed Oct 10 16:13:35 2018 > @@ -930,18 +930,21 @@ void EmitAssemblyHelper::EmitAssemblyWit > PGOOpt = PGOOptions(CodeGenOpts.InstrProfileOutput.empty() > ? DefaultProfileGenName > : CodeGenOpts.InstrProfileOutput, > - "", "", true, CodeGenOpts.DebugInfoForProfiling); > + "", "", "", true, > + CodeGenOpts.DebugInfoForProfiling); > else if (CodeGenOpts.hasProfileIRUse()) > // -fprofile-use. > - PGOOpt = PGOOptions("", CodeGenOpts.ProfileInstrumentUsePath, "", false, > + PGOOpt = PGOOptions("", CodeGenOpts.ProfileInstrumentUsePath, "", > + CodeGenOpts.ProfileRemappingFile, false, > CodeGenOpts.DebugInfoForProfiling); > else if (!CodeGenOpts.SampleProfileFile.empty()) > // -fprofile-sample-use > - PGOOpt = PGOOptions("", "", CodeGenOpts.SampleProfileFile, false, > + PGOOpt = PGOOptions("", "", CodeGenOpts.SampleProfileFile, > + CodeGenOpts.ProfileRemappingFile, false, > CodeGenOpts.DebugInfoForProfiling); > else if (CodeGenOpts.DebugInfoForProfiling) > // -fdebug-info-for-profiling > - PGOOpt = PGOOptions("", "", "", false, true); > + PGOOpt = PGOOptions("", "", "", "", false, true); > > PassBuilder PB(TM.get(), PGOOpt); > > @@ -1130,6 +1133,7 @@ static void runThinLTOBackend(ModuleSumm > const LangOptions &LOpts, > std::unique_ptr<raw_pwrite_stream> OS, > std::string SampleProfile, > + std::string ProfileRemapping, > BackendAction Action) { > StringMap<DenseMap<GlobalValue::GUID, GlobalValueSummary *>> > ModuleToDefinedGVSummaries; > @@ -1202,6 +1206,7 @@ static void runThinLTOBackend(ModuleSumm > Conf.CGOptLevel = getCGOptLevel(CGOpts); > initTargetOptions(Conf.Options, CGOpts, TOpts, LOpts, HeaderOpts); > Conf.SampleProfile = std::move(SampleProfile); > + Conf.ProfileRemapping = std::move(ProfileRemapping); > Conf.UseNewPM = CGOpts.ExperimentalNewPassManager; > Conf.DebugPassManager = CGOpts.DebugPassManager; > Conf.RemarksWithHotness = CGOpts.DiagnosticsWithHotness; > @@ -1268,7 +1273,7 @@ void clang::EmitBackendOutput(Diagnostic > if (!CombinedIndex->skipModuleByDistributedBackend()) { > runThinLTOBackend(CombinedIndex.get(), M, HeaderOpts, CGOpts, TOpts, > LOpts, std::move(OS), CGOpts.SampleProfileFile, > - Action); > + CGOpts.ProfileRemappingFile, Action); > return; > } > // Distributed indexing detected that nothing from the module is needed > > Modified: cfe/trunk/lib/CodeGen/CodeGenModule.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenModule.cpp?rev=344199&r1=344198&r2=344199&view=diff > ============================================================================== > --- cfe/trunk/lib/CodeGen/CodeGenModule.cpp (original) > +++ cfe/trunk/lib/CodeGen/CodeGenModule.cpp Wed Oct 10 16:13:35 2018 > @@ -154,7 +154,7 @@ CodeGenModule::CodeGenModule(ASTContext > > if (CodeGenOpts.hasProfileClangUse()) { > auto ReaderOrErr = llvm::IndexedInstrProfReader::create( > - CodeGenOpts.ProfileInstrumentUsePath); > + CodeGenOpts.ProfileInstrumentUsePath, > CodeGenOpts.ProfileRemappingFile); > if (auto E = ReaderOrErr.takeError()) { > unsigned DiagID = Diags.getCustomDiagID(DiagnosticsEngine::Error, > "Could not read profile %0: > %1"); > > Modified: cfe/trunk/lib/Driver/ToolChains/Clang.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/ToolChains/Clang.cpp?rev=344199&r1=344198&r2=344199&view=diff > ============================================================================== > --- cfe/trunk/lib/Driver/ToolChains/Clang.cpp (original) > +++ cfe/trunk/lib/Driver/ToolChains/Clang.cpp Wed Oct 10 16:13:35 2018 > @@ -4355,6 +4355,7 @@ void Clang::ConstructJob(Compilation &C, > else > A->render(Args, CmdArgs); > } > + Args.AddLastArg(CmdArgs, options::OPT_fprofile_remapping_file_EQ); > > RenderBuiltinOptions(TC, RawTriple, Args, CmdArgs); > > > Modified: cfe/trunk/lib/Frontend/CompilerInvocation.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CompilerInvocation.cpp?rev=344199&r1=344198&r2=344199&view=diff > ============================================================================== > --- cfe/trunk/lib/Frontend/CompilerInvocation.cpp (original) > +++ cfe/trunk/lib/Frontend/CompilerInvocation.cpp Wed Oct 10 16:13:35 2018 > @@ -657,6 +657,13 @@ static bool ParseCodeGenArgs(CodeGenOpti > Args.getLastArgValue(OPT_fprofile_instrument_use_path_EQ); > if (!Opts.ProfileInstrumentUsePath.empty()) > setPGOUseInstrumentor(Opts, Opts.ProfileInstrumentUsePath); > + Opts.ProfileRemappingFile = > + Args.getLastArgValue(OPT_fprofile_remapping_file_EQ); > + if (!Opts.ProfileRemappingFile.empty() && > !Opts.ExperimentalNewPassManager) { > + Diags.Report(diag::err_drv_argument_only_allowed_with) > + << Args.getLastArg(OPT_fprofile_remapping_file_EQ)->getAsString(Args) > + << "-fexperimental-new-pass-manager"; > + } > > Opts.CoverageMapping = > Args.hasFlag(OPT_fcoverage_mapping, OPT_fno_coverage_mapping, false); > > Added: cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map?rev=344199&view=auto > ============================================================================== > --- cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map (added) > +++ cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.map Wed Oct 10 16:13:35 > 2018 > @@ -0,0 +1,2 @@ > +name 3Foo 3Bar > +type N3Foo1XE N3Baz1YE > > Added: cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext?rev=344199&view=auto > ============================================================================== > --- cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext (added) > +++ cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.proftext Wed Oct 10 > 16:13:35 2018 > @@ -0,0 +1,7 @@ > +:ir > +_ZN3Foo8functionENS_1XE > +29667547796 > +2 > +10 > +90 > + > > Added: cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples?rev=344199&view=auto > ============================================================================== > --- cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples (added) > +++ cfe/trunk/test/CodeGenCXX/Inputs/profile-remap.samples Wed Oct 10 > 16:13:35 2018 > @@ -0,0 +1,3 @@ > +_ZN3Bar8functionEN3Baz1YE:100:0 > + 2: 10 > + 4: 90 > > Added: cfe/trunk/test/CodeGenCXX/profile-remap.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/profile-remap.cpp?rev=344199&view=auto > ============================================================================== > --- cfe/trunk/test/CodeGenCXX/profile-remap.cpp (added) > +++ cfe/trunk/test/CodeGenCXX/profile-remap.cpp Wed Oct 10 16:13:35 2018 > @@ -0,0 +1,27 @@ > +// RUN: %clang_cc1 -triple x86_64-linux-gnu > -fprofile-sample-use=%S/Inputs/profile-remap.samples > -fprofile-remapping-file=%S/Inputs/profile-remap.map > -fexperimental-new-pass-manager -O2 %s -emit-llvm -o - | FileCheck %s > --check-prefixes=CHECK,CHECK-SAMPLES > +// RUN: llvm-profdata merge -output %T.profdata > %S/Inputs/profile-remap.proftext > +// RUN: %clang_cc1 -triple x86_64-linux-gnu > -fprofile-instrument-use-path=%T.profdata > -fprofile-remapping-file=%S/Inputs/profile-remap.map > -fexperimental-new-pass-manager -O2 %s -emit-llvm -o - | FileCheck %s > --check-prefixes=CHECK,CHECK-INSTR > + > +namespace Foo { > + struct X {}; > + bool cond(); > + void bar(); > + void baz(); > + void function(X x) { > + if (cond()) > + bar(); > + else > + baz(); > + } > +} > + > +// CHECK: define {{.*}} @_ZN3Foo8functionENS_1XE() {{.*}} !prof > [[FUNC_ENTRY:![0-9]*]] > +// CHECK: br i1 {{.*}} !prof [[BR_WEIGHTS:![0-9]*]] > +// > +// FIXME: Laplace's rule of succession is applied to sample profiles... > +// CHECK-SAMPLES-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 1} > +// CHECK-SAMPLES-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i32 11, i32 91} > +// > +// ... but not to instruction profiles. > +// CHECK-INSTR-DAG: [[FUNC_ENTRY]] = !{!"function_entry_count", i64 100} > +// CHECK-INSTR-DAG: [[BR_WEIGHTS]] = !{!"branch_weights", i32 10, i32 90} > > Modified: cfe/trunk/test/Driver/clang_f_opts.c > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/clang_f_opts.c?rev=344199&r1=344198&r2=344199&view=diff > ============================================================================== > --- cfe/trunk/test/Driver/clang_f_opts.c (original) > +++ cfe/trunk/test/Driver/clang_f_opts.c Wed Oct 10 16:13:35 2018 > @@ -116,6 +116,7 @@ > // RUN: %clang -### -S -fcoverage-mapping %s 2>&1 | FileCheck > -check-prefix=CHECK-COVERAGE-AND-GEN %s > // RUN: %clang -### -S -fcoverage-mapping -fno-coverage-mapping %s 2>&1 | > FileCheck -check-prefix=CHECK-DISABLE-COVERAGE %s > // RUN: %clang -### -S -fprofile-instr-generate -fcoverage-mapping > -fno-coverage-mapping %s 2>&1 | FileCheck > -check-prefix=CHECK-DISABLE-COVERAGE %s > +// RUN: %clang -### -S -fprofile-remapping-file foo/bar.txt %s 2>&1 | > FileCheck -check-prefix=CHECK-PROFILE-REMAP %s > // CHECK-PROFILE-GENERATE: "-fprofile-instrument=clang" > // CHECK-PROFILE-GENERATE-LLVM: "-fprofile-instrument=llvm" > // CHECK-PROFILE-GENERATE-DIR: > "-fprofile-instrument-path=/some/dir{{/|\\\\}}{{.*}}" > @@ -126,6 +127,7 @@ > // CHECK-DISABLE-USE-NOT: "-fprofile-instr-use" > // CHECK-COVERAGE-AND-GEN: '-fcoverage-mapping' only allowed with > '-fprofile-instr-generate' > // CHECK-DISABLE-COVERAGE-NOT: "-fcoverage-mapping" > +// CHECK-PROFILE-REMAP: "-fprofile-remapping-file=foo/bar.txt" > > // RUN: %clang -### -S -fprofile-use %s 2>&1 | FileCheck > -check-prefix=CHECK-PROFILE-USE %s > // RUN: %clang -### -S -fprofile-instr-use %s 2>&1 | FileCheck > -check-prefix=CHECK-PROFILE-USE %s > > > _______________________________________________ > cfe-commits mailing list > cfe-commits@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits _______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
