r267059 - [esan] EfficiencySanitizer driver flags
Author: bruening
Date: Thu Apr 21 16:32:04 2016
New Revision: 267059
URL: http://llvm.org/viewvc/llvm-project?rev=267059&view=rev
Log:
[esan] EfficiencySanitizer driver flags
Summary:
Adds a framework to enable the instrumentation pass for the new
EfficiencySanitizer ("esan") family of tools. Adds a flag for esan's
cache fragmentation tool via -fsanitize=efficiency-cache-frag.
Adds appropriate tests for the new flag.
Reviewers: eugenis, vitalybuka, aizatsky, filcab
Subscribers: filcab, kubabrecka, llvm-commits, zhaoqin, kcc
Differential Revision: http://reviews.llvm.org/D19169
Added:
cfe/trunk/test/Driver/esan.c
cfe/trunk/test/Lexer/has_feature_efficiency_sanitizer.cpp
Modified:
cfe/trunk/include/clang/Basic/Sanitizers.def
cfe/trunk/include/clang/Driver/SanitizerArgs.h
cfe/trunk/lib/CodeGen/BackendUtil.cpp
cfe/trunk/lib/Driver/SanitizerArgs.cpp
cfe/trunk/lib/Driver/ToolChains.cpp
cfe/trunk/lib/Driver/Tools.cpp
cfe/trunk/lib/Lex/PPMacroExpansion.cpp
cfe/trunk/test/Driver/fsanitize.c
cfe/trunk/test/Driver/sanitize_unwind_tables.c
cfe/trunk/test/Driver/sanitizer-ld.c
Modified: cfe/trunk/include/clang/Basic/Sanitizers.def
URL:
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/Sanitizers.def?rev=267059&r1=267058&r2=267059&view=diff
==
--- cfe/trunk/include/clang/Basic/Sanitizers.def (original)
+++ cfe/trunk/include/clang/Basic/Sanitizers.def Thu Apr 21 16:32:04 2016
@@ -114,6 +114,11 @@ SANITIZER_GROUP("integer", Integer,
SANITIZER("local-bounds", LocalBounds)
SANITIZER_GROUP("bounds", Bounds, ArrayBounds | LocalBounds)
+// EfficiencySanitizer
+SANITIZER("efficiency-cache-frag", EfficiencyCacheFrag)
+// Meta-group only used internally.
+SANITIZER_GROUP("efficiency-all", Efficiency, EfficiencyCacheFrag)
+
// Magic group, containing all sanitizers. For example, "-fno-sanitize=all"
// can be used to disable all the sanitizers.
SANITIZER_GROUP("all", All, ~0ULL)
Modified: cfe/trunk/include/clang/Driver/SanitizerArgs.h
URL:
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/SanitizerArgs.h?rev=267059&r1=267058&r2=267059&view=diff
==
--- cfe/trunk/include/clang/Driver/SanitizerArgs.h (original)
+++ cfe/trunk/include/clang/Driver/SanitizerArgs.h Thu Apr 21 16:32:04 2016
@@ -58,6 +58,9 @@ class SanitizerArgs {
bool needsCfiRt() const;
bool needsCfiDiagRt() const;
bool needsStatsRt() const { return Stats; }
+ bool needsEsanRt() const {
+return Sanitizers.hasOneOf(SanitizerKind::Efficiency);
+ }
bool requiresPIE() const;
bool needsUnwindTables() const;
Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL:
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=267059&r1=267058&r2=267059&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Thu Apr 21 16:32:04 2016
@@ -248,6 +248,17 @@ static void addDataFlowSanitizerPass(con
PM.add(createDataFlowSanitizerPass(LangOpts.SanitizerBlacklistFiles));
}
+static void addEfficiencySanitizerPass(const PassManagerBuilder &Builder,
+ legacy::PassManagerBase &PM) {
+ const PassManagerBuilderWrapper &BuilderWrapper =
+ static_cast(Builder);
+ const LangOptions &LangOpts = BuilderWrapper.getLangOpts();
+ EfficiencySanitizerOptions Opts;
+ if (LangOpts.Sanitize.has(SanitizerKind::EfficiencyCacheFrag))
+Opts.ToolType = EfficiencySanitizerOptions::ESAN_CacheFrag;
+ PM.add(createEfficiencySanitizerPass(Opts));
+}
+
static TargetLibraryInfoImpl *createTLII(llvm::Triple &TargetTriple,
const CodeGenOptions &CodeGenOpts) {
TargetLibraryInfoImpl *TLII = new TargetLibraryInfoImpl(TargetTriple);
@@ -407,6 +418,13 @@ void EmitAssemblyHelper::CreatePasses(Mo
addDataFlowSanitizerPass);
}
+ if (LangOpts.Sanitize.hasOneOf(SanitizerKind::Efficiency)) {
+PMBuilder.addExtension(PassManagerBuilder::EP_OptimizerLast,
+ addEfficiencySanitizerPass);
+PMBuilder.addExtension(PassManagerBuilder::EP_EnabledOnOptLevel0,
+ addEfficiencySanitizerPass);
+ }
+
// Set up the per-function pass manager.
legacy::FunctionPassManager *FPM = getPerFunctionPasses();
if (CodeGenOpts.VerifyModule)
Modified: cfe/trunk/lib/Driver/SanitizerArgs.cpp
URL:
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/SanitizerArgs.cpp?rev=267059&r1=267058&r2=267059&view=diff
==
--- cfe/trunk/lib/Driver/SanitizerArgs.cpp (original)
+++ cfe/trunk/lib/Driver/SanitizerArgs.cpp Thu Apr 21 16:32:04 2016
@@ -311,7 +311,12 @@ SanitizerArgs::Saniti
r270641 - [esan|wset] Add working set tool driver flags
Author: bruening
Date: Tue May 24 19:41:24 2016
New Revision: 270641
URL: http://llvm.org/viewvc/llvm-project?rev=270641&view=rev
Log:
[esan|wset] Add working set tool driver flags
Summary:
Adds a new -fsanitize=efficiency-working-set flag to enable esan's working
set tool. Adds appropriate tests for the new flag.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits
Differential Revision: http://reviews.llvm.org/D20484
Modified:
cfe/trunk/include/clang/Basic/Sanitizers.def
cfe/trunk/lib/CodeGen/BackendUtil.cpp
cfe/trunk/test/Driver/esan.c
cfe/trunk/test/Driver/fsanitize.c
cfe/trunk/test/Driver/sanitize_unwind_tables.c
cfe/trunk/test/Driver/sanitizer-ld.c
cfe/trunk/test/Lexer/has_feature_efficiency_sanitizer.cpp
Modified: cfe/trunk/include/clang/Basic/Sanitizers.def
URL:
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/Sanitizers.def?rev=270641&r1=270640&r2=270641&view=diff
==
--- cfe/trunk/include/clang/Basic/Sanitizers.def (original)
+++ cfe/trunk/include/clang/Basic/Sanitizers.def Tue May 24 19:41:24 2016
@@ -116,8 +116,10 @@ SANITIZER_GROUP("bounds", Bounds, ArrayB
// EfficiencySanitizer
SANITIZER("efficiency-cache-frag", EfficiencyCacheFrag)
+SANITIZER("efficiency-working-set", EfficiencyWorkingSet)
// Meta-group only used internally.
-SANITIZER_GROUP("efficiency-all", Efficiency, EfficiencyCacheFrag)
+SANITIZER_GROUP("efficiency-all", Efficiency,
+EfficiencyCacheFrag | EfficiencyWorkingSet)
// Magic group, containing all sanitizers. For example, "-fno-sanitize=all"
// can be used to disable all the sanitizers.
Modified: cfe/trunk/lib/CodeGen/BackendUtil.cpp
URL:
http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/BackendUtil.cpp?rev=270641&r1=270640&r2=270641&view=diff
==
--- cfe/trunk/lib/CodeGen/BackendUtil.cpp (original)
+++ cfe/trunk/lib/CodeGen/BackendUtil.cpp Tue May 24 19:41:24 2016
@@ -258,6 +258,8 @@ static void addEfficiencySanitizerPass(c
EfficiencySanitizerOptions Opts;
if (LangOpts.Sanitize.has(SanitizerKind::EfficiencyCacheFrag))
Opts.ToolType = EfficiencySanitizerOptions::ESAN_CacheFrag;
+ else if (LangOpts.Sanitize.has(SanitizerKind::EfficiencyWorkingSet))
+Opts.ToolType = EfficiencySanitizerOptions::ESAN_WorkingSet;
PM.add(createEfficiencySanitizerPass(Opts));
}
Modified: cfe/trunk/test/Driver/esan.c
URL:
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/esan.c?rev=270641&r1=270640&r2=270641&view=diff
==
--- cfe/trunk/test/Driver/esan.c (original)
+++ cfe/trunk/test/Driver/esan.c Tue May 24 19:41:24 2016
@@ -2,8 +2,11 @@
// RUN: %clang -O1 -target x86_64-unknown-linux
-fsanitize=efficiency-cache-frag %s -S -emit-llvm -o - | FileCheck %s
// RUN: %clang -O2 -target x86_64-unknown-linux
-fsanitize=efficiency-cache-frag %s -S -emit-llvm -o - | FileCheck %s
// RUN: %clang -O3 -target x86_64-unknown-linux
-fsanitize=efficiency-cache-frag %s -S -emit-llvm -o - | FileCheck %s
-// RUN: %clang -target x86_64-unknown-linux
-fsanitize=efficiency-cache-frag %s -S -emit-llvm -o - | FileCheck %s
-// Verify that -fsanitize=efficiency-cache-frag invokes esan instrumentation.
+// RUN: %clang -target x86_64-unknown-linux
-fsanitize=efficiency-working-set %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang -O1 -target x86_64-unknown-linux
-fsanitize=efficiency-working-set %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang -O2 -target x86_64-unknown-linux
-fsanitize=efficiency-working-set %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang -O3 -target x86_64-unknown-linux
-fsanitize=efficiency-working-set %s -S -emit-llvm -o - | FileCheck %s
+// Verify that -fsanitize=efficiency-* invokes esan instrumentation.
int foo(int *a) { return *a; }
// CHECK: __esan_init
Modified: cfe/trunk/test/Driver/fsanitize.c
URL:
http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/fsanitize.c?rev=270641&r1=270640&r2=270641&view=diff
==
--- cfe/trunk/test/Driver/fsanitize.c (original)
+++ cfe/trunk/test/Driver/fsanitize.c Tue May 24 19:41:24 2016
@@ -84,19 +84,24 @@
// CHECK-SANKA-SANL: '-fsanitize=kernel-address' not allowed with
'-fsanitize=leak'
// RUN: %clang -target x86_64-linux-gnu
-fsanitize=efficiency-cache-frag,address -pie -fno-rtti %s -### 2>&1 |
FileCheck %s --check-prefix=CHECK-SANE-SANA
-// CHECK-SANE-SANA: '-fsanitize=efficiency-cache-frag' not allowed with
'-fsanitize=address'
+// RUN: %clang -target x86_64-linux-gnu
-fsanitize=efficiency-working-set,address -pie -fno-rtti %s -### 2>&1 |
FileCheck %s --check-prefix=CHECK-SANE-SANA
+// CHECK-SANE-SANA: '-fsanitize=efficiency-{{.*}}' not allowed with
'-fsanitize=address'
Re: [PATCH] D21753: Comprehensive Static Instrumentation (2/2): Clang flag
bruening added inline comments. Comment at: docs/CSI.rst:75 @@ +74,3 @@ + % clang++ -c -O3 -g -fcsi -emit-llvm bar.cpp -o bar.o + % clang++ foo.o bar.o my-tool.o libclang_rt.csi-x86_64.a -fuse-ld=gold -flto -lrt -ldl -o foo + See below: the sanitizers pass the -f flag (-fcsi here) to the link line and have the library automatically linked in, which is a simpler usage model than the user having to name this static library explicitly. Comment at: docs/CSI.rst:78 @@ +77,3 @@ +Notice that in the final stage of linking, the tool user also needs to link in +the static library of the CSI runtime to produce the final TIX. The runtime +archive is distributed under the ``build/lib/clang//lib/`` This should not be necessary: as mentioned above, if -fcsi is passed to the link line you should be able to have clang automatically add the static csi library, just like is done for the sanitizers. Comment at: docs/CSI.rst:81 @@ +80,3 @@ +directory. We plan to investigate means of linking with the runtime +automatically in the future, but for the time being, the tool user should link +it in explicitly. Hmm, see above comments: is this already implemented and was deliberately split from this one for simplicity? Comment at: docs/CSI.rst:97 @@ +96,3 @@ +* functions, +* function exits, +* basic blocks, Wouldn't the after-hook here be the same as the after-hook for the function category? Generally the reason to have a post-function or function-exit hook would be to view or change the return value: couldn't that be done equally easily from a post-function hook at the instruction after the call site? I guess I'm asking why this is a separate category. Comment at: docs/CSI.rst:99 @@ +98,3 @@ +* basic blocks, +* call sites, +* loads, and It seems like there is some redundancy here? This seems very similar to the "functions" category: I'm curious as to why they are separate? Comment at: docs/CSI.rst:164 @@ +163,3 @@ +library loads. + + Some tools also need thread-local handling: do you plan to provide thread initialization and exit hooks in the future? Comment at: docs/CSI.rst:164 @@ +163,3 @@ +library loads. + + bruening wrote: > Some tools also need thread-local handling: do you plan to provide thread > initialization and exit hooks in the future? What about a fini or destructor function called at program exit? Many profiling or analysis tools gather data and want to report it or dump it to a file at program exit. Comment at: docs/CSI.rst:173 @@ +172,3 @@ + + void __csi_func_entry(const csi_id_t func_id); + void __csi_func_exit(const csi_id_t func_exit_id, const csi_id_t func_id); Generally, tools that hook application functions want to examine the arguments. How does a hook access (or modify) the application function's arguments? Comment at: docs/CSI.rst:174 @@ +173,3 @@ + void __csi_func_entry(const csi_id_t func_id); + void __csi_func_exit(const csi_id_t func_exit_id, const csi_id_t func_id); + Similarly, how does a hook access or change the return value? Comment at: docs/CSI.rst:181 @@ +180,3 @@ +hook ``__csi_func_exit`` is invoked just before the function returns +normally). (We have not yet defined the API for exceptions.) +The ``func_exit_id`` parameter allows the tool writer to distinguish the s/normally)/normally/ Comment at: docs/CSI.rst:189 @@ +188,3 @@ + +CSI also provide instrumentation hooks basic block entry and exit. +A basic block consists of strands of instructions with no incoming branches Grammar: provides, for Comment at: docs/CSI.rst:212 @@ +211,3 @@ + +CSI provides the following hooks for call sites: + See above: I'm not sure why both call sites and function entry hooks are needed? Perhaps there could be some explanation of that here. Comment at: docs/CSI.rst:221 @@ +220,3 @@ +parameter identifies the called function. Note that it may not always be +possible to CSI to produce the function ID corresponds to the called function +statically --- for example, if a function is called indirectly Grammar: s/to CSI/for CSI/; s/ID/ID that/ Comment at: docs/CSI.rst:258 @@ +257,3 @@ +plan to extend the CSI to include more property values and incorporate property +into other types of hooks. + Hmm, there seems to be a missing feature in this interface design in general: static analysis or static operation of some kind. Tools often want to take one action if a memory address is aligned, but a different one if it's not aligned (usually a fastpath when aligned and a slowpath when unaligned). The compiler ofte
Re: [PATCH] D21753: Comprehensive Static Instrumentation (2/2): Clang flag
bruening added inline comments. Comment at: docs/CSI.rst:78 @@ +77,3 @@ +Notice that in the final stage of linking, the tool user also needs to link in +the static library of the CSI runtime to produce the final TIX. The runtime +archive is distributed under the ``build/lib/clang//lib/`` mehdi_amini wrote: > bruening wrote: > > This should not be necessary: as mentioned above, if -fcsi is passed to the > > link line you should be able to have clang automatically add the static csi > > library, just like is done for the sanitizers. > This is not clear to me: the sanitizers are auto-linking the clang supplied > runtime. Here it seems to be about a user-supplied library. No, the CSI runtime is not the user-supplied part: it is part of the clang build, just like the sanitizer runtime libraries (see line 79 below showing where it lives). The user-supplied part is "my-tool.o". Repository: rL LLVM http://reviews.llvm.org/D21753 ___ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
Re: [PATCH] D21753: Comprehensive Static Instrumentation (2/2): Clang flag
bruening added inline comments. Comment at: docs/CSI.rst:30 @@ +29,3 @@ +To create a CSI tool, add ``#include `` at the top of the tool source +and implement function bodies for the hooks relevant to the tool. + Are there any constraints on what libraries the tool library is allowed to use? Generally there are, for tool code that runs in the same process as the application. The tool library will be operating at arbitrary points during application execution. This means that it should avoid using the same resources as the instrumented application, because the application's routines are not all re-entrant and they use global state, and to minimize perturbation of the application's behavior (such as heap layout patterns) from how it behaves with no tool present. A tool using standard libraries becomes more likely to cause issues when libc routines are being intercepted by the tool (see related comment below) or libc itself is instrumented. The existing LLVM instrumentation runtime libraries, for the sanitizer tools, avoid calling libc routines and are not able to use the STL: they use their own custom implementations of all data structures and algorithms that they need, but this is a small set. Dynamic tool platforms like Pin and DynamoRIO go to great lengths to isolate tool libraries by loading separate copies of libc. Has any thought been put into isolating the tool library and its imports from the application? I realize that some of these may seem more long-term topics, but if the idea is to create a framework for use with a wide range of tools it is good to consider all issues up front. Comment at: docs/CSI.rst:230 @@ +229,3 @@ + +CSI provides the following hooks for memory operations: + For observing loads and stores, typically compiler-based tools intercept libc's memcpy, memset, etc. (or in some cases libc is built and instrumented along with the application), to avoid missing many memory references. The existing LLVM sanitizer tools all intercept a large number of libc routines to ensure they see more than just events happening in application code proper. Has there been any thought about this for CSI? Repository: rL LLVM https://reviews.llvm.org/D21753 ___ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
