vsk created this revision.
vsk added a reviewer: bogner.
vsk added subscribers: kcc, cfe-commits, silvas.
It would be helpful to have a user-friendly guide for code coverage. There is
some overlap with [1], but this document visits issues which may affect users
in more depth.
Prompted by: https://llvm.org/bugs/show_bug.cgi?id=27781
[1] http://llvm.org/docs/CoverageMappingFormat.html
http://reviews.llvm.org/D20715
Files:
docs/SourceBasedCodeCoverage.rst
Index: docs/SourceBasedCodeCoverage.rst
===================================================================
--- /dev/null
+++ docs/SourceBasedCodeCoverage.rst
@@ -0,0 +1,174 @@
+==========================
+Source-based Code Coverage
+==========================
+
+.. contents::
+ :local:
+
+Introduction
+============
+
+This document explains how to use clang's source-based code coverage feature.
+It's called "source-based" because it operates on AST and preprocessor
+information directly. This allows it to generate very precise coverage data.
+
+Clang ships two other code coverage implementations:
+
+* :doc:`SanitizerCoverage` - A low-overhead tool meant for use alongside the
+ various sanitizers. It can provide up to edge-level coverage.
+
+* gcov - A GCC-compatible coverage implementation which operates on DebugInfo.
+
+From this point onwards "code coverage" will refer to the source-based kind.
+
+The code coverage workflow
+==========================
+
+The code coverage workflow consists of three main steps:
+
+1. Compile with coverage enabled.
+
+2. Run the program.
+
+3. Create a report out of the generated profile.
+
+The next few sections work through a complete, copy-'n-paste friendly example
+based on this program:
+
+.. code-block:: console
+
+ % cat <<EOF > foo.cc
+ #define BAR(x) ((x) || (x))
+ template <typename T> void foo(T x) {
+ for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+ }
+ int main() {
+ foo<int>(0);
+ foo<float>(0);
+ return 0;
+ }
+ EOF
+
+Compiling with coverage enabled
+===============================
+
+To compile code with coverage enabled pass ``-fprofile-instr-generate
+-fcoverage-mapping`` to the compiler:
+
+.. code-block:: console
+
+ # Step 1: Compile with coverage enabled.
+ % cc -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
+
+Note that linking together code with and without coverage instrumentation is
+supported: any uninstrumented code simply won't be accounted for.
+
+Running the instrumented program
+================================
+
+The next step is to run the instrumented program. When the program exits it
+will write a **raw profile** to the path specified by the ``LLVM_PROFILE_FILE``
+environment variable. If that variable does not exist the profile is written to
+``./default.profraw``.
+
+If ``LLVM_PROFILE_FILE`` contains a path to a non-existent directory the
+missing directory structure will be created. Additionally, the following
+special **pattern strings** are replaced:
+
+* "%p" expands out to the PID.
+
+* "%h" expands out to the hostname of the machine running the program.
+
+.. code-block:: console
+
+ # Step 2: Run the program.
+ % LLVM_PROFILE_FILE="foo.profraw" ./foo
+
+Creating coverage reports
+=========================
+
+Raw profiles have to be **indexed** before they can be used to generated
+coverage reports:
+
+.. code-block:: console
+
+ # Step 3(a): Index the raw profile.
+ % llvm-profdata merge -sparse foo.profraw -o foo.profdata
+
+There are multiple different ways to render coverage reports. One option is to
+generate a line-oriented report:
+
+.. code-block:: console
+
+ # Step 3(b): Create a line-oriented coverage report.
+ % llvm-cov show ./foo -instr-profile=foo.profdata
+
+This report includes a summary view as well as dedicated sub-views for
+templated functions and their instantiations. For our example program, we get
+distinct views for ``foo<int>(...)`` and ``foo<float>(...)``. If
+``-show-line-counts-or-regions`` is enabled, ``llvm-cov`` displays nested
+region counts (even in macro expansions):
+
+.. code-block:: console
+
+ 20| 1|#define BAR(x) ((x) || (x))
+ ^20 ^2
+ 2| 2|template <typename T> void foo(T x) {
+ 22| 3| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+ ^22 ^20 ^20^20
+ 1| 4|}
+ ------------------
+ | _Z3fooIiEvT_:
+ | 1| 2|template <typename T> void foo(T x) {
+ | 11| 3| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+ | ^11 ^10 ^10^10
+ | 1| 4|}
+ ------------------
+ | _Z3fooIfEvT_:
+ | 1| 2|template <typename T> void foo(T x) {
+ | 11| 3| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+ | ^11 ^10 ^10^10
+ | 1| 4|}
+ ------------------
+
+It's possible to generate a file-level summary of coverage statistics (instead
+of a line-oriented report) with:
+
+.. code-block:: console
+
+ # Step 3(c): Create a coverage summary.
+ % llvm-cov report ./foo -instr-profile=foo.profdata
+
+A few final notes:
+
+* The ``-sparse`` flag is optional but can result in dramatically smaller
+ indexed profiles. This option should not be used if the indexed profile will
+ be reused for PGO.
+
+* Raw profiles can be discarded after they are indexed. It **is** possible for
+ instrumented programs to merge profiling information directly into an existing
+ raw profile on disk. The details are out of this document's scope.
+
+* The ``llvm-profdata`` tool can be used to merge together multiple raw or
+ indexed profiles. To combine profiling data from multiple runs of a program,
+ try e.g:
+
+.. code-block:: console
+
+ % llvm-profdata merge -sparse sort1.profraw sort2.profdata -o sort.profdata
+
+Format compatibility guarantees
+===============================
+
+* There are no backwards or forwards compatibility guarantees whatsoever for
+ the raw profile format. Raw profiles may be dependent on the specific
+ compiler revision used to generate them. It's inadvisable to store raw
+ profiles for long periods of time.
+
+* Tools must retain backwards compatibility with indexed profile formats. These
+ formats are not forwards-compatible: i.e, a tool which uses format version X
+ will not be able to understand format version (X+k).
+
+* There is a third format in play: the format of the coverage mappings emitted
+ into instrumented binaries. Tools must retain backwards compatibility with
+ these formats. These formats are not forwards-compatible.
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
