snehasish updated this revision to Diff 503850.
snehasish added a comment.
Update the script for Transforms/PGOProfile/memprof.ll.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D145644/new/
https://reviews.llvm.org/D145644
Files:
clang/test/CodeGen/Inputs/memprof.exe
clang/test/CodeGen/Inputs/memprof.memprofraw
clang/test/CodeGen/Inputs/update_memprof_inputs.sh
clang/test/CodeGen/memprof.cpp
llvm/test/Transforms/PGOProfile/Inputs/memprof.exe
llvm/test/Transforms/PGOProfile/Inputs/memprof.memprofraw
llvm/test/Transforms/PGOProfile/Inputs/memprof_pgo.profraw
llvm/test/Transforms/PGOProfile/Inputs/update_memprof_inputs.sh
llvm/test/Transforms/PGOProfile/memprof.ll
llvm/test/tools/llvm-profdata/Inputs/basic.memprofexe
llvm/test/tools/llvm-profdata/Inputs/basic.memprofraw
llvm/test/tools/llvm-profdata/Inputs/inline.memprofexe
llvm/test/tools/llvm-profdata/Inputs/inline.memprofraw
llvm/test/tools/llvm-profdata/Inputs/memprof-inline.exe
llvm/test/tools/llvm-profdata/Inputs/multi.memprofexe
llvm/test/tools/llvm-profdata/Inputs/multi.memprofraw
llvm/test/tools/llvm-profdata/Inputs/pic.memprofexe
llvm/test/tools/llvm-profdata/Inputs/pic.memprofraw
llvm/test/tools/llvm-profdata/Inputs/update_memprof_inputs.sh
llvm/test/tools/llvm-profdata/memprof-basic.test
llvm/test/tools/llvm-profdata/memprof-inline.test
llvm/test/tools/llvm-profdata/memprof-merge.test
llvm/test/tools/llvm-profdata/memprof-multi.test
llvm/test/tools/llvm-profdata/memprof-pic.test
Index: llvm/test/tools/llvm-profdata/memprof-pic.test
===================================================================
--- llvm/test/tools/llvm-profdata/memprof-pic.test
+++ llvm/test/tools/llvm-profdata/memprof-pic.test
@@ -1,40 +1,12 @@
REQUIRES: x86_64-linux
-This test ensures that llvm-profdata fails with a descriptive error message
-when invoked on a memprof profiled binary which was built with position
-independent code.
-
-The input raw profile test has been generated from the following source code:
-
-```
-#include <stdlib.h>
-#include <string.h>
-int main(int argc, char **argv) {
- char *x = (char *)malloc(10);
- memset(x, 0, 10);
- free(x);
- x = (char *)malloc(10);
- memset(x, 0, 10);
- free(x);
- return 0;
-}
-```
-
-The following commands were used to compile the source to a memprof instrumented
-executable and collect a raw binary format profile. Since the profile contains
-virtual addresses for the callstack, we do not expect the raw binary profile to
-be deterministic. The summary should be deterministic apart from changes to
-the shared libraries linked in which could change the number of segments
+Since the profile contains virtual addresses for the callstack,
+we do not expect the raw binary profile to be deterministic. The
+summary should be deterministic apart from changes to the shared
+libraries linked in which could change the number of segments
recorded.
-```
-clang -fuse-ld=lld -Wl,--no-rosegment -gmlt -fdebug-info-for-profiling \
- -fmemory-profile -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer \
- -fno-optimize-sibling-calls -m64 -Wl,-build-id -pie \
- source.c -o pic.memprofexe
-
-env MEMPROF_OPTIONS=log_path=stdout ./pic.memprofexe > pic.memprofraw
-```
-
+To update the inputs used below run Inputs/update_memprof_inputs.sh /path/to/updated/clang
RUN: not llvm-profdata show --memory %p/Inputs/pic.memprofraw --profiled-binary %p/Inputs/pic.memprofexe -o - 2>&1 | FileCheck %s
+
CHECK: Unsupported position independent code
Index: llvm/test/tools/llvm-profdata/memprof-multi.test
===================================================================
--- llvm/test/tools/llvm-profdata/memprof-multi.test
+++ llvm/test/tools/llvm-profdata/memprof-multi.test
@@ -1,39 +1,6 @@
REQUIRES: x86_64-linux
-The input raw profile test has been generated from the following source code:
-
-```
-#include <sanitizer/memprof_interface.h>
-#include <stdlib.h>
-#include <string.h>
-int main(int argc, char **argv) {
- char *x = (char *)malloc(10);
- memset(x, 0, 10);
- free(x);
- __memprof_profile_dump();
- x = (char *)malloc(10);
- memset(x, 0, 10);
- free(x);
- return 0;
-}
-```
-
-The following commands were used to compile the source to a memprof instrumented
-executable and collect a raw binary format profile. Since the profile contains
-virtual addresses for the callstack, we do not expect the raw binary profile to
-be deterministic. The summary should be deterministic apart from changes to
-the shared libraries linked in which could change the number of segments
-recorded.
-
-```
-clang -fuse-ld=lld -Wl,--no-rosegment -gmlt -fdebug-info-for-profiling \
- -fmemory-profile -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer \
- -fno-optimize-sibling-calls -m64 -Wl,-build-id -no-pie \
- source.c -o multi.memprofexe
-
-env MEMPROF_OPTIONS=log_path=stdout ./multi.memprofexe > multi.memprofraw
-```
-
+To update the inputs used below run Inputs/update_memprof_inputs.sh /path/to/updated/clang
RUN: llvm-profdata show --memory %p/Inputs/multi.memprofraw --profiled-binary %p/Inputs/multi.memprofexe -o - | FileCheck %s
We expect 2 MIB entries, 1 each for the malloc calls in the program.
Index: llvm/test/tools/llvm-profdata/memprof-merge.test
===================================================================
--- llvm/test/tools/llvm-profdata/memprof-merge.test
+++ llvm/test/tools/llvm-profdata/memprof-merge.test
@@ -1,40 +1,13 @@
REQUIRES: x86_64-linux
-The input memprof and instrumented raw profiles were generated from the following source code:
-
-```
-#include <stdlib.h>
-#include <string.h>
-int main(int argc, char **argv) {
- char *x = (char *)malloc(10);
- memset(x, 0, 10);
- free(x);
- x = (char *)malloc(10);
- memset(x, 0, 10);
- free(x);
- return 0;
-}
-```
-
-Steps to collect the memprof raw profile and the instrprof raw profile:
-
-```
-# Collect instrprof profile with name compression disabled since some buildbots
-# do not have zlib.
-clang -mllvm -enable-name-compression=false -fprofile-generate source.c -o instr.out
-./instr.out
-mv *.profraw basic.profraw
-
-# Collect memprof profile.
-clang -fuse-ld=lld -Wl,--no-rosegment -gmlt -fdebug-info-for-profiling \
- -fmemory-profile -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer \
- -fno-optimize-sibling-calls -m64 -Wl,-build-id -no-pie \
- source.c -o basic.memprofexe
-
-env MEMPROF_OPTIONS=log_path=stdout ./basic.memprofexe > basic.memprofraw
-```
-
-RUN: llvm-profdata merge %p/Inputs/basic.profraw %p/Inputs/basic.memprofraw --profiled-binary %p/Inputs/basic.memprofexe -o %t.prof
+RUN: echo ":ir" > %t.proftext
+RUN: echo "main" >> %t.proftext
+RUN: echo "742261418966908927" >> %t.proftext
+RUN: echo "1" >> %t.proftext
+RUN: echo "1" >> %t.proftext
+
+To update the inputs used below run Inputs/update_memprof_inputs.sh /path/to/updated/clang
+RUN: llvm-profdata merge %t.proftext %p/Inputs/basic.memprofraw --profiled-binary %p/Inputs/basic.memprofexe -o %t.prof
RUN: llvm-profdata show %t.prof | FileCheck %s
For now we only check the validity of the instrumented profile since we don't
Index: llvm/test/tools/llvm-profdata/memprof-inline.test
===================================================================
--- llvm/test/tools/llvm-profdata/memprof-inline.test
+++ llvm/test/tools/llvm-profdata/memprof-inline.test
@@ -1,41 +1,6 @@
REQUIRES: x86_64-linux
-The input raw profile test has been generated from the following source code:
-
-```
-#include <stdlib.h>
-#include <string.h>
-
-__attribute__((always_inline))
-void qux(int x) {
- char *ptr = malloc(x);
- memset(ptr, 0, x);
- free(ptr);
-}
-
-__attribute__((noinline))
-void foo(int x){ qux(x); }
-
-__attribute__((noinline))
-void bar(int x) { foo(x); }
-
-int main(int argc, char **argv) {
- bar(argc);
- return 0;
-}
-```
-
-Compile and run with the following commands:
-
-```
-bin/clang -fuse-ld=lld -Wl,--no-rosegment -gmlt -fdebug-info-for-profiling \
- -fmemory-profile -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer \
- -fno-optimize-sibling-calls -m64 -Wl,-build-id -no-pie \
- inline.c -o inline.memprofexe
-
-env MEMPROF_OPTIONS=log_path=stdout ./inline.memprofexe > inline.memprofraw
-```
-
+To update the inputs used below run Inputs/update_memprof_inputs.sh /path/to/updated/clang
RUN: llvm-profdata show --memory %p/Inputs/inline.memprofraw --profiled-binary %p/Inputs/inline.memprofexe | FileCheck %s
CHECK: MemprofProfile:
Index: llvm/test/tools/llvm-profdata/memprof-basic.test
===================================================================
--- llvm/test/tools/llvm-profdata/memprof-basic.test
+++ llvm/test/tools/llvm-profdata/memprof-basic.test
@@ -1,37 +1,6 @@
REQUIRES: x86_64-linux
-The input raw profile test has been generated from the following source code:
-
-```
-#include <stdlib.h>
-#include <string.h>
-int main(int argc, char **argv) {
- char *x = (char *)malloc(10);
- memset(x, 0, 10);
- free(x);
- x = (char *)malloc(10);
- memset(x, 0, 10);
- free(x);
- return 0;
-}
-```
-
-The following commands were used to compile the source to a memprof instrumented
-executable and collect a raw binary format profile. Since the profile contains
-virtual addresses for the callstack, we do not expect the raw binary profile to
-be deterministic. The summary should be deterministic apart from changes to
-the shared libraries linked in which could change the number of segments
-recorded.
-
-```
-clang -fuse-ld=lld -Wl,--no-rosegment -gmlt -fdebug-info-for-profiling \
- -fmemory-profile -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer \
- -fno-optimize-sibling-calls -m64 -Wl,-build-id -no-pie \
- source.c -o basic.memprofexe
-
-env MEMPROF_OPTIONS=log_path=stdout ./basic.memprofexe > basic.memprofraw
-```
-
+To update the inputs used below run Inputs/update_memprof_inputs.sh /path/to/updated/clang
RUN: llvm-profdata show --memory %p/Inputs/basic.memprofraw --profiled-binary %p/Inputs/basic.memprofexe -o - | FileCheck %s
We expect 2 MIB entries, 1 each for the malloc calls in the program. Any
Index: llvm/test/tools/llvm-profdata/Inputs/update_memprof_inputs.sh
===================================================================
--- /dev/null
+++ llvm/test/tools/llvm-profdata/Inputs/update_memprof_inputs.sh
@@ -0,0 +1,82 @@
+#!/bin/bash
+
+if [ -z $1 ]; then
+ echo "Path to clang required!"
+ echo "Usage: update_memprof_inputs.sh /path/to/updated/clang"
+ exit 1
+else
+ CLANG=$1
+fi
+
+# Allows the script to be invoked from other directories.
+OUTDIR=$(dirname $(realpath -s $0))
+
+read -r -d '' SRC1 << EOF
+#include <stdlib.h>
+#include <string.h>
+int main(int argc, char **argv) {
+ char *x = (char *)malloc(10);
+ memset(x, 0, 10);
+ free(x);
+ x = (char *)malloc(10);
+ memset(x, 0, 10);
+ free(x);
+ return 0;
+}
+EOF
+
+read -r -d '' SRC2 << EOF
+#include <stdlib.h>
+#include <string.h>
+
+__attribute__((always_inline))
+void qux(int x) {
+ char *ptr = malloc(x);
+ memset(ptr, 0, x);
+ free(ptr);
+}
+
+__attribute__((noinline))
+void foo(int x){ qux(x); }
+
+__attribute__((noinline))
+void bar(int x) { foo(x); }
+
+int main(int argc, char **argv) {
+ bar(argc);
+ return 0;
+}
+EOF
+
+read -r -d '' SRC3 << EOF
+#include <sanitizer/memprof_interface.h>
+#include <stdlib.h>
+#include <string.h>
+int main(int argc, char **argv) {
+ char *x = (char *)malloc(10);
+ memset(x, 0, 10);
+ free(x);
+ __memprof_profile_dump();
+ x = (char *)malloc(10);
+ memset(x, 0, 10);
+ free(x);
+ return 0;
+}
+EOF
+
+DEFAULT_MEMPROF_FLAGS="-fuse-ld=lld -Wl,--no-rosegment -gmlt -fdebug-info-for-profiling -fmemory-profile -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -m64 -Wl,-build-id -no-pie"
+
+# Map each test to their source and any additional flags separated by ;
+declare -A INPUTS
+INPUTS["basic"]="SRC1"
+INPUTS["inline"]="SRC2"
+INPUTS["multi"]="SRC3"
+INPUTS["pic"]="SRC1;-pie"
+
+for name in "${!INPUTS[@]}"; do
+ IFS=";" read -r src flags <<< "${INPUTS[$name]}"
+ echo "${!src}" > ${OUTDIR}/${name}.c
+ ${CLANG} ${DEFAULT_MEMPROF_FLAGS} ${flags} ${OUTDIR}/${name}.c -o ${OUTDIR}/${name}.memprofexe
+ env MEMPROF_OPTIONS=log_path=stdout ${OUTDIR}/${name}.memprofexe > ${OUTDIR}/${name}.memprofraw
+ rm ${OUTDIR}/${name}.c
+done
Index: llvm/test/Transforms/PGOProfile/memprof.ll
===================================================================
--- llvm/test/Transforms/PGOProfile/memprof.ll
+++ llvm/test/Transforms/PGOProfile/memprof.ll
@@ -7,87 +7,8 @@
; REQUIRES: x86_64-linux
;; TODO: Use text profile inputs once that is available for memprof.
-
-;; The input IR and raw profiles have been generated from the following source:
-;;
-;; #include <stdlib.h>
-;; #include <string.h>
-;; #include <unistd.h>
-;; char *foo() {
-;; return new char[10];
-;; }
-;; char *foo2() {
-;; return foo();
-;; }
-;; char *bar() {
-;; return foo2();
-;; }
-;; char *baz() {
-;; return foo2();
-;; }
-;; char *recurse(unsigned n) {
-;; if (!n)
-;; return foo();
-;; return recurse(n-1);
-;; }
-;; int main(int argc, char **argv) {
-;; // Test allocations with different combinations of stack contexts and
-;; // coldness (based on lifetime, since they are all accessed a single time
-;; // per byte via the memset).
-;; char *a = new char[10];
-;; char *b = new char[10];
-;; char *c = foo();
-;; char *d = foo();
-;; char *e = bar();
-;; char *f = baz();
-;; memset(a, 0, 10);
-;; memset(b, 0, 10);
-;; memset(c, 0, 10);
-;; memset(d, 0, 10);
-;; memset(e, 0, 10);
-;; memset(f, 0, 10);
-;; // a and c have short lifetimes
-;; delete[] a;
-;; delete[] c;
-;; // b, d, e, and f have long lifetimes and will be detected as cold by default.
-;; sleep(200);
-;; delete[] b;
-;; delete[] d;
-;; delete[] e;
-;; delete[] f;
-;; // Loop ensures the two calls to recurse have stack contexts that only differ
-;; // in one level of recursion. We should get two stack contexts reflecting the
-;; // different levels of recursion and different allocation behavior (since the
-;; // first has a very long lifetime and the second has a short lifetime).
-;; for (unsigned i = 0; i < 2; i++) {
-;; char *g = recurse(i + 3);
-;; memset(g, 0, 10);
-;; if (!i)
-;; sleep(200);
-;; delete[] g;
-;; }
-;; return 0;
-;; }
-;;
-;; The following commands were used to compile the source to instrumented
-;; executables and collect raw binary format profiles:
-;;
-;; # Collect memory profile:
-;; $ clang++ -fuse-ld=lld -no-pie -Wl,--no-rosegment -gmlt \
-;; -fdebug-info-for-profiling -mno-omit-leaf-frame-pointer \
-;; -fno-omit-frame-pointer -fno-optimize-sibling-calls -m64 -Wl,-build-id \
-;; memprof.cc -o memprof.exe -fmemory-profile
-;; $ env MEMPROF_OPTIONS=log_path=stdout ./memprof.exe > memprof.memprofraw
-;;
-;; # Collect IR PGO profile:
-;; $ clang++ -fuse-ld=lld -no-pie -Wl,--no-rosegment -gmlt \
-;; -fdebug-info-for-profiling -mno-omit-leaf-frame-pointer \
-;; -fno-omit-frame-pointer -fno-optimize-sibling-calls -m64 -Wl,-build-id \
-;; memprof.cc -o pgo.exe -fprofile-generate=.
-;; $ ./pgo.exe
-;; $ mv default_*.profraw memprof_pgo.profraw
-;;
-;; # Generate below LLVM IR for use in matching:
+;; # To update the Inputs below, run Inputs/update_memprof_inputs.sh.
+;; # To generate below LLVM IR for use in matching:
;; $ clang++ -gmlt -fdebug-info-for-profiling -fno-omit-frame-pointer \
;; -fno-optimize-sibling-calls memprof.cc -S -emit-llvm
Index: llvm/test/Transforms/PGOProfile/Inputs/update_memprof_inputs.sh
===================================================================
--- /dev/null
+++ llvm/test/Transforms/PGOProfile/Inputs/update_memprof_inputs.sh
@@ -0,0 +1,89 @@
+#!/bin/bash
+
+if [ -z $1 ]; then
+ echo "Path to clang required!"
+ echo "Usage: update_memprof_inputs.sh /path/to/updated/clang"
+ exit 1
+else
+ CLANG=$1
+fi
+
+# Allows the script to be invoked from other directories.
+OUTDIR=$(dirname $(realpath -s $0))
+
+# Note that changes in the code below which affect relative line number
+# offsets of calls from their parent function can affect callsite matching in
+# the LLVM IR.
+cat > ${OUTDIR}/memprof.cc << EOF
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+char *foo() {
+ return new char[10];
+}
+char *foo2() {
+ return foo();
+}
+char *bar() {
+ return foo2();
+}
+char *baz() {
+ return foo2();
+}
+char *recurse(unsigned n) {
+ if (!n)
+ return foo();
+ return recurse(n-1);
+}
+int main(int argc, char **argv) {
+ // Test allocations with different combinations of stack contexts and
+ // coldness (based on lifetime, since they are all accessed a single time
+ // per byte via the memset).
+ char *a = new char[10];
+ char *b = new char[10];
+ char *c = foo();
+ char *d = foo();
+ char *e = bar();
+ char *f = baz();
+ memset(a, 0, 10);
+ memset(b, 0, 10);
+ memset(c, 0, 10);
+ memset(d, 0, 10);
+ memset(e, 0, 10);
+ memset(f, 0, 10);
+ // a and c have short lifetimes
+ delete[] a;
+ delete[] c;
+ // b, d, e, and f have long lifetimes and will be detected as cold by default.
+ sleep(200);
+ delete[] b;
+ delete[] d;
+ delete[] e;
+ delete[] f;
+
+ // Loop ensures the two calls to recurse have stack contexts that only differ
+ // in one level of recursion. We should get two stack contexts reflecting the
+ // different levels of recursion and different allocation behavior (since the
+ // first has a very long lifetime and the second has a short lifetime).
+ for (unsigned i = 0; i < 2; i++) {
+ char *g = recurse(i + 3);
+ memset(g, 0, 10);
+ if (!i)
+ sleep(200);
+ delete[] g;
+ }
+ return 0;
+}
+EOF
+
+COMMON_FLAGS="-fuse-ld=lld -Wl,--no-rosegment -gmlt -fdebug-info-for-profiling -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -m64 -Wl,-build-id -no-pie"
+
+${CLANG} ${COMMON_FLAGS} -fmemory-profile ${OUTDIR}/memprof.cc -o ${OUTDIR}/memprof.exe
+env MEMPROF_OPTIONS=log_path=stdout ${OUTDIR}/memprof.exe > ${OUTDIR}/memprof.memprofraw
+
+${CLANG} ${COMMON_FLAGS} -fprofile-generate=. \
+ ${OUTDIR}/memprof.cc -o ${OUTDIR}/pgo.exe
+env LLVM_PROFILE_FILE=${OUTDIR}/memprof_pgo.profraw ${OUTDIR}/pgo.exe
+
+rm ${OUTDIR}/memprof.cc
+rm ${OUTDIR}/pgo.exe
Index: clang/test/CodeGen/memprof.cpp
===================================================================
--- clang/test/CodeGen/memprof.cpp
+++ clang/test/CodeGen/memprof.cpp
@@ -11,16 +11,7 @@
// TODO: Use text profile inputs once that is available for memprof.
//
-// The following commands were used to compile the source to instrumented
-// executables and collect raw binary format profiles:
-//
-// # Collect memory profile:
-// $ clang++ -fuse-ld=lld -no-pie -Wl,--no-rosegment -gmlt \
-// -fdebug-info-for-profiling -mno-omit-leaf-frame-pointer \
-// -fno-omit-frame-pointer -fno-optimize-sibling-calls -m64 -Wl,-build-id \
-// memprof.cpp -o memprof.exe -fmemory-profile
-// $ env MEMPROF_OPTIONS=log_path=stdout ./memprof.exe > memprof.memprofraw
-//
+// To update the inputs below, run Inputs/update_memprof_inputs.sh
// RUN: llvm-profdata merge %S/Inputs/memprof.memprofraw --profiled-binary %S/Inputs/memprof.exe -o %t.memprofdata
// Profile use:
Index: clang/test/CodeGen/Inputs/update_memprof_inputs.sh
===================================================================
--- /dev/null
+++ clang/test/CodeGen/Inputs/update_memprof_inputs.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+if [ -z $1 ]; then
+ echo "Path to clang required!"
+ echo "Usage: update_memprof_inputs.sh /path/to/updated/clang"
+ exit 1
+else
+ CLANG=$1
+fi
+
+# Allows the script to be invoked from other directories.
+OUTDIR=$(dirname $(realpath -s $0))
+
+DEFAULT_MEMPROF_FLAGS="-fuse-ld=lld -Wl,--no-rosegment -gmlt -fdebug-info-for-profiling -fmemory-profile -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -m64 -Wl,-build-id -no-pie"
+
+${CLANG} ${DEFAULT_MEMPROF_FLAGS} ${OUTDIR}/../memprof.cpp -o ${OUTDIR}/memprof.exe
+env MEMPROF_OPTIONS=log_path=stdout ${OUTDIR}/memprof.exe > ${OUTDIR}/memprof.memprofraw
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
