This is a fluke. The bisection is searching for an 8% regression in 464.h264ref and it tripped just above 2% threshold on this patch due to benchmarking noise.
-- Maxim Kuvyrkov https://www.linaro.org > On 27 Sep 2021, at 14:42, ci_not...@linaro.org wrote: > > After llvm commit e8e2edd8ca88f8b0a7dba141349b2aa83284f3af > Author: Erich Keane <erich.ke...@intel.com> > > Fix test from 8dd42f, capitalization in test > > the following benchmarks slowed down by more than 2%: > - 464.h264ref slowed down by 3% from 10973 to 11249 perf samples > - 464.h264ref:[.] FastFullPelBlockMotionSearch slowed down by 12% from 1446 > to 1619 perf samples > > Below reproducer instructions can be used to re-build both "first_bad" and > "last_good" cross-toolchains used in this bisection. Naturally, the scripts > will fail when triggerring benchmarking jobs if you don't have access to > Linaro TCWG CI. > > For your convenience, we have uploaded tarballs with pre-processed source and > assembly files at: > - First_bad save-temps: > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/24/artifact/artifacts/build-e8e2edd8ca88f8b0a7dba141349b2aa83284f3af/save-temps/ > - Last_good save-temps: > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/24/artifact/artifacts/build-77d200a546136c2855063613ff4bca1f682fb23a/save-temps/ > - Baseline save-temps: > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/24/artifact/artifacts/build-baseline/save-temps/ > > Configuration: > - Benchmark: SPEC CPU2006 > - Toolchain: Clang + Glibc + LLVM Linker > - Version: all components were built from their tip of trunk > - Target: aarch64-linux-gnu > - Compiler flags: -O3 > - Hardware: NVidia TX1 4x Cortex-A57 > > This benchmarking CI is work-in-progress, and we welcome feedback and > suggestions at linaro-toolchain@lists.linaro.org . In our improvement plans > is to add support for SPEC CPU2017 benchmarks and provide "perf > report/annotate" data behind these reports. > > THIS IS THE END OF INTERESTING STUFF. BELOW ARE LINKS TO BUILDS, > REPRODUCTION INSTRUCTIONS, AND THE RAW COMMIT. > > This commit has regressed these CI configurations: > - tcwg_bmk_llvm_tx1/llvm-master-aarch64-spec2k6-O3 > > First_bad build: > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/24/artifact/artifacts/build-e8e2edd8ca88f8b0a7dba141349b2aa83284f3af/ > Last_good build: > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/24/artifact/artifacts/build-77d200a546136c2855063613ff4bca1f682fb23a/ > Baseline build: > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/24/artifact/artifacts/build-baseline/ > Even more details: > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/24/artifact/artifacts/ > > Reproduce builds: > <cut> > mkdir investigate-llvm-e8e2edd8ca88f8b0a7dba141349b2aa83284f3af > cd investigate-llvm-e8e2edd8ca88f8b0a7dba141349b2aa83284f3af > > # Fetch scripts > git clone https://git.linaro.org/toolchain/jenkins-scripts > > # Fetch manifests and test.sh script > mkdir -p artifacts/manifests > curl -o artifacts/manifests/build-baseline.sh > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/24/artifact/artifacts/manifests/build-baseline.sh > --fail > curl -o artifacts/manifests/build-parameters.sh > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/24/artifact/artifacts/manifests/build-parameters.sh > --fail > curl -o artifacts/test.sh > https://ci.linaro.org/job/tcwg_bmk_ci_llvm-bisect-tcwg_bmk_tx1-llvm-master-aarch64-spec2k6-O3/24/artifact/artifacts/test.sh > --fail > chmod +x artifacts/test.sh > > # Reproduce the baseline build (build all pre-requisites) > ./jenkins-scripts/tcwg_bmk-build.sh @@ artifacts/manifests/build-baseline.sh > > # Save baseline build state (which is then restored in artifacts/test.sh) > mkdir -p ./bisect > rsync -a --del --delete-excluded --exclude /bisect/ --exclude /artifacts/ > --exclude /llvm/ ./ ./bisect/baseline/ > > cd llvm > > # Reproduce first_bad build > git checkout --detach e8e2edd8ca88f8b0a7dba141349b2aa83284f3af > ../artifacts/test.sh > > # Reproduce last_good build > git checkout --detach 77d200a546136c2855063613ff4bca1f682fb23a > ../artifacts/test.sh > > cd .. > </cut> > > Full commit (up to 1000 lines): > <cut> > commit e8e2edd8ca88f8b0a7dba141349b2aa83284f3af > Author: Erich Keane <erich.ke...@intel.com> > Date: Fri Sep 24 10:24:17 2021 -0700 > > Fix test from 8dd42f, capitalization in test > --- > clang/test/CXX/drs/dr17xx.cpp | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/clang/test/CXX/drs/dr17xx.cpp b/clang/test/CXX/drs/dr17xx.cpp > index 42303c83ae3c..c8648908ebda 100644 > --- a/clang/test/CXX/drs/dr17xx.cpp > +++ b/clang/test/CXX/drs/dr17xx.cpp > @@ -129,7 +129,7 @@ namespace dr1778 { // dr1778: 9 > namespace dr1762 { // dr1762: 14 > #if __cplusplus >= 201103L > float operator ""_E(const char *); > - // expected-error@+2 {{invalid suffix on literal; c++11 requires a space > between literal and identifier}} > + // expected-error@+2 {{invalid suffix on literal; C++11 requires a space > between literal and identifier}} > // expected-warning@+1 {{user-defined literal suffixes not starting with > '_' are reserved; no literal will invoke this operator}} > float operator ""E(const char *); > #endif > </cut> _______________________________________________ linaro-toolchain mailing list linaro-toolchain@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-toolchain
