On Wed, May 17, 2017 at 12:23 PM, Aldy Hernandez <al...@redhat.com> wrote: > Hi folks. > > I've been having troubles comparing the results of different test runs for > quite some time, and have finally decided to whine about it. Perhaps someone > can point out to whatever I may be doing wrong. > > I generally do "make check -k -j60" on two different trees and compare the > results with: > > cd /dir/current-build-tree > /gcc-sources/contrib/compare_tests /dir/previous-build-tree . > > However, this gives false positives virtually every time: > > New tests that FAIL: > > g++.dg/tree-prof/pr79259.C execution, -g > gcc.dg/tree-prof/ic-misattribution-1.c execution, -g > gcc.dg/tree-prof/merge_block.c execution, -g > gcc.dg/tree-prof/section-attr-1.c execution, -g > gcc.dg/tree-prof/section-attr-3.c execution, -g > ... > ... > > New tests that PASS: > > g++.dg/tree-prof/partition1.C compilation, -g > g++.dg/tree-prof/partition2.C compilation, -g > g++.dg/tree-prof/partition3.C compilation, -g > g++.dg/tree-prof/pr79259.C compilation, -g > ... > ... > > Old tests that passed, that have disappeared: (Eeek!) > > g++.dg/tree-prof/indir-call-prof-2.C compilation, -g > g++.dg/tree-prof/morefunc.C compilation, -g > g++.dg/tree-prof/pr63581.C compilation, -g > gcc.dg/tree-prof/20050826-2.c compilation, -g > gcc.dg/tree-prof/bb-reorg.c compilation, -g > ... > ... > > etc > etc > etc > > More often than not the tests that get mentioned are in the gcc.dg/tree-prof > directory, but I've seen failures in gcc.dg/tree-ssa, gcc.dg/vect and > others. > > When I run these tests manually (make check-gcc > RUNTESTFLAGS=tree-ssa.exp=whatever.c), they succeed. > > Running with smaller -jNN usually yields less false positives, but rarely do > I get a clean run. > > This has been happening for as long as I can remember (which may just be 12 > months ;-)). > > Has anyone seen this behavior? Is it my test box? Are there known problems > with parallel checks? > > FWIW, I'm running on an x86-64 Fedora 25 running a 4.9.6-200.fc25.x86_64 > kernel. The machine has 56 cores, and has dejagnu 1.6 installed.
Likely auto-fdo, see Andis recent patch to imrpove the siutation a bit. It helped for me (but I'm only using -j6 on the only machine I see the issue). Richard. > Thanks. > Aldy > > p.s. Attached is the full output from contrib/compare_tests.
