On 20050728 the results in gcc.sum for gcov-4.c said:
PASS: gcc.misc-tests/gcov-4.c (test for excess errors)
PASS: gcc.misc-tests/gcov-4.c execution test
PASS: gcc.misc-tests/gcov-4.c gcov
while on 20050730 with no change to the test they said:
PASS: gcc.misc-tests/gcov-4.c (test for excess errors)
PASS: gcc.misc-tests/gcov-4.c execution test
FAIL: 96:is -:should be 1
FAIL: gcc.misc-tests/gcov-4.c gcov: 1 failures in line counts, 0 in branch
percentages, 0 in return percentages
The text which appears after PASS or FAIL is meant to be a name for a test
assertion which is constant as long as the testcase itself is unchanged. Thus a
testsuite regression should be indicated by PASS: <test name> changing to FAIL:
<test name> rather than by the removal of a PASS and its replacement by FAILs
with different names; new FAILs in general can indicate newly added tests which
fail from the start and may or may not be regressions. (contrib/compare_tests
illustrates this logic, showing these new failures less helpfully than if the
PASS and FAIL lines had consistent test names, though it's not the tool I'm
actually using for this comparison.)
Thus, each gcov output line which is checked should have its own fixed test
assertion name which appears after PASS if the output is correct and after FAIL
if the output is incorrect; the .log file might have additional information
about what the difference was, but not on the PASS or FAIL lines. The final
summary PASS or FAIL line probably isn't needed in these circumstances, but if
it is again the test assertion name should not depend on whether the test passes
or fails (information about the number and kind of failures is a separate
matter, not for the PASS or FAIL lines).
--
Summary: gcov test assertion naming
Product: gcc
Version: 4.1.0
Status: UNCONFIRMED
Severity: normal
Priority: P2
Component: testsuite
AssignedTo: unassigned at gcc dot gnu dot org
ReportedBy: jsm28 at gcc dot gnu dot org
CC: gcc-bugs at gcc dot gnu dot org
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=23149
