Note that this issue seems to only be fixable from within dejagnu and
that the commit of 6d2e2d3791bcea70131a6cf64a0a5223333a7b8e
effectively broke the log_summary proc in dejagnu's lib/framework.exp
by reseting the testcnt global which is used as follows...
# If the tool set `testcnt', it wants us to do a sanity check on the
# total count, so compare the reported number of testcases with the
# expected number. Maintaining an accurate count in `testcnt' isn't easy
# so it's not clear how often this will be used.
if {[info exists testcnt]} {
if { $testcnt > 0 } {
set totlcnt 0
# total all the testcases reported
foreach x { FAIL PASS XFAIL KFAIL XPASS KPASS UNTESTED
UNRESOLVED UNSUPPORTED } {
incr totlcnt test_counts($x,$which)
}
set testcnt test_counts(total,$which)
if { $testcnt>$totlcnt || $testcnt<$totlcnt } {
if { $testcnt > $totlcnt } {
set mismatch "unreported [expr {$testcnt - $totlcnt}]"
}
if { $testcnt < $totlcnt } {
set mismatch "misreported [expr {$totlcnt - $testcnt}]"
}
} else {
verbose "# of testcases run $testcnt"
}
if {[info exists mismatch]} {
clone_output "### ERROR: totals do not equal number of
testcases run"
clone_output "### ERROR: # of testcases expected $testcnt"
clone_output "### ERROR: # of testcases reported $totlcnt"
clone_output "### ERROR: # of testcases $mismatch\n"
}
}
}
foreach x { PASS FAIL XPASS XFAIL KPASS KFAIL UNRESOLVED UNTESTED
UNSUPPORTED } {
set val $test_counts($x,$which)
if { $val > 0 } {
set mess "# of $test_counts($x,name)"
if { $xml } {
xml_output " <summary>"
xml_output " <result>$x</result>"
xml_output " <description>$mess</description>"
xml_output " <total>$val</total>"
xml_output " </summary>"
}
if { [string length $mess] < 24 } {
append mess "\t"
}
clone_output "$mess\t$val"
}
On Sat, Mar 28, 2015 at 4:58 PM, Jack Howarth
<howarth.mailing.li...@gmail.com> wrote:
> The breakage from commit 6d2e2d3791bcea70131a6cf64a0a5223333a7b8e is
> due to execution of...
>
> foreach x $test_names {
> set test_counts($x,count) 0
> }
>
> in proc runtest introduced by the usage of reset_vars. I can confirm that...
>
> diff -uNr dejagnu-1.5.2.orig/runtest.exp dejagnu-1.5.2/runtest.exp
> --- dejagnu-1.5.2.orig/runtest.exp 2015-02-03 21:25:26.000000000 -0500
> +++ dejagnu-1.5.2/runtest.exp 2015-03-28 16:48:10.000000000 -0400
> @@ -1441,13 +1441,22 @@
> #
>
> proc runtest { test_file_name } {
> + global test_names test_counts
> + global prms_id
> + global bug_id
> global test_result
> global errcnt
> global errorInfo
> global tool
>
> clone_output "Running $test_file_name ..."
> - reset_vars
> + set prms_id 0
> + set bug_id 0
> +
> + foreach x $test_names {
> + set test_counts($x,count) 0
> + }
> +
> set test_result ""
>
> if {[file exists $test_file_name]} {
>
> reproduces the the breakage.
>
>
> On Sat, Mar 28, 2015 at 3:20 PM, Jack Howarth
> <howarth.mailing.li...@gmail.com> wrote:
>> Okay, a regression hunt of dejagnu master git revealed that this
>> regression is actually quite old and that the offending commit was...
>>
>> http://git.savannah.gnu.org/cgit/dejagnu.git/commit/?id=6d2e2d3791bcea70131a6cf64a0a5223333a7b8e
>>
>> 2011-03-14 Jan Kratochvil <jan.kratoch...@redhat.com>
>>
>> * runtest.exp (runtest): Call reset_vars at initialisation so that
>> a perror does not affect test results in other test scripts.
>>
>>
>> This issue was latent in 1.5.1 because the 1.5 branch was reused for
>> developing 1.5.1 whereas 1.5.2 was created from master instead.
>> I can confirm that reverting this change in 1.5.2 eliminates the
>> problem and I get the expected...
>>
>> === obj-c++ Summary for unix/-m32 ===
>>
>> # of expected passes 3069
>> # of unexpected failures 39
>> # of expected failures 2
>> # of unresolved testcases 57
>> # of unsupported tests 68
>>
>> === obj-c++ Summary for unix/-m64 ===
>>
>> # of expected passes 3002
>> # of unexpected failures 45
>> # of expected failures 18
>> # of unresolved testcases 60
>> # of unsupported tests 74
>>
>> === obj-c++ Summary ===
>>
>> # of expected passes 6071
>> # of unexpected failures 84
>> # of expected failures 20
>> # of unresolved testcases 117
>> # of unsupported tests 142
>>
>> On Sat, Mar 28, 2015 at 1:51 PM, Jack Howarth
>> <howarth.mailing.li...@gmail.com> wrote:
>>> Just to clarify. I am looking at
>>> http://git.savannah.gnu.org/cgit/dejagnu.git which shows branches for
>>> dejagnu-1.5 which behaves like 1.5.1 is I do a hard reset to the
>>> 2013-03-08 commit which marked the ChangeLog to 1.5.1. The trouble is
>>> that there is no matching dejagnu-1.5.1 branch but only a
>>> dejagnu-1.5.2 branch. The 1.5.2 branch is problematic as I can't seem
>>> to find a commit in that branch that behaves like 1.5.1. One would
>>> think at going back to..
>>>
>>> http://git.savannah.gnu.org/cgit/dejagnu.git/commit/?h=dejagnu-1.5.2&id=33b71e46c0a27fee374a922eb0792b9c618c54a8
>>>
>>> would be just like 1.5.1 but it doesn't. Any idea when the 1.5.2
>>> branch was created?
>>>
>>> On Sat, Mar 28, 2015 at 1:32 PM, Jack Howarth
>>> <howarth.mailing.li...@gmail.com> wrote:
>>>> I am having problems trying to regression hunt this issue in dejagnu
>>>> git. Using the gcc test suite run...
>>>>
>>>> make -k check-obj-c++ RUNTESTFLAGS="--target_board=unix'{-m32,-m64}'"
>>>>
>>>> I get...
>>>>
>>>> dejagnu 1.5.1
>>>>
>>>>
>>>> === obj-c++ Summary for unix/-m32 ===
>>>>
>>>> # of expected passes 3069
>>>> # of unexpected failures 39
>>>> # of expected failures 2
>>>> # of unresolved testcases 57
>>>> # of unsupported tests 68
>>>>
>>>> === obj-c++ Summary for unix/-m64 ===
>>>>
>>>> # of expected passes 3002
>>>> # of unexpected failures 45
>>>> # of expected failures 18
>>>> # of unresolved testcases 60
>>>> # of unsupported tests 74
>>>>
>>>> === obj-c++ Summary ===
>>>>
>>>> # of expected passes 6071
>>>> # of unexpected failures 84
>>>> # of expected failures 20
>>>> # of unresolved testcases 117
>>>> # of unsupported tests 142
>>>>
>>>> but...
>>>>
>>>> dejagnu 1.5.2
>>>>
>>>> === obj-c++ Summary for unix/-m32 ===
>>>>
>>>> # of expected passes 128
>>>>
>>>> === obj-c++ Summary for unix/-m64 ===
>>>>
>>>> # of expected passes 128
>>>>
>>>> === obj-c++ Summary ===
>>>>
>>>> # of expected passes 6071
>>>> # of unexpected failures 84
>>>> # of expected failures 20
>>>> # of unresolved testcases 117
>>>> # of unsupported tests 142
>>>>
>>>> I can reproduce the dejagnu 1.5.1 results with a build using...
>>>>
>>>> git clone git://git.sv.gnu.org/dejagnu.git
>>>> cd dejagnu
>>>> git reset --hard 6bd8dddc81d8a5f2e835f148a9be1509f3edb147
>>>>
>>>>
>>>> which is the parent commit for
>>>> http://git.savannah.gnu.org/cgit/dejagnu.git/commit/?id=52dad3fd75c64d8e8cc0de5f8ca123c4aa7c32d2
>>>>
>>>> however if try the same for the first commit in the gcc-1.5.2 branch...
>>>>
>>>> http://git.savannah.gnu.org/cgit/dejagnu.git/commit/?h=dejagnu-1.5.2&id=d8912f7d0567831f425115eaeb6ee915e8ea5b21
>>>>
>>>> using...
>>>>
>>>> git reset --hard 33b71e46c0a27fee374a922eb0792b9c618c54a8
>>>>
>>>> the build acts like 1.5.2 even though it should only have one
>>>> additional commit compared to the 1.5.1 release. Any clarifications on
>>>> how these git trees are arranged would be appreciated. So far I am
>>>> having no luck walking back in trunk to a point where the 1.5.1
>>>> behavior can be reproduced.
>>>> Jack
_______________________________________________
DejaGnu mailing list
DejaGnu@gnu.org
https://lists.gnu.org/mailman/listinfo/dejagnu
