> On 6 May 2020, at 8:15 pm, Sebastian Huber > <sebastian.hu...@embedded-brains.de> wrote: > > On 06/05/2020 12:00, Chris Johns wrote: > >>> On 6/5/20 7:35 pm, Sebastian Huber wrote: >>>> On 06/05/2020 10:41, chr...@rtems.org wrote: >>> >>>> From: Chris Johns<chr...@rtems.org> >>>> >>>> Updates #2962 >>>> --- >>>> bsps/powerpc/psim/config/psim-testsuite.tcfg | 22 ++++++++++++++++++++ >>>> 1 file changed, 22 insertions(+) >>>> create mode 100644 bsps/powerpc/psim/config/psim-testsuite.tcfg >>>> >>>> diff --git a/bsps/powerpc/psim/config/psim-testsuite.tcfg >>>> b/bsps/powerpc/psim/config/psim-testsuite.tcfg >>>> new file mode 100644 >>>> index 0000000000..b0d2a05086 >>>> --- /dev/null >>>> +++ b/bsps/powerpc/psim/config/psim-testsuite.tcfg >>>> @@ -0,0 +1,22 @@ >>>> +# >>>> +# PSIM RTEMS Test Database. >>>> +# >>>> +# Format is one line per test that is_NOT_ built. >>>> +# >>>> + >>>> +expected-fail: fsimfsgeneric01 >>>> +expected-fail: block11 >>>> +expected-fail: rbheap01 >>>> +expected-fail: termios01 >>>> +expected-fail: ttest01 >>>> +expected-fail: psx12 >>>> +expected-fail: psxchroot01 >>>> +expected-fail: psxfenv01 >>>> +expected-fail: psximfs02 >>>> +expected-fail: psxpipe01 >>>> +expected-fail: spextensions01 >>>> +expected-fail: spfatal31 >>>> +expected-fail: spfifo02 >>>> +expected-fail: spmountmgr01 >>>> +expected-fail: spprivenv01 >>>> +expected-fail: spstdthreads01 >>> >>> I don't think these tests are expected to fail. If they fail, then there is >>> a bug somewhere. >> >> Yes we hope no tests fail but they can and do. Excluding tests because they >> fail would be incorrect. In the 5.1 release these bugs are present so we >> expect, or maybe it should say, we know the test will fail. With this change >> any thing that appears in the failure column is "unexpected" and that means >> the user build of the release does not match the state we "expect" and it is >> worth investigation by the user. >> >> Without these tests being tagged this way the user would have no idea where >> the stand after a build and test run and that would mean we would have to >> make sure a release has no failures. I consider that as not practical or >> realistic. > Maybe we need another state, e.g. something-is-broken-please-fix-it.
I do not think so, it is implicit in the failure or the test is broken. The only change is to add unexpected-pass, that will be on master after the 5 branch. Chris Chris _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
