Re: [PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

2017-05-16 Thread Sebastian Huber
On 17/05/17 00:40, Chris Johns wrote: On 16/05/2017 18:35, Sebastian Huber wrote: On 16/05/17 10:28, Chris Johns wrote: What we do need to do is make sure the test results express the true state. If a test is broken it should fail. If it is tagged an expected-fail we do not consider it a regres

Re: [PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

2017-05-16 Thread Chris Johns
On 17/05/2017 10:34, Gedare Bloom wrote: On Tue, May 16, 2017 at 6:40 PM, Chris Johns wrote: On 16/05/2017 18:35, Sebastian Huber wrote: On 16/05/17 10:28, Chris Johns wrote: What we do need to do is make sure the test results express the true state. If a test is broken it should fail. If

Re: [PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

2017-05-16 Thread Gedare Bloom
On Tue, May 16, 2017 at 6:40 PM, Chris Johns wrote: > On 16/05/2017 18:35, Sebastian Huber wrote: >> >> On 16/05/17 10:28, Chris Johns wrote: >>> >>> What we do need to do is make sure the test results express the true >>> state. If a test is broken it should fail. If it is tagged an >>> expected-

Re: [PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

2017-05-16 Thread Chris Johns
On 16/05/2017 18:35, Sebastian Huber wrote: On 16/05/17 10:28, Chris Johns wrote: What we do need to do is make sure the test results express the true state. If a test is broken it should fail. If it is tagged an expected-fail we do not consider it a regression. Loosing test coverage simply be

Re: [PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

2017-05-16 Thread Chris Johns
On 16/05/2017 18:32, Sebastian Huber wrote: On 16/05/17 10:28, Chris Johns wrote: >You >wanted more tests to run in the SMP mode if SMP is enabled. Now only 36 >tests are left over. I think it is a reasonable expectation when you build an SMP RTEMS the tests run with SMP enabled using all avail

Re: [PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

2017-05-16 Thread Sebastian Huber
On 16/05/17 10:28, Chris Johns wrote: What we do need to do is make sure the test results express the true state. If a test is broken it should fail. If it is tagged an expected-fail we do not consider it a regression. Loosing test coverage simply because a test uses a non-SMP feature somehow

Re: [PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

2017-05-16 Thread Sebastian Huber
On 16/05/17 10:28, Chris Johns wrote: >You >wanted more tests to run in the SMP mode if SMP is enabled. Now only 36 >tests are left over. I think it is a reasonable expectation when you build an SMP RTEMS the tests run with SMP enabled using all available cores. No, we don't run the tests usi

Re: [PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

2017-05-16 Thread Chris Johns
On 16/5/17 5:54 pm, Sebastian Huber wrote: > On 13/05/17 00:30, Chris Johns wrote: >> On 12/5/17 8:24 pm, Sebastian Huber wrote: >>> Mark tests that require review due to >>> CONFIGURE_DISABLE_SMP_CONFIGURATION. >> If a test fails it fails. I feel the test results need to correctly >> express the f

Re: [PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

2017-05-16 Thread Sebastian Huber
On 13/05/17 00:30, Chris Johns wrote: On 12/5/17 8:24 pm, Sebastian Huber wrote: Mark tests that require review due to CONFIGURE_DISABLE_SMP_CONFIGURATION. If a test fails it fails. I feel the test results need to correctly express the failures and we should not masks them in the way. I do not

Re: [PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

2017-05-12 Thread Chris Johns
On 12/5/17 8:24 pm, Sebastian Huber wrote: > Mark tests that require review due to > CONFIGURE_DISABLE_SMP_CONFIGURATION. If a test fails it fails. I feel the test results need to correctly express the failures and we should not masks them in the way. I do not agree with changes that suppress a fa

[PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

2017-05-12 Thread Sebastian Huber
Mark tests that require review due to CONFIGURE_DISABLE_SMP_CONFIGURATION. Update #3020. --- testsuites/libtests/block08/system.h | 1 + testsuites/libtests/cpuuse/system.h| 1 + testsuites/libtests/rtmonuse/system.h | 1 + testsuites/libtests/termios05/init.c | 1 + test