On Mon, Feb 1, 2021 at 11:56 AM Peter Dufault <dufa...@hda.com> wrote:
> Can the test be marked as a possible emulator failure? For example (I > haven't looked at the report) use "?" (Huh?) and not "!" (Failed!) or "." > )(Passed), that is, the test failed as an emulated system and the RTEMS > project suspects the emulation itself is causing this. > That's not bad. The BSPs which might want to use this sometimes are qemu only but we have a lot of BSPs (pc, sparc, etc) that run on both simulator and real hardware. For ones like the Zynq where you have a qemu only variant, this would be perfect. But for ones which can run on both, I don't know how to eliminate the use on hardware. > > You shouldn't be allowed to use this on physical devices. > +1 > > > On Feb 1, 2021, at 12:31 , Joel Sherrill <j...@rtems.org> wrote: > > > > > > > > On Mon, Feb 1, 2021 at 11:14 AM Gedare Bloom <ged...@rtems.org> wrote: > > > > > > On Mon, Feb 1, 2021 at 9:42 AM Joel Sherrill <j...@rtems.org> wrote: > > Hi > > > > On the aarch64 qemu testing, we are seeing some tests which seem to pass > most of the time but fail intermittently. It appears to be based somewhat > on host load but there may be other factors. > > > > There does not appear to be a good test results state for these. Marking > them expected pass or fail means they will get flagged incorrectly > sometimes. > > > > > > Are they expected to pass every time? > > > > Normally yes. But there appears to be some external factors particularly > load impacting qemu which lead to failures on target side code. > > > > > > Intermittent failures are suspicious, and there is limited value to > running a test that has an "intermittent failure" state, since it will > always be successful if you don't care if it passes or fails. > > > > Yeah. No matter what you mark them, it sucks. > > > > > > I don't see not running them as a good option. Beyond adding a new state > to reflect this oddity, any suggestions? > > > > --joel > > _______________________________________________ > > devel mailing list > > devel@rtems.org > > http://lists.rtems.org/mailman/listinfo/devel > > _______________________________________________ > > devel mailing list > > devel@rtems.org > > http://lists.rtems.org/mailman/listinfo/devel > > Peter > ----------------- > Peter Dufault > HD Associates, Inc. Software and System Engineering > > This email is delivered through the public internet using protocols > subject to interception and tampering. > >
_______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
