On Mon, Feb 1, 2021 at 11:02 AM Joel Sherrill <j...@rtems.org> wrote:
> > > On Mon, Feb 1, 2021 at 11:56 AM Peter Dufault <dufa...@hda.com> wrote: > >> Can the test be marked as a possible emulator failure? For example (I >> haven't looked at the report) use "?" (Huh?) and not "!" (Failed!) or "." >> )(Passed), that is, the test failed as an emulated system and the RTEMS >> project suspects the emulation itself is causing this. >> > > That's not bad. The BSPs which might want to use this sometimes are qemu > only but we have a lot of BSPs (pc, sparc, etc) that run on both simulator > and real hardware. For ones like the Zynq where you have a qemu only > variant, this would be perfect. But for ones which can run on both, I > don't know how to eliminate the use on hardware. > I'm fine with this. Unlike hardware, emulation/simulation bugs are transient and can be safely ignored (to an extent) by us. > >> You shouldn't be allowed to use this on physical devices. >> > > +1 > Yes, I had initially thought perhaps this is like hardware bugs/errata, but it is not. You would either fix/workaround the hardware (and want to know it is broken/failing), or you would abandon that device :) > >> > On Feb 1, 2021, at 12:31 , Joel Sherrill <j...@rtems.org> wrote: >> > >> > >> > >> > On Mon, Feb 1, 2021 at 11:14 AM Gedare Bloom <ged...@rtems.org> wrote: >> > >> > >> > On Mon, Feb 1, 2021 at 9:42 AM Joel Sherrill <j...@rtems.org> wrote: >> > Hi >> > >> > On the aarch64 qemu testing, we are seeing some tests which seem to >> pass most of the time but fail intermittently. It appears to be based >> somewhat on host load but there may be other factors. >> > >> > There does not appear to be a good test results state for these. >> Marking them expected pass or fail means they will get flagged incorrectly >> sometimes. >> > >> > >> > Are they expected to pass every time? >> > >> > Normally yes. But there appears to be some external factors >> particularly load impacting qemu which lead to failures on target side code. >> > >> > >> > Intermittent failures are suspicious, and there is limited value to >> running a test that has an "intermittent failure" state, since it will >> always be successful if you don't care if it passes or fails. >> > >> > Yeah. No matter what you mark them, it sucks. >> > >> > >> > I don't see not running them as a good option. Beyond adding a new >> state to reflect this oddity, any suggestions? >> > >> > --joel >> > _______________________________________________ >> > devel mailing list >> > devel@rtems.org >> > http://lists.rtems.org/mailman/listinfo/devel >> > _______________________________________________ >> > devel mailing list >> > devel@rtems.org >> > http://lists.rtems.org/mailman/listinfo/devel >> >> Peter >> ----------------- >> Peter Dufault >> HD Associates, Inc. Software and System Engineering >> >> This email is delivered through the public internet using protocols >> subject to interception and tampering. >> >>
_______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
