On Wed, Sep 18, 2019 at 01:13:29PM +0200, Philippe Mathieu-Daudé wrote: > On 9/18/19 9:16 AM, David Gibson wrote: > > Hi, > > > > I'm finding make check-acceptance is currently useless for me as a > > pre-pull test, because a bunch of the tests are not at all reliable. > > There are a bunch which I'm still investigating, but for now I'm > > looking at the MIPS Malta SSH tests. > > > > There seem to be at least two problems here. First, the test includes > > a download of a pretty big guest disk image. This can easily exhaust > > the 2m30 timeout on its own. > > Gerd raised this issue few months ago: > > https://www.mail-archive.com/[email protected]/msg615619.html
Ah, yes indeed.
> > Even without the timeout, it makes the test really slow, even on
> > repeated runs. Is there some way we can make the image download part
> > of "building" the tests rather than actually running the testsuite, so
> > that a) the test themselves go faster and b) we don't include the
> > download in the test timeout - obviously the download speed is hugely
> > dependent on factors that aren't really related to what we're testing
> > here.
> >
> > In the meantime, I tried hacking it by just increasing the timeout to
> > 10m. That got several of the tests working for me, but one still
> > failed. Specifically 'LinuxSSH.test_mips_malta32eb_kernel3_2_0' still
> > timed out for me, but now after booting the guest, rather than during
> > the image download. Looking at the avocado log file I'm seeing a
> > bunch of soft lockup messages from the guest console, AFAICT. So it
> > looks like we have a real bug here, which I suspect has been
> > overlooked precisely because the download problems mean this test
> > isn't reliable.
> >
> > Any thoughts on how to improve the situation?
>
> Maybe we should disable this test and run it manually...
Until we can fix it better, I really think we should. A test this
unreliable verges on worse than useless.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
signature.asc
Description: PGP signature
