On Wed, May 08, 2019 at 12:28:24PM +0200, Gerd Hoffmann wrote:
> On Thu, May 02, 2019 at 09:41:21PM -0300, Eduardo Habkost wrote:
> > From: Cleber Rosa <[email protected]>
> >
> > When running on very low powered environments, some tests may time out
> > causing false negatives. As a conservative change, and for
> > considering that human time (investigating false negatives) is worth
> > more than some extra machine cycles (and time), let's increase the
> > overall timeout.
>
> Tests can also timeout due to slow downloads of test kernels.
> Any chance to run the downloads without timeout?
I acknowledge this is an issue, and have thought about two possible
ways to solve it:
1) Downloading/caching/checking all the test assets in a job "pre-tests"
plugin. This would involve/require:
-
https://trello.com/c/WPd4FrIy/1479-add-support-to-specify-assets-in-test-docstring
- https://trello.com/c/CKP7YS6G/1481-on-cache-check-for-asset-fetcher
2) Report the test phase (say, setUp()) to the test runner, which
would allow the runner to:
- Not take that time into account for the test (similar to
https://github.com/avocado-framework/avocado/pull/3076)
- Retry the setUp a number of times (if the test opts in or doesn't
opt out)
I'm very much interested in your opinion so we can evolve the idea into
implementation.
Thanks!
- Cleber
>
> cheers,
> Gerd
>
