Re: [llvm-dev] Timeout tests timing out

2020-12-04 Thread Dan Liew
Sorry I haven't had the time to dig into the issue but it looks someone else already fixed it :) On Thu, 3 Dec 2020 at 21:00, David Blaikie wrote: > > This'll hopefully be addressed by https://reviews.llvm.org/D92563 > > On Mon, Nov 30, 2020 at 6:28 AM Diana Picus wrote: >> >> Ping again. We're

Re: [llvm-dev] Timeout tests timing out

2020-12-03 Thread David Blaikie
This'll hopefully be addressed by https://reviews.llvm.org/D92563 On Mon, Nov 30, 2020 at 6:28 AM Diana Picus wrote: > Ping again. We're seeing this on several aarch64 bots, what can we do > about it? > > On Mon, 23 Nov 2020 at 21:19, David Blaikie via llvm-dev < > llvm-...@lists.llvm.org> wrote

Re: [llvm-dev] Timeout tests timing out

2020-11-30 Thread Diana Picus
Ping again. We're seeing this on several aarch64 bots, what can we do about it? On Mon, 23 Nov 2020 at 21:19, David Blaikie via llvm-dev < llvm-...@lists.llvm.org> wrote: > Ping on this - Dan, any chance you could take a look here? > > On Mon, Nov 9, 2020 at 1:48 PM Arthur Eubanks wrote: > > > >

Re: [llvm-dev] Timeout tests timing out

2020-11-23 Thread David Blaikie
Ping on this - Dan, any chance you could take a look here? On Mon, Nov 9, 2020 at 1:48 PM Arthur Eubanks wrote: > > Another case: http://lab.llvm.org:8011/#/builders/43/builds/810 > shtest-timeout.py seems to be fairly flaky on the clang-cmake-aarch64-quick > bot: http://lab.llvm.org:8011/#/buil

Re: [llvm-dev] Timeout tests timing out

2020-11-09 Thread Arthur Eubanks
Another case: http://lab.llvm.org:8011/#/builders/43/builds/810 shtest-timeout.py seems to be fairly flaky on the clang-cmake-aarch64-quick bot: http://lab.llvm.org:8011/#/builders/43, I get notifications from it fairly often On Thu, Oct 22, 2020 at 7:15 PM David Blaikie via llvm-dev < llvm-...@li

Re: Timeout tests timing out

2020-10-22 Thread David Blaikie
Looks like there might still be some issues with the timeout tests? http://lab.llvm.org:8011/#/builders/126/builds/226/steps/13/logs/FAIL__lit___shtest-timeout_py On Sun, Oct 4, 2020 at 2:44 PM Dan Liew wrote: > > > One thing we could do to remove fragility in the test is to remove the > > > run

Re: Timeout tests timing out

2020-10-04 Thread Dan Liew
> > One thing we could do to remove fragility in the test is to remove the > > running of `short.py` in the test. This is only invoked to check that > > it's possible for a command to run to completion in the presence of a > > fixed timeout. If we can live without testing that part (i.e. we only >

Re: Timeout tests timing out

2020-09-21 Thread David Blaikie
On Fri, Sep 18, 2020 at 4:31 PM Dan Liew wrote: > > On Wed, 16 Sep 2020 at 22:24, David Blaikie wrote: > > > > I appreciate the value of the feature - but it's possible the test > > doesn't pull its weight. Is the code that implements the feature > > liable to failure/often touched? If it's prett

Re: [llvm-dev] Timeout tests timing out

2020-09-18 Thread Dan Liew
On Wed, 16 Sep 2020 at 22:32, Mehdi AMINI wrote: > > Hi, > > Depending on timing in a test is quite brittle in general: can we mock the > timeout instead and make this fully deterministic somehow? > > The test has this comment: > > > FIXME: This test is fragile because it relies on time which can

Re: Timeout tests timing out

2020-09-18 Thread Dan Liew
On Wed, 16 Sep 2020 at 22:24, David Blaikie wrote: > > I appreciate the value of the feature - but it's possible the test > doesn't pull its weight. Is the code that implements the feature > liable to failure/often touched? If it's pretty static/failure is > unlikely, possibly the time and flaky f

Re: [llvm-dev] Timeout tests timing out

2020-09-17 Thread Mehdi AMINI
Hi, Depending on timing in a test is quite brittle in general: can we mock the timeout instead and make this fully deterministic somehow? The test has this comment: > FIXME: This test is fragile because it relies on time which can > be affected by system performance. In particular we are current

Re: Timeout tests timing out

2020-09-17 Thread Dan Liew
Hi David, Unfortunately writing a reliable test is tricky given that the functionality we're trying to test involves timing. I would advise against disabling the test entirely because it actually tests functionality that people use. I'd suggest bumping up the time limits. This is what I've done in

Re: Timeout tests timing out

2020-09-16 Thread David Blaikie
I appreciate the value of the feature - but it's possible the test doesn't pull its weight. Is the code that implements the feature liable to failure/often touched? If it's pretty static/failure is unlikely, possibly the time and flaky failures aren't worth the value of possibly catching a low-chan

Re: Timeout tests timing out

2020-09-16 Thread David Blaikie
Ping on this On Wed, Sep 9, 2020 at 8:27 PM David Blaikie wrote: > > The clang-cmake-armv8-lld (linaro-toolchain owners) buildbot is timing out > trying to run some timeout tests (Dan Liew author): > > Pass: http://lab.llvm.org:8011/builders/clang-cmake-armv8-lld/builds/5672 > Fail: http://lab.l

Timeout tests timing out

2020-09-09 Thread David Blaikie
The clang-cmake-armv8-lld (linaro-toolchain owners) buildbot is timing out trying to run some timeout tests (Dan Liew author): Pass: http://lab.llvm.org:8011/builders/clang-cmake-armv8-lld/builds/5672 Fail: http://lab.llvm.org:8011/builders/clang-cmake-armv8-lld/builds/5673 Is there anything we c