> The other 1 commit build is https://lab.llvm.org/buildbot/#/builders/4/builds/940, I will ask the author if they received anything. Though even if they did I'm not sure what we could conclude from that.
They did not receive any notifications. And I'm out of ideas at this point, maybe someone else can spot something. There is https://github.com/llvm/llvm-zorg/issues/219 which is along the same lines but no solid evidence that it's the same cause. David thanks for reporting this, even if we don't manage to figure it out it's still better to know about these than not. On Thu, 8 Aug 2024 at 16:36, David Spickett <david.spick...@linaro.org> wrote: > I've looked at the build status reporter, which I think is > https://github.com/llvm/llvm-zorg/blob/ec94a29bfb0c247f0123c4efaf080a9b215d4433/zorg/buildbot/reporters/utils.py#L206. > I don't see anything wrong with that, and nothing that it removed from > Buildbot's version of that function looks like it would cause this problem. > > If the previous result was somehow still the last green build, this could > happen. However I don't see how that could be the case as there is a > finished build on the same worker prior to this one. So even if every other > worker was still busy, there's at least one previously failed result. My > guess from the Buildbot code is that prev_build["result"] will be None if > the build has not finished, so even in this case it would return False and > not send a message (though I suspect in progress builds aren't part of > these checks or perhaps it waits for the previous one to finish, not sure). > > I do see that the build prior to yours finished later, and some of the > builds immediately after the green build have some unusual timings where > earlier builds finish after later builds. This is expected with the 3 > workers we have, but perhaps could have affected when the results get > written to the database? Perhaps every build between the last green and > yours was notified and either no one said anything, or only builds with 1 > commit were notified and didn't say anything. > > The other 1 commit build is > https://lab.llvm.org/buildbot/#/builders/4/builds/940, I will ask the > author if they received anything. Though even if they did I'm not sure what > we could conclude from that. > > On Wed, 7 Aug 2024 at 02:12, David Blaikie <dblai...@gmail.com> wrote: > >> Any update on this? >> >> On Fri, Jul 26, 2024 at 1:28 AM David Spickett <david.spick...@linaro.org> >> wrote: >> >>> I don't see any special configuration for this builder that would make >>> it do this. >>> ( >>> https://github.com/llvm/llvm-zorg/blob/9496234c3e76d33aaae3988903c641fd406e1098/buildbot/osuosl/master/config/builders.py#L527 >>> ) >>> >>> It's unlikely that this build finished before previous jobs because it >>> only takes an hour and we have several builds in the hours before it. I >>> thought it may also be the first build on that worker that failed, but this >>> is also not the case (and if it were, this would be a buildbot bug since >>> it's supposed to look at the builder as a whole, not specific workers). >>> >>> It is a single commit blame list, and I know we started posting results >>> to GitHub in these cases. Presumably those have to also be green -> red >>> transitions, but perhaps something slipped through here and because there >>> was no PR to associate this change with, it sent an email instead? >>> >>> Thanks, >>> David Spickett. >>> >>> On Fri, 26 Jul 2024 at 03:03, David Blaikie <dblai...@gmail.com> wrote: >>> >>>> ---------- Forwarded message --------- >>>> From: David Blaikie <dblai...@gmail.com> >>>> Date: Thu, Jul 25, 2024 at 6:43 PM >>>> Subject: Re: ☠ Buildbot (LLVM Buildbot): clang-aarch64-sve-vls-2stage - >>>> failed test (failure) (main) >>>> To: <llvm.buildmas...@lab.llvm.org> >>>> Cc: <llvm.buildmas...@lab.llvm.org> >>>> >>>> >>>> This bot seems to be sending fail-mail when it's already red. My >>>> understanding was that fail mail only went out on green->red >>>> transitions. >>>> Is that understanding incorrect? Should this be reconfigured to match >>>> that >>>> understanding/not send fail mail when it's already red? >>>> >>>> On Thu, Jul 25, 2024 at 4:10 PM <llvm.buildmas...@lab.llvm.org> wrote: >>>> >>>> > The Buildbot has detected a failed build on builder >>>> > clang-aarch64-sve-vls-2stage while building clang. >>>> > >>>> > Full details are available at: >>>> > https://lab.llvm.org/buildbot/#/builders/4/builds/943 >>>> > >>>> > Worker for this Build: linaro-g3-02 >>>> > Blamelist: >>>> > David Blaikie <dblai...@gmail.com> >>>> > >>>> > BUILD FAILED: failed test (failure) >>>> > >>>> > Step 18 (test-suite) failure: test (failure) >>>> > ... >>>> > [ 24%] Built target >>>> > gfortran-regression-execute-regression__internal_readwrite_1_f90 >>>> > cd >>>> > >>>> /home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Fortran/gfortran/regression >>>> > && >>>> > >>>> /home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/stage2.install/bin/llvm-size >>>> > --format=sysv >>>> > >>>> /home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Fortran/gfortran/regression/gfortran-regression-execute-regression__internal_readwrite_4_f90 >>>> > > >>>> > >>>> /home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Fortran/gfortran/regression/gfortran-regression-execute-regression__internal_readwrite_4_f90.size >>>> > make[2]: Leaving directory >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build' >>>> > [ 24%] Built target >>>> > gfortran-regression-execute-regression__internal_readwrite_4_f90 >>>> > make[1]: Leaving directory >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build' >>>> > make: *** [Makefile:136: all] Error 2 >>>> > 2024-07-25 23:09:27 INFO: Testing... >>>> > 2024-07-25 23:09:27 INFO: Execute: >>>> > >>>> /home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/stage1/bin/llvm-lit >>>> > -v -j 32 >>>> > >>>> /home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build >>>> > -o >>>> > >>>> /home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/outputqee5waec.json >>>> > -- Testing: 9770 tests, 32 workers -- >>>> > NOEXE: test-suite :: Bitcode/simd_ops/simd_ops_test_op_abs_1026.test >>>> (1 of >>>> > 9770) >>>> > ******************** TEST 'test-suite :: >>>> > Bitcode/simd_ops/simd_ops_test_op_abs_1026.test' FAILED >>>> ******************** >>>> > Executable >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Bitcode/simd_ops/simd_ops_test_op_abs_1026' >>>> > is missing >>>> > ******************** >>>> > NOEXE: test-suite :: Bitcode/simd_ops/simd_ops_test_op_abs_1028.test >>>> (2 of >>>> > 9770) >>>> > ******************** TEST 'test-suite :: >>>> > Bitcode/simd_ops/simd_ops_test_op_abs_1028.test' FAILED >>>> ******************** >>>> > Executable >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Bitcode/simd_ops/simd_ops_test_op_abs_1028' >>>> > is missing >>>> > ******************** >>>> > NOEXE: test-suite :: Bitcode/simd_ops/simd_ops_test_op_abs_367.test >>>> (3 of >>>> > 9770) >>>> > ******************** TEST 'test-suite :: >>>> > Bitcode/simd_ops/simd_ops_test_op_abs_367.test' FAILED >>>> ******************** >>>> > Executable >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Bitcode/simd_ops/simd_ops_test_op_abs_367' >>>> > is missing >>>> > ******************** >>>> > NOEXE: test-suite :: Bitcode/simd_ops/simd_ops_test_op_abs_40.test (4 >>>> of >>>> > 9770) >>>> > ******************** TEST 'test-suite :: >>>> > Bitcode/simd_ops/simd_ops_test_op_abs_40.test' FAILED >>>> ******************** >>>> > Executable >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Bitcode/simd_ops/simd_ops_test_op_abs_40' >>>> > is missing >>>> > ******************** >>>> > NOEXE: test-suite :: Bitcode/simd_ops/simd_ops_test_op_abs_366.test >>>> (5 of >>>> > 9770) >>>> > ******************** TEST 'test-suite :: >>>> > Bitcode/simd_ops/simd_ops_test_op_abs_366.test' FAILED >>>> ******************** >>>> > Executable >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Bitcode/simd_ops/simd_ops_test_op_abs_366' >>>> > is missing >>>> > ******************** >>>> > NOEXE: test-suite :: Bitcode/simd_ops/simd_ops_test_op_add_1030.test >>>> (6 of >>>> > 9770) >>>> > ******************** TEST 'test-suite :: >>>> > Bitcode/simd_ops/simd_ops_test_op_add_1030.test' FAILED >>>> ******************** >>>> > Executable >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Bitcode/simd_ops/simd_ops_test_op_add_1030' >>>> > is missing >>>> > ******************** >>>> > NOEXE: test-suite :: Bitcode/simd_ops/simd_ops_test_op_abs_699.test >>>> (7 of >>>> > 9770) >>>> > ******************** TEST 'test-suite :: >>>> > Bitcode/simd_ops/simd_ops_test_op_abs_699.test' FAILED >>>> ******************** >>>> > Executable >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Bitcode/simd_ops/simd_ops_test_op_abs_699' >>>> > is missing >>>> > ******************** >>>> > NOEXE: test-suite :: Bitcode/simd_ops/simd_ops_test_op_abs_368.test >>>> (8 of >>>> > 9770) >>>> > ******************** TEST 'test-suite :: >>>> > Bitcode/simd_ops/simd_ops_test_op_abs_368.test' FAILED >>>> ******************** >>>> > Executable >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Bitcode/simd_ops/simd_ops_test_op_abs_368' >>>> > is missing >>>> > ******************** >>>> > NOEXE: test-suite :: Bitcode/simd_ops/simd_ops_test_op_add_1031.test >>>> (9 of >>>> > 9770) >>>> > ******************** TEST 'test-suite :: >>>> > Bitcode/simd_ops/simd_ops_test_op_add_1031.test' FAILED >>>> ******************** >>>> > Executable >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Bitcode/simd_ops/simd_ops_test_op_add_1031' >>>> > is missing >>>> > ******************** >>>> > NOEXE: test-suite :: Bitcode/simd_ops/simd_ops_test_op_abs_39.test >>>> (10 of >>>> > 9770) >>>> > ******************** TEST 'test-suite :: >>>> > Bitcode/simd_ops/simd_ops_test_op_abs_39.test' FAILED >>>> ******************** >>>> > Executable >>>> > >>>> '/home/tcwg-buildbot/worker/clang-aarch64-sve-vls-2stage/test/sandbox/build/Bitcode/simd_ops/simd_ops_test_op_abs_39' >>>> > is missing >>>> > ******************** >>>> > >>>> > Sincerely, >>>> > LLVM Buildbot >>>> > >>>> > >>>> _______________________________________________ >>>> linaro-toolchain mailing list -- linaro-toolchain@lists.linaro.org >>>> To unsubscribe send an email to linaro-toolchain-le...@lists.linaro.org >>>> >>> _______________________________________________ linaro-toolchain mailing list -- linaro-toolchain@lists.linaro.org To unsubscribe send an email to linaro-toolchain-le...@lists.linaro.org
