Christophe Lyon via Gcc <gcc@gcc.gnu.org> writes: > On Wed, 23 Sep 2020 at 01:47, Martin Sebor <mse...@gmail.com> wrote: >> >> On 9/22/20 9:15 AM, Christophe Lyon wrote: >> > On Tue, 22 Sep 2020 at 17:02, Martin Sebor <mse...@gmail.com> wrote: >> >> >> >> Hi Christophe, >> >> >> >> While checking recent test results I noticed many posts with results >> >> for various flavors of arm that at high level seem like duplicates >> >> of one another. >> >> >> >> For example, the batch below all have the same title, but not all >> >> of the contents are the same. The details (such as test failures) >> >> on some of the pages are different. >> >> >> >> Can you help explain the differences? Is there a way to avoid >> >> the duplication? >> >> >> > >> > Sure, I am aware that many results look the same... >> > >> > >> > If you look at the top of the report (~line 5), you'll see: >> > Running target myarm-sim >> > Running target >> > myarm-sim/-mthumb/-mcpu=cortex-m3/-mfloat-abi=soft/-march=armv7-m >> > Running target >> > myarm-sim/-mthumb/-mcpu=cortex-m0/-mfloat-abi=soft/-march=armv6s-m >> > Running target >> > myarm-sim/-mcpu=cortex-a7/-mfloat-abi=hard/-march=armv7ve+simd >> > Running target >> > myarm-sim/-mthumb/-mcpu=cortex-m7/-mfloat-abi=hard/-march=armv7e-m+fp.dp >> > Running target >> > myarm-sim/-mthumb/-mcpu=cortex-m4/-mfloat-abi=hard/-march=armv7e-m+fp >> > Running target >> > myarm-sim/-mthumb/-mcpu=cortex-m33/-mfloat-abi=hard/-march=armv8-m.main+fp+dsp >> > Running target >> > myarm-sim/-mcpu=cortex-a7/-mfloat-abi=soft/-march=armv7ve+simd >> > Running target >> > myarm-sim/-mthumb/-mcpu=cortex-a7/-mfloat-abi=hard/-march=armv7ve+simd >> > >> > For all of these, the first line of the report is: >> > LAST_UPDATED: Tue Sep 22 09:39:18 UTC 2020 (revision >> > r11-3343-g44135373fcdbe4019c5524ec3dff8e93d9ef113c) >> > TARGET=arm-none-eabi CPU=default FPU=default MODE=default >> > >> > I have other combinations where I override the configure flags, eg: >> > LAST_UPDATED: Tue Sep 22 11:25:12 UTC 2020 (revision >> > r9-8928-gb3043e490896ea37cd0273e6e149c3eeb3298720) >> > TARGET=arm-none-linux-gnueabihf CPU=cortex-a9 FPU=neon-fp16 MODE=thumb >> > >> > I tried to see if I could fit something in the subject line, but that >> > didn't seem convenient (would be too long, and I fear modifying the >> > awk script....) >> >> Without some indication of a difference in the title there's no way >> to know what result to look at, and checking all of them isn't really >> practical. The duplication (and the sheer number of results) also >> make it more difficult to find results for targets other than arm-*. >> There are about 13,000 results for September and over 10,000 of those >> for arm-* alone. It's good to have data but when there's this much >> of it, and when the only form of presentation is as a running list, >> it's too cumbersome to work with. >> > > To help me track & report regressions, I build higher level reports like: > https://people.linaro.org/~christophe.lyon/cross-validation/gcc/trunk/0latest/report-build-info.html > where it's more obvious what configurations are tested. > > Each line of such reports can send a message to gcc-testresults. > > I can control when such emails are sent, independently for each line: > - never > - for daily bump > - for each validation > > So, I can easily reduce the amount of emails (by disabling them for > some configurations), > but that won't make the subject more informative. > I included the short revision (rXX-YYYY) in the title to make it clearer. > > The number of configurations has grown over time because we regularly > found regressions > in configurations not tested previously.
Yeah. And thanks for doing this. It's caught all sorts of things that we didn't see previously. > I can probably easily add the values of --with-cpu, --with-fpu, > --with-mode and RUNTESTFLAGS > as part of the [<branch> revision rXX-YYYY-ZZZZZ] string in the title, > would that help? > I fear that's going to make very long subject lines. > > It would probably be cleaner to update test_summary such that it adds > more info as part of $host > (as in "... testsuite on $host"), so that it grabs useful configure > parameters and runtestflags, however > this would be more controversial. FWIW, taking a recent arbitrary example: Results for 8.4.1 20200918 [releases/gcc-8 revision r8-10517-gbbb72c2ccc271541e0d1feb25d2256d47041df59] (GCC) testsuite on arm-none-linux-gnueabihf For the subject line, I think we can drop the branch name if it's the obvious one for the version number. And we could drop the git hash and just use the rN-MMMM id. The “(GCC) testsuite” part seems like noise too; AFAICT that's a de facto invariant string for everyone who's posted this month. So that would give: Results for 8.4.1 20200918 [r8-10517] on arm-none-linux-gnueabihf and hopefully free up some space at the end for the kind of thing you mention. Thanks, Richard
