On Fri, Sep 13, 2024 at 4:19 PM Frank Ch. Eigler <f...@redhat.com> wrote: > > Hi - > > The gcc-testresults mailing list is a well-established place to plop > snippets of testsuite results. It's an okay way to archive and > distribute overall counts, but it's not machine readable, and it's way > incomplete (lacks .log content, a lot of metadata, barely meaningfully > searchable) for trying to understand why something failed. > > A few years ago, our team @ RH built the bunsen system to serve as a > structured repository for dejagnu (and other) testsuite types. [1] An > instance running here [2] has been collecting data from the sourceware > buildbots and individuals (hi iains!) for years. It convers all the > toolchain pieces and then some, including gcc. It features machine > readable indexing, cross-referenced test case browsing, regression > analysis, old data aging, and many other capabilities. (It even has a > piece that extracts data from the gcc-testresults@ mailing list > archives, but there's too little there to bother. [4]) > > This patch attempts to make it easy for gcc developers who use the > contrib/test_summary script today to opt in to contributing their > results to a bunsen server (defaulting to the sourceware one [2]). > > The prerequites are: > - git > - the bunsen t-upload-git-push script [3], feel free to grab just that > into your $PATH > - sourceware account with bunsendb commit access [ask on > admin-reque...@sourceware.org] > - pretty much nothing else - specifically, gcc commit access is not > necessary! > > Here's how it looks when it's run after a random small gcc build/test: > .../contrib/test_summary -h > [...] > -b: instead of emailing, push test logs into a bunsen git repo > -bg REPO: specify the bunsen git repo to override default > -bt TAG: specify the bunsen git commit tag to override default > > .../contrib/test_summary -b | env PATH=$BUNSEN/INST/bin:$PATH sh -x > + echo master > + echo basepoints/gcc-15-3524-ga523c2ba5862 > + echo a523c2ba58621c3630a1cd890d6db82879f92c90 > + echo git://gcc.gnu.org/git/gcc.git > + find . -name '*.log' -o -name '*.sum' -o -name '.bunsen.*' > + t-upload-git-push ssh://sourceware.org/git/bunsendb.git/ > fche/gcc/x86_64-20240913-1516 > ec57fb8ee928e341d1f0d1b09c1d571fb590bd2b > refs/tags/fche/gcc/x86_64-20240913-1516 > > And here's what that dataset looks like on bunsen a few minutes later: > > https://builder.sourceware.org/testrun/ec57fb8ee928e341d1f0d1b09c1d571fb590bd2b > > If this is of any interest, I'd be glad to hack on this script further > to make it acceptable. >
I pretty much only use `contrib/test_summary` via the `mail-report.log` target in the top-level Makefile; maybe add a `bunsen` target to the top-level Makefile, too, to simplify invoking it? > > [1] https://sourceware.org/bunsen/ > [2] https://builder.sourceware.org/testruns/ > [3] https://sourceware.org/git/?p=bunsen.git;a=blob;f=bin/t-upload-git-push > [4] > https://sourceware.org/git/?p=bunsen.git;a=blob;f=bin/t-sourceware-mails-import > > > diff --git a/contrib/ChangeLog b/contrib/ChangeLog > index 9b36caf02bb1..def8dd8a8a73 100644 > --- a/contrib/ChangeLog > +++ b/contrib/ChangeLog > @@ -1,3 +1,9 @@ > +2024-09-13 Frank Ch. Eigler <f...@redhat.com> > + > + * test_summary: Add -b (bunsen) mode to report all test results > + into a https://sourceware.org/bunsen/ system instead of emailing > + extracts. > + > 2024-08-01 Thomas Schwinge <tschwi...@baylibre.com> > > * gcc_update (files_and_dependencies): Update for > diff --git a/contrib/test_summary b/contrib/test_summary > index 5760b053ec27..867ada4d6b81 100755 > --- a/contrib/test_summary > +++ b/contrib/test_summary > @@ -39,6 +39,9 @@ if test x"$1" = "x-h"; then > should be selected from the log files. > -f: force reports to be mailed; if omitted, only reports that differ > from the sent.* version are sent. > + -b: instead of emailing, push test logs into a bunsen git repo > + -bg REPO: specify the bunsen git repo to override default > + -bt TAG: specify the bunsen git commit tag to override default > _EOF > exit 0 > fi > @@ -57,6 +60,9 @@ fi > : ${filesuffix=}; export filesuffix > : ${move=true}; export move > : ${forcemail=false}; export forcemail > +: ${bunsen=false}; > +: ${bunsengit=ssh://sourceware.org/git/bunsendb.git/}; > +: ${bunsentag=`whoami`/gcc/`uname -m`-`date +%Y%m%d-%H%M`}; > while true; do > case "$1" in > -o) filesuffix=.sent; move=false; : ${mailto=nobody}; shift;; > @@ -64,10 +70,30 @@ while true; do > -p) prepend_logs=${prepend_logs+"$prepend_logs "}"$2"; shift 2;; > -i) append_logs=${append_logs+"$append_logs "}"$2"; shift 2;; > -m) mailto=$2; forcemail=true; shift 2;; > + -b) bunsen=true; shift;; > + -bg) bunsengit=$2; shift 2;; > + -bt) bunsentag=$2; shift 2;; > -f) unset mailto; forcemail=true; shift;; > *) break;; > esac > done > +if [ "x$bunsen" = "xtrue" ]; then > + gitsrcdir=`dirname "$0"` # this script, contrib/test_summary > + gitsrcdir=`dirname "$gitsrcdir"` # and the parent directory > + if [ -d "$gitsrcdir/.git" ]; then # is this a git-hosted source tree? > + # gather basic build metadata for sourceware-buildbot-style .bunsen > data > + gitbranch=`cd "$gitsrcdir"; git rev-parse --abbrev-ref HEAD` > + echo "echo '$gitbranch' > .bunsen.source.gitbranch &&" > + gitdescribe=`cd "$gitsrcdir"; git describe` > + echo "echo '$gitdescribe' > .bunsen.source.gitdescribe &&" > + gitname=`cd "$gitsrcdir"; git rev-parse HEAD` > + echo "echo '$gitname' > .bunsen.source.gitname &&" > + gitremote=`cd "$gitsrcdir"; git config --get remote.origin.url` > + echo "echo '$gitremote' > .bunsen.source.gitrepo &&" > + fi > + echo "(find . -name '*.log' -o -name '*.sum' -o -name '.bunsen.*' | > t-upload-git-push '$bunsengit' '$bunsentag')" > + exit 0 > +fi > : ${mailto="\" address \""}; export mailto > files=`find . -name \*.sum$filesuffix -print | sort` > anyfile=false anychange=$forcemail && >
