On Thu, 2025-07-03 at 10:12 +0100, Joern Wolfgang Rennecke wrote:
>
> On 02/07/2025 18:59, David Malcolm wrote:
> ...
> > Brainstorming some ideas on other possible approaches on making our
> > tests less brittle; for context I did some investigation back in
> > 2018
> > about implementing "optimizations remarks" like clang does:
> > diagnostics
> > about optimization decisions, so you could have a dg directive like
> > this on a particular line:
> >
> > foo (); /* { dg-remark "inlined call to 'foo' into 'bar'" } */
>
> I like the idea. However, it seems unlikely that we can make a
> clean switchover in this decade, unless you find one or more
> corporate sponsors.
> We probably always want dump files without a rigid structure, because
> it makes it easier to add debug output when you flesh out a new pass
> or a change to an existing one. We can make the calls that generate
> the json output also emit output in the dump file, so we won't carry
> a doubled maintenance burden; however, this means the current ad-hoc
> messages would become more unified; thus the testsuite will have to
> be adjusted.
> FWIW, even you you were to get rid of the current dump files (which
> I think would be stifling for GCC development for the above
> reasons), you would have to adjust the testsuite.
> So, we could use the json framework for new dump output that is
> contributed before or along with the parts of the testsuite that
> scan for it, but for any legacy dump output that is scanned for
> in the testsuite, that requires to adjust the testsuite. More
> than 26K dejagnu scan-*-dump* directives in the gcc15 testsuite.
> And you'll have a bit flag day, or a ton of small ones. Plus
> all the friction that this will create with porting patches up
> and down gcc versions.
> That is a lot of thankless work, which I can't imagine doing as a
> hobby. And condifering people at the start of their career who
> might think of doing some unpaid drudge work in hope of getting
> recognition that'll get them some paid work, with paying work
> for GCC drying up, they would more likely do something for LLVM,
> which also seems to better align with the skills of recent
> graduates.
Hi Joern
I don't think the two approaches (dumpfiles vs remarks) are mutually
exclusive: I was thinking of an approach where we extend the existing
dump mechanism so that messages go both to the dumpfiles, *and* are
emitted as remarks, with a new command-line option to enable the latter
sink.
Thus people writing new testcases would have the option of using
remarks to get greater precision about what is being tested, but the
existing testcases continue to work without needing porting, and
dumpfiles can continue to contain ad-hoc information. In particular,
there wouldn't need to be a big "flag day" change, but obviously test
cases using dg-remark would only work for versions of GCC that support
remarks.
>
> So, unless/until you have (a) corporate sponsor(s) to pay for
> the work on the existing testsuite - and that work is
> successfully concluded - we will have to find a way to
> make the scans of the dump files more maintainable.
> In fact, if we can solve the maintenance hassle of having
> multiple in a test by making the scan patterns more specific,
> so we don't have to split the tests up, that will put us in
> better position if/when the transition to a more organized
> optimization records system is made.
>
I can try prototyping something before Cauldron (though I have a fairly
full plate already for GCC 16).
Dave
