> On 13 Aug 2025, at 15:43, Jeff Law via Gcc <gcc@gcc.gnu.org> wrote:
> On 8/13/25 12:53 AM, Richard Biener via Gcc wrote:
>
>> That said, I'd like to move away from gcc-testresults as a vetting
>> tool to something
>> more modern. Possibly a good(?) GSoC project, set up github CI
>> runners for this?
> Yes, it'd be a fine GSoC project. Getting it set up is the easy part, the
> monitoring of results to address regressions is the recurring cost. But I've
> also found it invaluable for testing in-flight work across a variety of
> targets.
One thing that would help a lot with that is to have better flagging of real
regressions c.f. things like line number changes and/or twinkling tests (I know
there’s nothing new in what I’m saying here, but nevertheless when resources
are limited - false positives are a significant drain). I suspect that we can
make some real improvements by properly bracketing test results (e.g. using
Dave’s alternate output formats - or even a simple stupid “start / end test ID”
pair) so that the sorting is stable in the presence of larger numbers of fails
and multilibs.
Iain
