> -----Original Message-----
> From: Jakub Jelinek <ja...@redhat.com>
> Sent: Monday, March 17, 2025 12:46
> To: Robert Dubner <rdub...@symas.com>
> Cc: gcc-patches@gcc.gnu.org
> Subject: Re: [PATCH] cobol: Eighteen new testcases to cobol.dg/group1.
>
> On Mon, Mar 17, 2025 at 11:34:36AM -0500, Robert Dubner wrote:
> > > This would be a major problem.
> > > Generally, we want to allow even cross-testing from 32-bit
> > architectrures,
> > > so they'd better fit into 3GB of memory at most.
> >
> > At the present time, the executables cannot be compiled for 32-bit
> > architectures, so for now I would think that is moot.
>
> But they should, at least e.g. in i686-linux -> x86_64-linux etc.
> cross-compilers.
> In any case, usually people test with make -jN check for large N and
> if testing of a single language would need gigabytes of memory that
> would cause problems for the parallel testing.
>
> > A directive that says, "Compare the executable's output to a file"
would
> > be glorious. I have trouble understanding how the project has existed
> > through eight American presidential administrations and the
dissolution
> of
> > the Soviet Union without that capability.
>
> The answer is easy, out of more than 100000 tests in gcc/testsuite/
> only 727 actually look at the output (have dg-output directives) and
> there is 5.5 dg-output directives per test that does use those on
average.
> Most of the testsuite either looks for compilation errors or warnings
> from the compiler or runs tests and just expect they don't abort/crash
> or compiles without static assertion failures etc.
> I/O is not the main thing being tested and even if it is tested, often
> tests use something like sprintf/snprintf or C++ I/O into strings rather
> than file streams.
>
> Jakub
The thing is: I have about 800 testcases with no IP problems that are
based on looking at the output. That's the cumulative result of adding
tests over several years of work.
I fully understand that I am not doing things the way that GCC has been.
I fully understand that any of those tests could be modified to work the
way that GCC grew to implement test cases.
I am not terribly excited at the prospect of rewriting 800 of them.
The reason all that memory is disappearing is because of the length -- not
the complexity, but just the length -- of the regular expressions
generated by those stacked-up dg-output statements. So, the problem crops
up with programs that have a lot of output.
I am going to rework what I did so that it rejects group1 programs with
more than some number of lines of output. That number is to be
determined; I'll do some experimenting. That should address -- you should
excuse the expression -- the memory problem.
And then I'll start work on group2, which is more involved because it
involves parsing and extracting source code and expected output from the
autotest .at files.
Those will be less of an issue because they generate small amounts of
output. But it'll take a little while; group1 was easy.
Bob D.
