It's not practical to ship a C file and
compile that when the user runs the tests (right?).
Right. Also right that we don't want to induce a dependency on
perl/python/whatever for running tests.
(a) not piping stderr through the prefixing function when the prefix
is the empty string, and (b) avoiding stdout buffering when stderr
is prefixed.
Those sound desirable in any case. Especially the first.
It's highly desirable for existing behavior to remain the same, as much
as possible.
These changes might provide good enough results for most users.
I suppose so, since "most users" won't be using the stderr prefixing
since it's never been available before :).
I guess that would fix the problem with the existing
tap-merge-stdout-stderr test, since then there should be no change from
how it currently works.
But what about the new test you wrote? Isn't any test that a) outputs
to both stdout and stderr and b) uses the new merge prefix going to be
subject to the race condition?
Users that require stronger guarantees can wrap their test program
in a custom helper utility rather than rely on Automake's TAP driver
to do the prefixing and merging.
That feels like asking an awful lot of the users. If we have to resort
to that, it sounds like something we should show an example of. And
maybe use for the new test, if that's possible.
Overall, a change that creates a known race condition with no fix feels
bad to me. We can be sure that the people who run automake tests with -j
cranked up will be reporting it, and "there's no fix, just ignore the
failure" isn't an acceptable answer to automated build systems.
Thoughts and suggestions would be appreciated.
Maybe the answer is for the new test to use --no-merge, or otherwise
avoid the race condition by outputting all of stdout and then all of
stderr (with the prefix). Which admittedly somewhat, though not
entirely, defeats the purpose of the new change, but we just can't have
a test that will randomly fail.
In practice, outside the Automake test suite, I would guess that your
alleviations above would often suffice, as you suggest. On the other
hand, inducing race conditions in a package's test just because it
happens to use the new merge prefixing doesn't seem desirable, even if
it would rarely be noticed.
Or am I misunderstanding? Wdyt? --thanks, karl.
