bug#42883: Selecting a subset of tests in recursive make causes unnecessary failures

2020-08-24 Thread Karl Berry
Hi Glenn, The example is meant to be such that the test X in root and SUBDIR are tests that would be run when not explicitly specifying tests. Yep, I got that. So its to be able to run any test that would be run when not explicitly specifying tests. Clearly yes, it would be desi

bug#42883: Selecting a subset of tests in recursive make causes unnecessary failures

2020-08-24 Thread Glenn Washburn
On Fri, 21 Aug 2020 16:05:24 -0600 Karl Berry wrote: >> Another issue is that if there is a test X in the root makefile >> and a test X in SUBDIR > > Isn't that fundamentally not supported, or supportable? I admit I > don't actually know if there's any history about it, but using the > same name

bug#42883: Selecting a subset of tests in recursive make causes unnecessary failures

2020-08-21 Thread Karl Berry
at a minimum this behavior should be documented with the test subsets documentation and this work around mentioned. Yes, I agree. Contemplating. Ideally, the subset tests feature design should be updated to play nice with makefile recursion. I think I agree again, but ... not sur

bug#42883: Selecting a subset of tests in recursive make causes unnecessary failures

2020-08-21 Thread Glenn Washburn
Hi Karl, On Wed, 19 Aug 2020 19:17:13 -0600 Karl Berry wrote: > Hi Glenn, > > I noticed that when specifying a subset of tests to run via the > TESTS environment variable, I could have all tests complete > successfully > > Thanks for the report. That does seem suboptimal, but on the other

bug#42883: Selecting a subset of tests in recursive make causes unnecessary failures

2020-08-19 Thread Karl Berry
Hi Glenn, I noticed that when specifying a subset of tests to run via the TESTS environment variable, I could have all tests complete successfully Thanks for the report. That does seem suboptimal, but on the other hand, subsets of tests are a complication. Does it work to override SUBDIRS

bug#42883: Selecting a subset of tests in recursive make causes unnecessary failures

2020-08-15 Thread Glenn Washburn
Hi, I've been working on some automake tests for grub, and ran in to a bug. I noticed that when specifying a subset of tests to run via the TESTS environment variable, I could have all tests complete successfully and yet still have make return an error. Here's a sequence of commands that should r