On Wed, Jan 27, 2016 at 01:30:01PM +0000, Morton, Derek J wrote: > > > > > >-----Original Message----- > >From: Ville Syrjälä [mailto:[email protected]] > >Sent: Wednesday, January 27, 2016 12:33 PM > >To: Morton, Derek J > >Cc: [email protected] > >Subject: Re: [Intel-gfx] [PATCH i-g-t] lib/igt_core.c: Expand --run-subtest > >functionality. > > > >On Wed, Jan 27, 2016 at 10:05:56AM +0000, Derek Morton wrote: > >> Added support for specifying arbitary lists of subtests to run or to > >> exclude from being run by using : or ^ as a seperator. > >> > >> :subtest1:subtest2: Will run subtest1 and subtest2 ^subtest1^subtest2^ > >> will run all subtests except subtest1 and subtest2 > > > >Hmm. Getting a bit complicated perhaps. Would it be simpler to just allow > >specifying the --r option multiple times? So we'd start with the full list > >of subtests, and each --r option would filter the list in some way? > > By -r I assume you mean --run-subtest? Running a test with -r <subtest> just > prints the usage options at the moment.
--r not -r ;) > Allowing it to be added multiple times could be a way of building up a list > of subtests to run, but a completely new command line option would need to > then be added (--skip-subtest?) to allow building up a list of subtests to be > skipped. Or could be something like --r !subtest-name > With my change as is, it allows a string to be passed to the test which > details which subtests should be run. If a new parameter such as > --skip-subtest is added then it would require knowledge of whether the string > contains subtests to run or not to run. That would complicate any scripts > that use this to automate testing. > Allowing --run-subtest (and --skip-subtest) to be specified multiple times > will also complicate the code in igt_core.c. Currently there is a simple call > to strdup(). If --run-subtest can be specified multiple times the code will > have to deal with concatenating strings and any memory reallocation that > needs. Also how to deal with the possibility of multiple wildcard expressions? Hmm. I suppoose my original idea of start with full list and filter stuff out doesn't work entirely well. Eg. --r foo --r bar would run nothing. So I guess the option would have to be able to add to the list as well. So I guess we could make it so that '!' removes the specified test(s), no-'!' adds them, and if this is the first --r option and there's no '!' we'd clear the list entirely before adding the specified test(s) to it. > > I think that will just end up more complicated than the simple separated list > solution this patch introduces. I suppose. But it would avoid adding a new language which can look a bit like a weird regexp but isn't. Maybe if you just use ',' to separate the subtests specifications, and '!' to specify the not condition things would look a bit more standardish. > > //Derek > > > > >> > >> Any subtest string not starting : or ^ is treated as a normal wildcard > >> expression. > >> > >> This is required mainly on android to exclude subtests that test > >> features that do not exist in the android driver while still being > >> able to run other subtests in the binary when a wildcard expression is > >> insufficient. > >> > >> Signed-off-by: Derek Morton <[email protected]> > >> --- > >> lib/igt_core.c | 42 ++++++++++++++++++++++++++++++++++++++++-- > >> 1 file changed, 40 insertions(+), 2 deletions(-) > >> > >> diff --git a/lib/igt_core.c b/lib/igt_core.c index 6b69bb7..b9e7470 > >> 100644 > >> --- a/lib/igt_core.c > >> +++ b/lib/igt_core.c > >> @@ -207,7 +207,15 @@ > >> * To do that obtain the lists of subtests with "--list-subtests", which > >> can be > >> * run as non-root and doesn't require the i915 driver to be loaded (or > >> any > >> * intel gpu to be present). Then individual subtests can be run with > >> - * "--run-subtest". Usage help for tests with subtests can be > >> obtained with the > >> + * "--run-subtest". --run-subtest accepts wildcard characters. A list > >> + of > >> + * subtests to run may be specified by using : as a seperator. A list > >> + of > >> + * subtests to exclude may be specified using ^ as a seperator. > >> + * > >> + * - --run-subtest basic* will run all subtests starting basic. > >> + * - --run-subtest :subtest1:subtest2: will run only subtest1 and > >> + subtest2 > >> + * - --run-subtest ^subtest1^subtest2^ will run all except subtest1 > >> + and subtest2 > >> + * > >> + * Usage help for tests with subtests can be obtained with the > >> * "--help" command line option. > >> */ > >> > >> @@ -786,6 +794,35 @@ void igt_simple_init_parse_opts(int *argc, char > >> **argv, > >> extra_opt_handler, handler_data); } > >> > >> +static bool check_testlist(const char *subtest_name) { > >> + char *p; > >> + > >> + /* Run subtests in list > >> + * Look for subtest_name in list of form :subtest1:subtest2:subtest3: > >> + * return true if found. > >> + */ > >> + if (run_single_subtest[0] == ':') { > >> + p = strstr(run_single_subtest, subtest_name); > >> + if ((p) && (*(p-1) == ':') && (*(p+strlen(subtest_name)) == ':' > >> )) > >> + return true; > >> + } > >> + /* Run subtests not in list > >> + * Look for subtest_name in list of form ^test1^subtest2^subtest3^ > >> + * return true if not found. > >> + */ > >> + else if (run_single_subtest[0] == '^') { > >> + p = strstr(run_single_subtest, subtest_name); > >> + if (!((p) && (*(p-1) == '^') && (*(p+strlen(subtest_name)) == > >> '^' ))) > >> + return true; > >> + } > >> + /* Run subtests that match shell wildcard */ > >> + else if (fnmatch(run_single_subtest, subtest_name, 0) == 0) > >> + return true; > >> + > >> + return false; > >> +} > >> + > >> /* > >> * Note: Testcases which use these helpers MUST NOT output anything to > >> stdout > >> * outside of places protected by igt_run_subtest checks - the piglit > >> @@ -814,7 +851,8 @@ bool __igt_run_subtest(const char *subtest_name) > >> } > >> > >> if (run_single_subtest) { > >> - if (fnmatch(run_single_subtest, subtest_name, 0) != 0) > >> + > >> + if (check_testlist(subtest_name) == false) > >> return false; > >> else > >> run_single_subtest_found = true; > >> -- > >> 1.9.1 > >> > >> _______________________________________________ > >> Intel-gfx mailing list > >> [email protected] > >> http://lists.freedesktop.org/mailman/listinfo/intel-gfx > > > >-- > >Ville Syrjälä > >Intel OTC > > -- Ville Syrjälä Intel OTC _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
