Hi folks I've been looking at a potential large change to the GDAL test suite, replacing some of the worst parts. I thought I'd bring it up for discussion here, in case anyone has any wisdom or strong feelings about it.
I've made a ticket at https://github.com/OSGeo/gdal/issues/949 . Essentially my plan is to use pytest <https://docs.pytest.org/en/latest/> as both a test runner and a helper library. We would remove or replace large parts of the `gdaltest` utilities with pytest equivalents. To give an idea of the main changes, this code: if ds.GetLayer(0).GetSpatialRef() is not None: > gdaltest.post_reason('did not get expected spatial ref') > return 'fail' > would change to: > assert ds.GetLayer(0).GetSpatialRef() is None, 'did not get expected > spatial ref' > The pytest runner gives contextual tracebacks from assert statements, as well as relevant variables and line numbers. Other changes: * The `gdaltest_list` and the `__main__` entry point at the bottom of each file will be removed. Some tests will be renamed if they don't already start with `test_`. * The `<name>_cleanup` fixture will be turned into a module-scoped fixture <https://docs.pytest.org/en/latest/fixture.html#scope-sharing-a-fixture-instance-across-tests-in-a-class-module-or-session>. This allows users to run any number of tests in the module, and the cleanup fixture will be invoked once after all of them are finished. If I find corresponding setup functions I will make them into fixtures too, but I haven't seen those so far in my digging. Other, more intrusive changes are possible but this is what I'm considering targeting as a first step. It is fairly straightforward to automate this change; I have a script in progress already to do that. I intend to make a compatibility shim for the old-style tests that can't easily be automatically updated. The shim would be a decorator, perhaps called `@old_test`, which would take 'skip' or 'fail' return values from tests and turn them into the appropriate `assert False` or `pytest.skip()` calls. I don't intend to make any changes to the C++ tests. A couple of complicating factors to be aware of: - pytest only supports python 2.7+. The tests already only run in 2.7+ in Travis, so no big deal there. I think it's perfectly reasonable to only support 2.7+ these days, given that 2.6 has been unsupported since 2013, but I thought I'd mention it. - pytest has no builtin way of *ordering* tests. It is assumed that tests are independent of each other. In practice, all tests are run in alphabetical name order within a given module. It appears to me that some of GDAL's tests are *not* independent of each other within the same module. However, most are already in alphabetical name order, so I don't believe this is a problem; we are not worse off with pytest than previously. Please let me know your thoughts. -- Regards, Craig de Stigter Developer Koordinates +64 21 256 9488 <+64%2021%20256%209488> / koordinates.com / @koordinates <https://twitter.com/koordinates>
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
