thanks for doing this @lunderberg, i think this will be a very positive change.
I do think it's worth discussing the "how do we use this?" angle more. As a
reviewer, there needs to be some very simple way to decide whether a test needs
to be parameterized or whether it can be allowed to remain unparameterized. It
may be easier to do this by incrementally expanding a subset of the tests that
"require parameterization," rather than starting with a reorganization (I'm not
diminishing the point that the tests are not exactly organized according to
source file, but it may be quite a large project to reorganize everything). I
think a great starting point would be operator tests, where we already have a
need to produce an automated report of our operator coverage.
This also implies that we'll stop allowing tests to be committed with this
syntax:
```
if __name__ == "__main__":
test_foo()
test_bar()
```
and instead require e.g.
```
if __name__ == "__main__":
tvm.testing.main()
```
where tvm.testing.main:
```
import sys
import pytest
def main():
main_file = # use traceback to get calling __file__
sys.exit(pytest.main([main_file] + sys.argv[1:]))
```
I'm very supportive of doing this.
Finally, we need to address test time, which might be more easily done through
incremental implementation. The CI is pretty maxed out right now, and while
we're working to improve this, we can't really address a need for a lot of
added compute immediately. We may need a way to mark tests as "for CI," "for
nightly," etc in the event we split the CI.
---
[Visit
Topic](https://discuss.tvm.apache.org/t/rfc-parametrized-unit-tests/9946/5) to
respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https://discuss.tvm.apache.org/email/unsubscribe/827eb82fe7279684b3d2c2a300ec850488fa500ee915e571cb5cbc5546572eb2).
