Thanks for the proposal. Some of the general thoughts that I think we could likely agree on, e.g. having a centeralized location for the list of dependencies (pyproject.toml or requirement.txt) It would be useful to think about different use case scenarios and how users use the package.
As of now we can see two different use scenarios: - S0: normal usage case, mixing tvm with the other packages. Normally in this cases loose dependencies are preferred, as we do not want to dictate what version of numpy user want to install, and tvm is supposed to work well with most versions of them. See Pytorch's requirement as an example https://github.com/pytorch/pytorch/blob/master/requirements.txt - S1: single application, where tvm is the sole application running, and there might be desire for precise pin-pointing the version range that are well tested, poetry seems to be a useful tool for this type of development flow. - S2: Development consistency: in CI building, we usually hope to pin the exact version of dependencies, notably, we don't have to do so for all versions, but the useful ones include: pylint, and the frontend importer versions. The first primary goal for us serve the case of S0. In this case, thare are two major ways people use to install packages: - pip: manual installation or venv flow. - conda: very popular among ml scientists. The first thing to consider for us is to have a natural first class flow for conda and pip. Which means a requirements.txt and conda recipe that encodes most of the requirements loosely. Notably because the natural favor of the range dependency `^1.2` in poetry(for S1) and the need for S0. We might want to consider have a first class requirement.txt in the first place. In the mean time, it would be useful to think about whether or not we want to strictly adopt a lock to pin down the CI dependencies. My personally take is that it is too complicated. While a lock file gives exact reproducibility (within the python land), it is not necessarily the platform of choice(in windows conda usually is easier), nor did it pin down all other dependencies(e.g. LLVM versions). For most part of the development we do not have to be that precise, and usually a centralized `ci_constraint.txt` file that specifies the exact version of the CI packages via `==` constraint might be sufficient for most of the cases. We also do not want ot pin down the versions that we think should be loose(e.g. in the case of numpy), so developers who develop not via a sole app purposes can still make use of the constraint. In summary, having centralized location to track dependencies is a good thing (A1 or A2). We want to provide such information to developers who needs them. A1 might be the first step that most of dev environment(conda, pip) can agree on and use and also is the current status quo(e.g. pytorch). It would be usefult to discuss whether we need to go to the extra complexity to use a lock file for CI, my personal take is that it might not be necessary and we can get away with a `constraint.txt` with all the `==` dependencies for our frontend (tensorflow, pytorch) and linters. --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-consolidating-tvm-python-dependencies/8329/3) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/aad0a202f2fab123d03e735382c887a6553a128b466e92d072ffb901a8658fc3).
