[apache/tvm] Incomplete message given by TVM-0.8 dev when executing parse_module (#7426)
Thanks for participating in the TVM community! We use https://discuss.tvm.ai for any general usage questions and discussions. The issue tracker is used for actionable items such as feature proposals discussion, roadmaps, and bug tracking. You are always welcomed to post on the forum first :) Issues that are inactive for a period of time may get closed. We adopt this policy so that we won't lose track of actionable issues that may fall at the bottom of the pile. Feel free to reopen a new one if you feel there is an additional problem that needs attention when an old one gets closed. For bug reports, to help the developer act on the issues, please include a description of your environment, preferably a minimum script to reproduce the problem. For feature proposals, list clear, small actionable items so we can track the progress of the change. Here is the url of the corresponding problem posted in TVM discussion: https://discuss.tvm.apache.org/t/incomplete-message-given-by-tvm-0-8-dev-when-executing-parse-module/9013 Is it better to regard it as a bug and offer more specific warning message? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/tvm/issues/7426
[Apache TVM Discuss] [Development/RFC] [RFC] Add While loop node to TIR
A PR posted https://github.com/apache/tvm/pull/7425 --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-add-while-loop-node-to-tir/9028/15) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/10504d62fea130c0a2833b47e80fc01b2083de42f27b0508e771bb68c12c0659).
[Apache TVM Discuss] [Development/RFC] [RFC] TVMC: Add support for µTVM
Catching up with all the messages here, it looks like there are two problems and two solutions for the subsets of `tvmc` commands in discussion: **P1:** **How can we create something that allows uTVM to be integrated bare-metal and RTOS?** This is aimed to provide TVM/TVMC as *tools* to be integrated as part of an embedded process. The target user knows about their project (how to build it in the most optimised way, how to flash it on a board, how to debug, etc.) and will fit something we provide in the Model Library Format as part of their workflows. * **S1:** This is solved by using the improving `tvmc compile` to generate a more comprehensive set of outputs in the Model Library Format, so that users can pick what is relevant for them (header files, static object files, boilerplate glue, etc). `tvmc run` has no role in this "uTVM" use case. **P2: How to test/tune operators and models using TVM/TVMC, when aiming at embedded devices?** This relates to TVM developers, that not necessarily know a lot about embedded development, but want to validate and check performance using embedded targets. * **S2:** In this case, we would have the proposed `tvmc micro` namespace, which would be basically aimed for testing and to bridge TVM with already existing tools, provided by each RTOS flavour the community decides to integrate with. This also would need to enhance `tvmc tune`, so that we could support uTVM tuning on this workflow. With all that, I just wanted to check we are talking about the same end-users when we say _users_, and to seeing the problems we are solving them with new commands, and who are the commands aimed at. Do you see any other fundamental problems we are trying to solve with all being discussed in this thread, that is not captured here? --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-tvmc-add-support-for-tvm/9049/13) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/43eeae7bee2c055a1faf0dff16848342cd3f88edea412a6d9d65941776c37586).
[Apache TVM Discuss] [Development/RFC] [RFC] Rename TVMContext to TVMDevice
I'm not worried about the µTVM case especially if that's the only concern. The main thing I'm wondering about is whether it makes sense to let the DLPack RFC get accepted before we make changes here. --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-rename-tvmcontext-to-tvmdevice/9090/9) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/49ee93c5ff37ecceabdc2b2b8b3b2b44d484a593d00bf0ea8b79364af6f3a0d7).
[Apache TVM Discuss] [Development/RFC] [RFC] Python Dependencies in TVM CI Containers
Thanks for the discussions so far. I think everyone seems to agree on T1, it is good to have a way to have a place where users and developers can look into. T2 and T3 are the ones that worth discussing. Trying to summarizing a few rationales of the current CI pipeline. ### R0: Use Binary Tag for Reproducibility and Stablization During testing, the dependency being used goes beyond python dependencies(what a poetry lock file can capture) and will include things like blas version, nnpack code and cuda runtime. **Docker binary image** is still the only reliable source of truth if we want to exactly reproduce the test runs. This is why we opted for the usage of binary tag and stage dependency images during test dependency update. So developers can pick up these tags. ### R1: Manual Fixes are Unavoidable During Dependency Update Changes to APIs like tensorflow, pytorch will likely requires upgrade of the test cases, which means manual fixes are not avoidable. This factor also creates slight favor of building docker images separately, so incremental changes can be made. ### R2: Simple Flow to Build the Docker Image For different down stream users e.g. folks might prefer a different CI flow that directly build the docker image from the source. Having a simple flow `docker/build.sh` to do so is super useful to these points. ### Discussions The above factors might lead to the conclusion that l1 is likely the simplest approach to take. They are also the plan that is immediately actionable. On one hand, l2/l3 indeed captures more python dependencies via a lock, they do not necessarily capture all the dependency needed(see R0). Additionally, l2/l3 might involve checking in a lock file((which cannot be reviewed as we do not immediately know the impact of the change) into the repo that was produced from a file in the repo creating a cyclic-like dependency in the upgrading process(txt file to lock file). The original constraint file(can be approved by a human) and binary tag(can be verified by CI) generally satisfies the two needs(of readability and final pining pointing), and makes R1 generally easier. Additionally, the set of python dependencies can be generated through `pip freeze` in the `ci-gpu` image(or `ci-cpu` image for cpu only packages), which means we still meed our original requirements of tracking ci deps. --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-python-dependencies-in-tvm-ci-containers/9011/6) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/6b078c3d9f0c8a5955da227883cd084adcec76e20d44a0fa7172ff550978).
[Apache TVM Discuss] [Development/RFC] [RFC] TVMC: Add support for µTVM
@manupa-arm @leandron thanks for the reply!
@manupa-arm wrote:
> I was hinting that we could do use export_library on the runtime.Module
> (which is a multi-module hierarchical tree) produced via relay.build using
> the correct cross compiler and target object format as .o/.a.
I agree we should support binary output. My thought is this would depend on the
TVM target in use--but we can talk about it. Either way, Model Library Format
should work with source and/or binary.
> I think its OK to bundle them in the model library as optional/helper sources
> but I think we should not link them in the compiled artifact (.o/.a). Yes, we
> would need a tvmc compile flag to specify the output type. I think we should
> discuss the contents of Model Library in that RFC
Let's discuss this further in the RFC--ideally I'd like to leave anything
specific to integration out, but perhaps there are some generic glue code we
could include. In this particular situation--would it not make sense to treat
`bundle.{c,cc}` as an implementation of the Project API?
@leandron wrote:
> Do you see any other fundamental problems we are trying to solve with all
> being discussed in this thread, that is not captured here?
I think that is a fair summary of the different aspects here, but I'd like to
point out one particular implication of P1:
- by "improving `tvmc compile` to generate a more comprehensive set of outputs
in the Model Library Format," we are also proposing that the current µTVM
compilation flow be split into two pieces: 1) generating Model Library Format
and 2) building a project which can be compiled. This solves a fundamental
problem of how to integrate `tvmc` with a variety of projects without adding
all of their dependencies to TVM's dependency list.
Finally, I'd say that depending on how we choose to implement Model Library
Format, we should consider whether it should just become _the_ output format
from `tvmc`--but let's discuss that on the RFC.
---
[Visit
Topic](https://discuss.tvm.apache.org/t/rfc-tvmc-add-support-for-tvm/9049/14)
to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https://discuss.tvm.apache.org/email/unsubscribe/cd67553670c4da3384b2266823b0572723c0b1cd0f1e6c20522941593aeb484e).
[Apache TVM Discuss] [Development/RFC] [RFC] TVMC: Add support for µTVM
> I agree we should support binary output. My thought is this would depend on
> the TVM target in use–but we can talk about it
Yeah I think so too. Like, I won't be good to always generated a .so and a .o
(for example) for the static runtime case, which afaics will only be
"interested" in the .o to wrap it using the "adapters" (`bundle.{c,cc}`) to
create a .so (dynamic case) or use link all the objects in a static executable
(static case). In both cases I understand `tvmc compile` should not generate a
`.so`.
Other use-cases, like for applications dlopen'ing the model library (.so)
generated by the `tvmc compile`, it would be good to have .so as an output for
command `compile`.
> –would it not make sense to treat `bundle.{c,cc}` as an implementation of
> the Project API?
Yep, I'm wondering it too. It makes sense to have "adapters" like that
implemented like a kind of project, so use the mechanism of the Project API.
---
[Visit
Topic](https://discuss.tvm.apache.org/t/rfc-tvmc-add-support-for-tvm/9049/15)
to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https://discuss.tvm.apache.org/email/unsubscribe/d233b77cfe6d3bbeafedeba76ec9cb39542aa29ab435e1169d0fd3aeb984be36).
[Apache TVM Discuss] [Development/RFC] [RFC] Rename TVMContext to TVMDevice
I agree with the change, internally it would be nice to drop the TVM prefix as well and just refer to it as a `tvm::Device` like we do with almost every other data structure in the code base. Thanks for doing this Haichen! --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-rename-tvmcontext-to-tvmdevice/9090/10) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/01c560a579036ef54304d4d3003522d1065e84d304bc96fb8b6edc2ee7a54bfb).
[Apache TVM Discuss] [Development/RFC] [RFC] Rename TVMContext to TVMDevice
My take on dropping the TVM prefix: given DLTensor/DLContext is the de facto standard, we do not really need to define their TVM alias (e.g. TVMContext) and can just use them directly --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-rename-tvmcontext-to-tvmdevice/9090/11) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/c0adf6f9431c79b2450b553c942b31599ca1cf0e62a828baeda3b2b9259b8a21).
[Apache TVM Discuss] [Development/RFC] [RFC] Rename TVMContext to TVMDevice
I think we can keep `TVMDevice` in the C++ and backend, but use `tvm.device` in the python frontend. It can reduce the confusion when integrating TVM into other frameworks if we keep `TVM` prefix. --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-rename-tvmcontext-to-tvmdevice/9090/12) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/2cd4afc627b6282763672e39f3cbe950f0fd0e7c34a5c57b2695a99b35f317d0).
[Apache TVM Discuss] [Development/RFC] [RFC] Python Dependencies in TVM CI Containers
@tqchen Thanks for the reply! A couple of points: - You're right that I2/I3 don't capture all dependencies. There is a notable gap between a set of Python dependencies and the docker container: the remaining libraries which may be present in the container. - The method by which those libraries are installed is platform-dependent, but so far our approach for the CI has been mostly to depend on a package manager to satisfy them. - I think that even if a lockfile can't be used primarily to reproduce the test environment, there are plenty of other uses--here are a couple: - Determining which version of a C library is suitable when trying to install TVM on a system other than the docker containers. You might prefer to at least know the version of the Python package which uses it. I had trouble finding a non-dev dependency like this but I think this pattern is very common, and [Pillow](https://pillow.readthedocs.io/en/latest/installation.html#external-libraries) is an example of such a package. - Often times a user isn't trying to exactly reproduce the regression--they just want to know which minor version of a package to install e.g. tensorflow 2.1 vs 2.2. This can make a big difference, especially when the main problem is API incompatibilty.. Even if a user might not 100% be able to reproduce the regression, it's often the case that if they're using TVM in an application, they won't be able to use the TVM CI containers as a starting point. ### Updating Docker Containers I think it's worth thinking through the case of updating the docker containers a little more because I'm not sure I follow your reasoning here. The current process is as follows, per-container: 1. update docker/install and Dockerfile as needed to modify the container 2. run `docker/build.sh` to build the new container image 3. Fix problems and rerun steps 1 & 2 as necessary. 4. test the containers -- discussed later The current build process is quite straightforward--it's just building a docker container. I2 and I3 propose to add these changes: 1. prior to building the containers, create a lockfile 2. rebuild all containers when the lockfile changes, rather than just updating one Change 2 shouldn't significantly burden developers unless the build process is error-prone. We should fix that if it is, and perhaps a nightly container build isn't a bad Jenkins job to configure so we can address these failures as they happen. Change 1 is the one that needs to be thought through. Ordinarily, building a lockfile is running a single command on the target platform, and often this process takes about a minute. This would not be a significant burden. The tricky part is: we have multiple platforms in the CI (three, in fact: manylinux1_i686, manylinux1_x86_64, and manylinux_arm) and it's not possible to build the lockfile for a platform other than the one you're running on. In thinking this through again, I realized that actually this is the same problem as the one we have with Windows: it's not possible to use a single lockfile for all platforms. If you are switching platforms, you are going to have to accept you may need to switch dependency versions. So let's return to the ultimate goals of I2/I3 approach: 1. Ensure all features of TVM can actually be used simultaneously on a given platform. 2. Standardize dependencies so that the unittests don't fail for different reasons on different containers. 3. provide enough input to allow the `tlcpack` build process to place narrower constraints on `install_requires` included there, based on the versions used in CI In light of the above, all of these goals can be achieved only within a single pip platform. So I'd propose a new way to install dependencies while addressing this, I4: I4. A multi-part approach: 1. For each pip platform that has a ci- container built around it: produce a lockfile either by: 1. `pip install -r` all TVM requirements and `pip freeze` 2. `poetry lock` The lockfile becomes part of the artifacts created when the containers are built. When multiple containers are built for the same pip platform (e.g. ci-lint, ci-cpu, ci-gpu, ci-qemu, ci-wasm), pick one container which will build the lockfile. The rest will merely consume this lockfile when installing packages either as pip constraints file or poetry lockfile. 2. I would argue that between platforms, the version standardization we care about is effectively TVM direct Python dependencies to the minor version level. For instance, if we add Windows tests, we prefer to test against e.g. tensorflow 1.6 on both linux and Windows. To address this, when building containers, we would first build a pilot platform e.g. `manylinux1_x86_64`, scan the resulting lockfile for direct dependencies of TVM, and produce a loose constraint file for the remaining platforms. For example, if the lockfile contained an entry `tensorflow==1.6.8`, we may add `tensorflow>=1.6,<1.7` to the cr
[Apache TVM Discuss] [Development/RFC] [RFC] Python Dependencies in TVM CI Containers
I'm open to changing `build.sh` to reference the previously-used lockfile, which should make it possible to update containers independent of the Python deps used. --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-python-dependencies-in-tvm-ci-containers/9011/8) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/8728592c048d6b2bbd5ef3142503a3ba278388381de3a66e4015acc9c58cd8c9).
Re: [apache/tvm] [DEV] TVM v0.7 Roadmap (#4845)
"Which are summarized summarized from the forum discussion." double summarized may be a small mistakes. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/tvm/issues/4845#issuecomment-776379632
Re: [apache/tvm] [DEV] TVM v0.7 Roadmap (#4845)
Closed #4845. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/tvm/issues/4845#event-4312506615
Re: [apache/tvm] [DEV] TVM v0.7 Roadmap (#4845)
v0.8 cycle tracking issue is created https://github.com/apache/tvm/issues/7434 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/tvm/issues/4845#issuecomment-776396641
[Apache TVM Discuss] [Development/RFC] [DISCUSS] TVM v0.8 Roadmap
[quote="lsy643, post:4, topic:8139"] VM [/quote] Sorry for acting a bit late on this thread. It slipped through due to the conference and holiday break. Thankfully a lot of things are already under way. A formal tracking thread is created here https://github.com/apache/tvm/issues/7434 --- [Visit Topic](https://discuss.tvm.apache.org/t/discuss-tvm-v0-8-roadmap/8139/10) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/58fa097018af3c699f284f101046f5c18146f8b4454b8d56f205293c9b3dd29d).
