Re: [dmlc/tvm] [Relay][RFC] Improve testing infrastructure of Relay (#2884)
+1 for refactoring. BTW, we probably also need to have some discussion about adding some regression tests in CI pipeline because some passes could noticeably affect perf. But this can be a separate issue. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/2884#issuecomment-475924539
Re: [dmlc/tvm] [RFC] Decompile TensorFlow Control Flow Primitives to Relay (#2812)
Closed #2812. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/2812#event-2224962041
Re: [dmlc/tvm] [RFC] Decompile TensorFlow Control Flow Primitives to Relay (#2812)
closed by #2830 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/2812#issuecomment-475979226
Re: [dmlc/tvm] [VOTE] Apache Transition Plan (#2973)
+1 -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/2973#issuecomment-481032819
Re: [dmlc/tvm] [Vote] Deprecate Python2 Support (#2994)
+1 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/2994#issuecomment-481453561
Re: [dmlc/tvm] [RFC][DISCUSS] Tuple-related Fusion (#3039)
@junrushao1994 > @tqchen where is %2? There might be some code emitted, but the idea is to the problem when dealing with duplicate values in return tuples. > why is the example bad for codegen The output tensor is scheduled twice in compute_engine here: https://github.com/dmlc/tvm/blob/552d4aa3587aa3a0443d050ceb324386489ff793/src/relay/backend/compile_engine.cc#L128 There are some problems and discussions in #2932 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3039#issuecomment-484313254
Re: [dmlc/tvm] [RFC][DISCUSS] Tuple-related Fusion (#3039)
@masahi Can we prevent from passing duplicated tensors instead? It looks that we otherwise need to change all schedules for all targets in topi, right? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3039#issuecomment-484340450
Re: [dmlc/tvm] [RFC][DISCUSS] Tuple-related Fusion (#3039)
BTW, I am not certain that stopping fusing return tuple will fully solve the problem because it looks to me that we will still have two identical tensor in the tuple, right? Am I missing something? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3039#issuecomment-484376563
Re: [dmlc/tvm] [RFC][DISCUSS] Tuple-related Fusion (#3039)
@masahi I see, thanks. Another option is probably using a copy operator if there are duplicates. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3039#issuecomment-484385353
[dmlc/tvm] [RFC][Relay] Port Relay passes to pass manager (#3146)
# Porting Relay Passes to Pass Manager
As the pass manager framework has been merged, we should start to move passes
to the pass manager. This RFC proposes the plans to move the Relay passes.
## Proposal (take constant folding as an example):
The proposal needs to solve problems from both the backend and the frontend. As
this could be a lot of refactoring work, we will mainly focus on the API
changes of the pass functions first, but leave others (like resolving
dependency) as follow-up work.
### C++ Backend
The C++ backend implements the functionality of a certain pass as the
`pass_func` for `module_pass` and `function_pass`, i.e. constant folding in
this example. Some passes are passed with a `Module` to identify global vars
and/or help report errors, but others are not.
1. Pass `Module` to every pass, or have a global `Module`? When `Module` is not
passed, we can call `GetModule`, like `FromExpr`. For backward compatibility,
we can default it to null. Therefore, users can still invoke the pass as
before. All frontend converters should return a module as well.
2. Previously, many passes are not aware of PassContext. We probably should
pass PassContext into a pass so that the internal context, e.g. ErrorReporter,
can be used by each pass.
3. How can a pass provide the frontend with the most convenient API so that it
could be invoked easily?
```c++
class ConstantFolder : public ExprMutator {
private:
PassContext pass_ctx;
// Other members are omitted..
};
Expr FoldConstant(const Expr& expr, const PassContext& ctx) {
return ConstantFolder(CreateInterpreter(...), ctx).Mutate(expr);
}
TVM_REGISTER_API("relay._ir_pass.FoldConstant")
.set_body([](TVMArgs args, TVMRetValue *ret) {
PassContext ctx = args[1];
if (!ctx.defined()) {
ctx = PassContext();
}
*ret = FoldConstant(args[0], ctx);
});
```
### Python frontend
Python frontend performs the following tasks
- Link to a pass function, i.e. `ir_pass.fold_constant`
- Register the pass using flattened pass info, e.g. `opt_level`, `name`, and
`required` passes.
- Setup `PassContext` when necessary. Otherwise, the backend will create it.
- Create a module for pass execution if it is not passed.
- Have an API to invoke pass functions in different manner?
- The old style invocation, `mypass(mod/expr)`, depending on the type
of pass, e.g. `ir_pass.fold_constant(expr)`
- Through pass manager
```python
# Create a module for all passes?
mod = relay.Module({})
# Method 1, Use decorator:
@ir_pass.function_pass(opt_level=opt_level, name=pass_name)
def mypass(expr, ctx):
class Transform(ExprVisitor):
def __init__():
def __call__(expr, ctx):
return Transform(expr, ctx)
mypass(mod)
# Method 2, pass registered passes directly:
pass_func = ir_pass.fold_constant
cf = ir_pass.function_pass(pass_func, opt_level=2, name=”FoldConstant”,
required=[])
cf(mod)
```
## Next Steps
- One pass may call another pass directly. We should setup the `required` field
and resolve dependencies. After that, this pass should be able to access the
returned analysis/optimization results.
- How to represent a `Pass` using pass info, particularly pass name. To resolve
pass dependency, we may only have the `required` passes. We then need to create
theses pass and execute them.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3146
Re: [dmlc/tvm] [RFC][Relay] Port Relay passes to pass manager (#3146)
For anyone who is interested in this, please comment. We appreciate your thoughts and suggestions. @MarisaKirisame and I will start working on it once we get some cycles. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3146#issuecomment-491479253
Re: [dmlc/tvm] [RFC][Relay] API Naming in Pass Manager (#3202)
Why `transform` is a better namespace than `pass`? I am fine with `Sequential` as it is also used by Pytorch. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3202#issuecomment-493266455
Re: [dmlc/tvm] [RFC][Relay] Pass API Discussion (#3202)
Sounds like a good plan. I think main is currently used as the entry function. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3202#issuecomment-493577403
Re: [dmlc/tvm] [RFC][Relay] Feature Manager (#3236)
Yes, I agree this is annoying. It looks we might need to introduce some metadata for a pass. Usually when we do sequential passes, we may need to consider about preserving information from the updated passes and also validate if we can proceed. We should think about it more when we start resolving pass dependency and tackling phase ordering. How do you guys think? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3236#issuecomment-495437073
Re: [dmlc/tvm] [RFC][Frontend] Return module for Relay frontend converter (#3346)
Yeah, this is exactly what I think as well. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3346#issuecomment-501072122
Re: [dmlc/tvm] [RFC][Frontend] Return module for Relay frontend converter (#3346)
@icemelon9 #3353 was the draft I haven't finished yet. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3346#issuecomment-501150175
Re: [dmlc/tvm] [RFC][Quantization] Designing and lowering of quantized ops (#3512)
Is this ready for review? Have we been converged on the design in the quantization RFC? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/pull/3512#issuecomment-509430172
Re: [dmlc/tvm] [RFC][relay][vm] Relay virtual machine serialization (#3594)
@icemelon9 Yeah, thanks. Putting the `length` before the filed with variable length seems reasonable. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3594#issuecomment-513987982
Re: [dmlc/tvm] [RFC][relay][vm] Relay virtual machine serialization (#3594)
@MarisaKirisame I think we need it to make deserialization easier. Otherwise, we may need many checks. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3594#issuecomment-513988425
Re: [dmlc/tvm] [RFC][relay][vm] Relay virtual machine serialization (#3594)
@icemelon9 we probably don't need to have the length for each field with variable length because we should be able to derive it based on the fixed fields? It means we usually put the length of it as a field of an instruction, right? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3594#issuecomment-513995521
Re: [dmlc/tvm] [RFC][relay][vm] Relay virtual machine serialization (#3594)
@icemelon9 The length is mainly for sanity check before we decode the instructions. We could remove it. There could be multiple fields with variable length. I thought we should always have a field in the fixed field to indicate the length of the variable one, is this right? For example, https://github.com/dmlc/tvm/blob/master/src/runtime/vm/vm.cc#L272 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3594#issuecomment-513998899
Re: [dmlc/tvm] [RFC][relay][vm] Relay virtual machine serialization (#3594)
@tqchen My bad. The APIs started with `Serialize` and `Deserialize` are actually not exposed. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3594#issuecomment-514012903
Re: [dmlc/tvm] [RFC][relay][vm] Relay virtual machine serialization (#3594)
I feel heterogenous execution will be mainly related to memory management in the VM. We don't need to encode any information in VM for the compilation and codegen. I think we probably need to handle `AllocTensor` a little differently, e.g. making it device aware. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3594#issuecomment-515582713
Re: [dmlc/tvm] [RFC][relay][vm] Relay virtual machine serialization (#3594)
The serialization itself doesn't have much to do with quantization. If quantized model needs new opcode in the VM, we need to introduce them first and then extend the serialization/deserialization to support these instructions. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3594#issuecomment-516085721
Re: [dmlc/tvm] [RFC][relay][vm] Relay virtual machine serialization (#3594)
#3647 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3594#issuecomment-516925996
Re: [dmlc/tvm] [RFC][relay][vm] Relay virtual machine serialization (#3594)
Closed #3594. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/3594#event-2523847252
Re: [dmlc/tvm] [DEV] TVM v0.6 Roadmap (#2623)
# TVM Monthly - August 2019 https://discuss.tvm.ai/t/tvm-monthly-august-2019 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/2623#issuecomment-527019813
Re: [dmlc/tvm] [RFC][RUNTIME] Introduce new object protocol. (#4115)
Merged #4115 into master. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/pull/4115#event-2712622451
Re: [dmlc/tvm] [RFC][RUNTIME] Introduce new object protocol. (#4115)
@tqchen thanks. This is now merged. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/pull/4115#issuecomment-542047511
Re: [dmlc/tvm] [VOTE] Add "Organizations contributing using and contributing to TVM" Section to Community Webpage (#4162)
+1 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/dmlc/tvm/issues/4162#issuecomment-544302489
Re: [apache/incubator-tvm] [VOTE] Release Apache TVM (incubating) v0.6.0.rc2 (#4443)
+1 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/4443#issuecomment-559613402
Re: [apache/incubator-tvm] [RFC] Enhance TensorFlow Frontend Control Flow Support (#4969)
Just a reminder, to support these models we need some patches for tensor array as well. mask_rcnn seems requiring some more debugging. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/4969#issuecomment-592675406
Re: [apache/incubator-tvm] [RFC] Enhance TensorFlow Frontend Control Flow Support (#4969)
ahh, one more reminder, for all these models, we will have OOM problem for pretty printing after the ANF pass. It is very likely because recursively visiting the AST saves all the intermediate results. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/4969#issuecomment-592677784
Re: [apache/incubator-tvm] [RFC] Enhance TensorFlow Frontend Control Flow Support (#4969)
Yes, @yongwww had one months back https://github.com/apache/incubator-tvm/pull/4312 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/4969#issuecomment-592746247
Re: [apache/incubator-tvm] [DISCUSS][RFC] Apache TVM Graduation (#6299)
+1 Looking forward to the continued success after graduation. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/6299#issuecomment-676823099
Re: [apache/incubator-tvm] [VOTE] Apache TVM Graduation (#6332)
+1 (binding) -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/6332#issuecomment-679418594
Re: [apache/incubator-tvm] [RFC] v0.7 Release Planning (#6421)
Yeah, I also prefer to document it instead of throwing many warnings. In addition, we have some checker in the codebase claiming that some APIs will be deprecated in the next release. We probably want to take some actions on them as well. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/6421#issuecomment-689079996
[apache/incubator-tvm] [COMMUNITY] lhutton1 -> Reviewer (#6461)
Please join us to welcome @lhutton1 as a new reviewer. He has been actively contributing to bring-your-own-codegen (BYOC), ConvertLayout, and integrating the Arm Compute Library into TVM. He also helped review BYOC and Relay pass PRs. - [Commits History](https://github.com/apache/incubator-tvm/commits?author=lhutton1) - [Code Review](https://github.com/apache/incubator-tvm/pulls?utf8=%E2%9C%93&q=reviewed-by:lhutton1) - [Community Forum Summary](https://discuss.tvm.ai/u/lhutton1/summary) You can view, comment on, or merge this pull request online at: https://github.com/apache/incubator-tvm/pull/6461 -- Commit Summary -- * [COMMUNITY] lhutton1 -> Reviewer -- File Changes -- M CONTRIBUTORS.md (1) -- Patch Links -- https://github.com/apache/incubator-tvm/pull/6461.patch https://github.com/apache/incubator-tvm/pull/6461.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/pull/6461
[apache/incubator-tvm] [COMMUNITY] Zhi's key for ASF release (#6554)
cc @tqchen @ZihengJiang You can view, comment on, or merge this pull request online at: https://github.com/apache/incubator-tvm/pull/6554 -- Commit Summary -- * Zhi's key for ASF release -- File Changes -- M KEYS (56) -- Patch Links -- https://github.com/apache/incubator-tvm/pull/6554.patch https://github.com/apache/incubator-tvm/pull/6554.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/pull/6554
Re: [apache/incubator-tvm] [RFC] v0.7 Release Planning (#6421)
@comaniac cool, thanks. We plan to make a cut tomorrow. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/6421#issuecomment-702523862
Re: [apache/incubator-tvm] [VOTE] Release Apache TVM (incubating) v0.7.0.rc0 (#6622)
+1 (binding) - Checked the signature and hash - The code compiles - Checked LICESE and NOTICE -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/6622#issuecomment-703743085
[apache/incubator-tvm] [COMMUNITY] areusch -> Reviewer (#6637)
Please join us to welcome @areusch as a new TVM reviewer. Andrew has been actively contributing to uTVM, on-device RPC server, and various runtime changes. He proposed the roadmap for uTVM and presented the work at the online meetup. Andrew has also been very actively sharing his thoughts at the discussion forum. - [Commits History](https://github.com/apache/incubator-tvm/commits?author=areusch) - [Code Review](https://github.com/apache/incubator-tvm/pulls?utf8=%E2%9C%93&q=reviewed-by:areusch) - [Community Forum Summary](https://discuss.tvm.apache.org/u/areusch/summary) You can view, comment on, or merge this pull request online at: https://github.com/apache/incubator-tvm/pull/6637 -- Commit Summary -- * [COMMUNITY] areusch -> Reviewer -- File Changes -- M CONTRIBUTORS.md (1) -- Patch Links -- https://github.com/apache/incubator-tvm/pull/6637.patch https://github.com/apache/incubator-tvm/pull/6637.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/pull/6637
Re: [apache/incubator-tvm] [RFC][VM] Heterogeneous execution in Relay VM (#4178)
Closed #4178. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/4178#event-3856212990
Re: [apache/incubator-tvm] [RFC][VM] Heterogeneous execution in Relay VM (#4178)
ahh, thanks for reminding. This is closed by #6337 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/4178#issuecomment-705664905
[apache/incubator-tvm] [COMMUNITY] junrushao1994 -> committer (#6719)
Please join us to welcome Junru Shao(@junrushao1994) as a new Committer. Junru has been actively contributing to various aspects of the TVM codebase. He reimplemented and refactored the Target system which greatly helped code lowering and code generation. Junru also largely contributed to the runtime and node objects to improve the object passing. From the frontend side, Junru has also implemented some Relay operators. In addition, Junru has been very actively sharing his thoughts in the discussion forum to help improve the design of new features and provide new developers with better understanding of TVM. He has been constantly providing suggestive comments to the pull request. - [Commits History](https://github.com/apache/incubator-tvm/commits?author=junrushao1994) - [Code Review](https://github.com/apache/incubator-tvm/issues?q=reviewed-by+%40junrushao1994) - [Community Forum Summary](https://discuss.tvm.apache.org/u/junrushao1994/summary) You can view, comment on, or merge this pull request online at: https://github.com/apache/incubator-tvm/pull/6719 -- Commit Summary -- * [COMMUNITY] junrushao1994 -> committer -- File Changes -- M CONTRIBUTORS.md (1) -- Patch Links -- https://github.com/apache/incubator-tvm/pull/6719.patch https://github.com/apache/incubator-tvm/pull/6719.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/pull/6719
Re: [apache/incubator-tvm] [Relay][RFC] Support layout altering for heterogeneous compilation (#2566)
Thanks for reminding. I think we should probably close this for now. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/2566#issuecomment-720120936
Re: [apache/incubator-tvm] [Relay][RFC] Support layout altering for heterogeneous compilation (#2566)
Closed #2566. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-tvm/issues/2566#event-3945389904
Re: [apache/tvm-rfcs] [RFC] Relax Upstreaming (PR #89)
Based on my experience at several organizations, dynamic shape support is obviously very important, particularly along with the popularity of large language models. Also, efficiently supporting dynamic shape would be one of the major appealing features of a "modern" DLC. I think the above comments have also reached the agreement of importance of dynamic shape. The major argument is whether we need to have separate PRs to land this feature. IMHO, Relax is already one of the components of Unity, and the current proposal again only contains the most value part of Relax which provides a minimum E2E compilation flow to enable the support of a dynamic model. This somehow has been working well before in both TVM and other open source project since the component doesn't blocking/breaking the current uses/deployment. For example, the first version of Relay also had IR, simple lowering, and necessary passes to quickly unblock the users/developers (e.g. AWS) who want to give it a try. Afterwards, we iterated on it many times to improve both the design and implementation. As a contributor of TVM, I would encourage we focus more on the design itself and spot the design flaws and the missing key features that we should address so that users (some of them are already waiting for Relax as mentioned here) can quickly check it out and bring us back with more insightful feedback or directly contribute to the project. -- Reply to this email directly or view it on GitHub: https://github.com/apache/tvm-rfcs/pull/89#issuecomment-1272328462 You are receiving this because you are subscribed to this thread. Message ID:
Re: [apache/tvm] [VOTE] Clarify Community Strategy Decision Process (Issue #15521)
+1 -- Reply to this email directly or view it on GitHub: https://github.com/apache/tvm/issues/15521#issuecomment-1674098084 You are receiving this because you are subscribed to this thread. Message ID:
Re: [apache/tvm] [VOTE] Transition Main to Unity (Issue #16368)
+1 -- Reply to this email directly or view it on GitHub: https://github.com/apache/tvm/issues/16368#issuecomment-1882043633 You are receiving this because you are subscribed to this thread. Message ID:
Re: Test to see if podling is listening
ACK On Thu, Aug 27, 2020 at 5:53 PM Henry Saputra wrote: > Hear ya > > On Thu, Aug 27, 2020 at 10:37 AM Dave Fisher wrote: > > > This is a test message to see if the project is listening on the dev@tvm > > mailing list or is treating this only as an archive. > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@tvm.apache.org > > For additional commands, e-mail: dev-h...@tvm.apache.org > > > > >
[TVM Discuss] [Development/RFC] Bring Your Own Codegen to TVM
# Bring your own codegen to TVM + Graph Partitioning
The goal is to come up with a right Relay subgraph data structure/abstraction
so that we can more conveniently allow thrid-party library and hardware vendors
to bring their own codegen tools to TVM.
This RFC involves design and implementation in the following aspects at least.
* Graph coloring
* Providing HW vendors an infra to customize where they want to execute an op.
* Graph partitioning
* A Relay pass that partitions a program into segments that could be executed
on various hardware platforms.
* Code generation
* Generate code for each segment of a partition Relay program.
* Artifact serialization
* Provide functionality to support save/load of the compiled artifacts.
* Runtime
* Integrate other runtimes/execution engines or invoke the external library
code/subgraph through both graphruntime and VM (the current POC implementation
is using VM).
### Model Coverage
* CNN: MLP, VGG, ResNet, SqueezeNet, Inception V3, etc.
* CV: SSD with ResNet 50, MobileNet, VGG-16, etc.
* NLP models are not supported well yet in Relay so we will revisit them in the
future.
* And more...
### Coloring - Group nodes with the annotation to a minimum Number of subgraphs.
* Problem Formulation
* Input
* Given a Relay graph with extern op annotations (added by users or by some
internal mechanisms).
* The I/O of each node (op) may or may not have annotations to indicate if
this node is suggested to be offloaded.
* Output
* A graph with minimum annotations on edges indicating the boundary of
subgraphs.
* Implementation 1: Op-level annotation
* For each op, we have a corresponding check function registered and the
checker will be invoked at the compilation time to indicate if we should
annotate the op for the 3rd party accelerator to offload. For example, the
following shows a checker of `conv2d` :
* `@reg.register_extern_op("nn.conv2d")def conv2d(attrs, args, comp):
return get_extern_op(comp, "conv2d")(attrs, args)`
* Note that `comp` is a string to represent the 3rd party compiler name;
the `get_extern_op` uses `hasattr` and `getattr` to obtain the 3rd party
specified checkers.
* For HW partners/3rd party library, they only need to implement simply
checker functions for each op to specify if they could support an op under
certain conditions. The following example shows a case that the accelerator
only supports `conv2d` with floating types.
* `def conv2d(attrs, args):type = args[0].output_type_.dtypereturn
(type == 'float32' or type == 'float64')`
* Note that HW partners do not need to register this function but just need
to implement it under Relay backend/contrib/compiler_name so that the function
can be discovered and imported dynamically.
* A Relay IR pass in Python will invoke above function, insert annotations to
the graph, and run Algorithm 1 for coloring.
* Implementation 2: Subgraph-level annotation
* We also provide an option for HW partners to annotate the graph directly.
In this case, they have to implement a Relay IR pass with a use of our APIs to
annotate boundary annotations (i.e., `subgraph_start` and `subgraph_end` ).
### Partitioning - Check/Validate the graph and process graph I/Os.
* Problem Formulation
* Input
* Given a Relay program with boundary annotations (i.e., `subgraph_start`
and `subgraph_end` ).
* The boundary annotations can be added by the coloring stage. In this
case, the boundary annotations are always valid.
* Users can directly add boundary annotations to their Relay programs. In
this case, we need to validate the annotations before partitioning.
* Output
* The updated Relay program with subgraphs replaced with sub functions. All
annotations should be removed and calls should be inserted to invoked the sub
functions.
### Codegen - To tell the Relay backend to use external codegen instead of TVM.
* Invoke different codegen tools from TVM directly. **This needs HW partners to
register their codegen tool to TVM as a runtime module.**
* During compilation, we can traverse the graph and check the attributes of
different subgraphs. For example, an external codegen tool has to be invoked
once we found that the attribute of subgraph is annotated with an external
compiler. For the example above, we can generate a runtime module for 1x1 conv,
but we have to invoke external compilers to generate code for the two subgraphs.
* How to register?
* **HW vendors need to register their compiler as a runtime module and at
least be able to deal with the following tasks**
* Ingest a Relay function/module and compile it.
* Ingest TVM input data structures, e.g. **NDArray.** TVM feeds data in
the NDArray format to the subgraph and expects the external accelerator to
execute it and return the output in the **NDArray** as well. Therefore, HW
vendors will need consider the conversion of TVM data to whatever da
[TVM Discuss] [Development/RFC] Bring Your Own Codegen to TVM
@jonso Thanks for making these points and I am very glad to work together. Most of questions are answered by @comaniac. One thing is that putting extern in the target string might not be sufficient because 1) we need to change the way how target is parsed now, 2) what if there are multiple targets invoked? This may not quite possible for now. Directly adding an `extern_target` option in build of build_config might be simpler. This is the tutorial we have for now: http://tracking.discuss.tvm.ai/tracking/click?d=Qc7VGIU-YEgEC_WCG464y8V7_CE8rOsGU9rjYbzs0zOIOLThuqYZ3GiIiD7kzkw1C_y7QkbgWUccvkxZuaZrrMNSvs6ffO1b8Zx7Vlygrb7i-vLFI-mvZkOeGZzgsZMbSE29UN6dSgxODuBug42W6sIBzxDOUlfNX_ZiI8OkeSWC4VreIQVZafnkOTwUoMj_cn6lZKd3l7il9bENloYHNeo1 I plan to have an iteration on it to clean the branch and integrate the points we agreed here recently once the memory PR is merged. --- [Visit Topic](http://tracking.discuss.tvm.ai/tracking/click?d=2zHJ-oLuPzy-Sp0s1DTQwv6ZdHOSJcD8Hf0pAjYlbVALR8i8RXQQV7sOji-PiTDWHxn4bTA2GKvmmV4egbtEqGAz9DlnLSCmOztehPuju0Bk49KEdshjKQ9qddD6SMtbZi67cgExIX-U9TEoe1b3u2N3rtug3mqlnEC6gsEr5Adl0) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](http://tracking.discuss.tvm.ai/tracking/click?d=7cFgOaAA4XIBVlVKt_oyC07uihTjg4Q6cjeBRNRTiPr97E1gx5nxDe8Lm6V7P_ua0eF4ElESU4pk4ryGbqpHZJshxo3fOwAdOjUOkx40PmkvnGJLe6iE599ntEvxvvP0C8uSfWeHzIFaLuQzGV1r6S9DtA0FhdQDF3Cs950f__Aeu4ZKwphCpmzaqxd8AEmRWniAqids85tmmoUKpOPW9yDcgTZ7tKaUc7LdhlZBg8ho0). Tianqi Chen, UW, Seattle, WA, 98105, United States http://tracking.discuss.tvm.ai/tracking/unsubscribe?msgid=HSg2Z7yeSnGSOs6mV343pg2
[TVM Discuss] [Development] [PyTorch] [Frontend] graph input names can change using loaded torchscript
The input names are really annoying. I think one use case of the name to shape dict is to avoid the wrong order of the inputs. How hard is it for users to supply the inputs in the correct order? And it is possible to connect the names after _run_jin_passes? --- [Visit Topic](https://discuss.tvm.ai/t/pytorch-frontend-graph-input-names-can-change-using-loaded-torchscript/6055/5) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/58ed8c56c0ce791fb6b78b9e8b05346ea9530a980422e50e9c2f96faf1d3b34d).
[TVM Discuss] [Development] [PyTorch] [Frontend] graph input names can change using loaded torchscript
Thanks for clarification. I think this change makes sense to me. --- [Visit Topic](https://discuss.tvm.ai/t/pytorch-frontend-graph-input-names-can-change-using-loaded-torchscript/6055/7) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/9413f0046c4f636d0774ea47770b9f06365b110fc05440c4323796ad9832207b).
[TVM Discuss] [Development/RFC] Slice_like can't be constant folded
I am not sure. But I sort of remember that striced_slice may also need to change the `begin` and `end` into expr for dynamic shapes. @kevinthesun and @yongwww can comment more on this. --- [Visit Topic](https://discuss.tvm.ai/t/slice-like-cant-be-constant-folded/6206/2) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/370f557455e3b15788636df9f9e2542d4f90c172bdfee3c8accee142be340516).
[TVM Discuss] [Development] Missing memoization in ExprFunctor
Yeah, I am not a big fun of introducing this base class either as I think the only duplication code would be really just the caching map. If you are concerning about that 10 locs. I can actually just do it this way, I can actually remove them and replace it by calling the Functor::VisitExpr(expr); I did it that way because I wanted to give an error for unsupported node. Alternatively, I can have a checker for that. --- [Visit Topic](https://discuss.tvm.ai/t/missing-memoization-in-exprfunctor/6334/9) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/b331cfcbe3709c4a982e7330fdbf6ed12365c46eb72e73f7e36e1bdcaa920105).
[TVM Discuss] [Development] Missing memoization in ExprFunctor
ahh, I didn't notice we have this one. Thanks. --- [Visit Topic](https://discuss.tvm.ai/t/missing-memoization-in-exprfunctor/6334/12) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/e7ca0cccaae4fc4ec03d58846eadff30695a7b47a45cb755c0c7fe5f621ab1f7).
[TVM Discuss] [Development] Missing memoization in ExprFunctor
To be honest, among C0-C3 I wouldn't not want to introduce ANF to codegen. This means we either want to do ANF on the whole program or run the pass internally in the extern codegen to convert it. If we run it on the whole program, I think some passes that work on the DFG would not work well/or break. If we invoke it internally, we would want developers to be aware of this and perform this conversion explicitly. Actually, I also didn't see how much help non-recursive form would bring us here as these ASTs are usually simple (at least the ones we currently use), i.e. only some nodes need to be handled. It looks that MixedModeVisitor would need to do similar things. I may miss something here because I haven't used MixedModeVisitor yet. --- [Visit Topic](https://discuss.tvm.ai/t/missing-memoization-in-exprfunctor/6334/13) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/b6ea2617d2318ef91102148dd1c85de81522a5e84cf63232d5b6d060a175e8e2).
[TVM Discuss] [Development] Missing memoization in ExprFunctor
I have another thought on this, how about just put this one in the backend/utils.h since the current usage of them would be for the code under there? For general passes, it might be different though (like, to_a_norm_form, to_cps, PE, etc) --- [Visit Topic](https://discuss.tvm.ai/t/missing-memoization-in-exprfunctor/6334/15) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/d6b6ffa1da4c749f8c71ab064005306b59d9d3eac1c214b77bd27e074ac35904).
[TVM Discuss] [Development/RFC] [BYOC][runtime] JSON runtime for BYOC
We have currently built the infra for Bring-Your-Own-Codegen. For demonstration
purpose, a simple CSourceModule style codegen and runtime is used for ccompiler
and dnnl (now called oneDNN). CSourceModule runtime works reasonably well on
small examples and it is easy to understand. However, it also poses quite a few
challenges on development and deployment of relatively large models or models
with relatively large inputs.
- The serialization is quite cumbersome as it normally works on per operator
and emits a wrapper to invoke the library.
- Handling last constants is difficult. We currently either have to introduce
countless assignments or allocate a large chunk of memory on the static
segment. These approaches may significantly increase the compilation time.
- For certain backends, like TRT and dnnl, CSourceModule complicates the use of
or even makes it impossible to use their execution engine.
This RFC proposes a JSON runtime associated with a JSON serializer for BYOC
which effectively solves the above problems. In addition, this type of runtime
is more familiar to the community as the graph runtime is more or less in this
style and we have already implemented a minimal example runtime. This RFC
extends the minimal example and makes it more general to all backends with
execution engine.
- JSON nodes and code generator/serializer
- Data structures to represent the nodes and entries in a json runtime.
The serializer converts a Relay program into JSON format.
```c++
class JSONGraphNodeEntry {};
class JSONGraphNode {};
SOE // Serialize a Relay program into JSON frormat, graph and params
// should be saved in the same artifact
class JSONSerializer : public ExprVisitor {};
- JSONRuntimeDriver
- Deserialize the artifact and manage the initialization and invocation
of the runtime.
- Cache the engine when loading the library
```c++
JSONRuntimeDriver : public ModuleNode {
void Deserialize(); // Deserialize the artifact and engines
PackedFunc GetFunctioin(); // Invoke a subgraph using symbol
static Module LoadFromBinary(); // Load the JSON binary
void SaveToBinary(); // Save the module
```
- JSONRuntimeBase
- The base for handling a graph. It will be extended by the concrete
backends, like TRT, dnnl, and other accelerators.
```c++
class JSONRuntimeBase : public ModuleNode {
virtual void Run() = 0; // Invoke an engine
virtual void Init() = 0; // Build an engine
// Utilities to save and load a json graph.
};
```
- Open questions
- Symbolic representation of op attribute, i.e. `Expr start` and `Expr
end` in the `arange` op. Normally, we should not offload this type of nodes to
accelerators, but how can we serialize them if we want to support as some of
them may not be data-dependent?
- It's intuitive for BYOC to be used along with uTVM. How this JSON
runtime will be connected with other runtimes like utvm?
@tqchen @thierry @matt-arm @masahi @comaniac @manupa-arm @jonso @ramana-arm
---
[Visit
Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/1) to
respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https://discuss.tvm.ai/email/unsubscribe/ca373fe6d23cd9ce1e0e52e7af83e0da8ecb3735cb0113f818b2be05c1e6e37d).
[TVM Discuss] [Development/RFC] [BYOC][runtime] JSON runtime for BYOC
@tqchen Thanks for the comment and sharing of thoughts. Yes, the fundamental problem here is the serialization of code and weights. Code is relatively easy to handle and weights are the real problem. I agree that a json runtime introduces another layer of abstraction for graph which the current CSourceMdoule way doesn't. I think I don't fully understand the layered approach you proposed here. Could you please elaborate a bit more about the execution flow after introducing it? and also when should we build and cache the engine, i.e. what's the input for the process to build the engine? Thanks. --- [Visit Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/6) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/fa96b487cf8824ee39b58f26a00789b93aeee2d12ce9ec736c965a71f5eb4ebc).
[TVM Discuss] [Development/RFC] [RFC] [ETHOSN] Arm Ethos-N integration
@Leo-arm Thanks for the proposal and the interest in BYOC. I have a few questions, 1) are you using the CSourceModule runtime/serialization or something different? 2) Is the codegen toolchain ACL and do you plan to set the CI for testing because I see there are several stages for testing? --- [Visit Topic](https://discuss.tvm.ai/t/rfc-ethosn-arm-ethos-n-integration/6680/2) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/310f060d5a845a423d909b7a352a32821d23a42a7744adf48547c5fa8fcccd8c).
[TVM Discuss] [Development/RFC] [BYOC][runtime] JSON runtime for BYOC
I am not sure if the clarification of packaging part is clear enough, but there
is actually a potential problem. The goal is to be able to conveniently
assemble code and metadata separately from the frontend in a modular way. The
generated artifact is intended to be usable by AOT, graph runtime, and VM for
both CSourceModule and JSON style runtime.
Here we need to pass back code and weight to the Python side. They may have to
be part of the lib because we only return bytecode/(graph, params) and lib for
VM and graph runtime, respectively. We are trying to use the so-called
`PackagingModule` (TBD) to do this. They will be imported to the lib after
compilation. Therefore, the module with external library code would look like
the following for `CSourceModule` after compilation for graph runtime and VM:
```
DSO (A)
|---PackageModule(code, {var_name, metadata}) (B)
```
>From the Python side, we can assemble it by extracting code and metadata from
>the imported `PackageModule` B (i.e. code = code of A.imported_modules[0],
>metadata = metadata of A.imported_modules[0]).
Then, we can assembly the modules and compile/interpret them. But I do have one
question when we are assembling them. The DSO module (A) contains the other
part of the graph that should be handled by TVM. We now actually want to
replace (B) with the newly created module (i.e. the `ModuleMetaDataWrapper`)
and then do `export_library` and load them back for execution. It seems we are
not really able to remove/replace it. One possible way I can think of is that
we can add a `ClearImports` method to `Module` to clear the imports for the DSO
and then we can package the new modules. @tqchen Does this sound good? Or do
you have any comments/suggestions?
---
[Visit
Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/14) to
respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https://discuss.tvm.ai/email/unsubscribe/4ec0b8d69d553458bfa95d240324b8c9ed6e7c2164f6133c98fbf5edf584ea89).
[TVM Discuss] [Development] Add the document for TVMDSOOp
I think we actually need two things. One is thinking about how should we enable the tests to make sure other changes in TVM wouldn't break this functionality. The other is adding an official tutorial. There are examples under docs/dev. You can probably take a look at them and add it there. Please also add a bit background and motivation in the doc. A section about the design and implementation would be helpful as well. --- [Visit Topic](https://discuss.tvm.ai/t/add-the-document-for-tvmdsoop/6622/4) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/a871081b7c48ceb5e871cc5c4c357dedaf1c80d54c64ab4cee8051a626bb8011).
[TVM Discuss] [Development/RFC] [BYOC][runtime] JSON runtime for BYOC
Here is the draft PR: https://github.com/apache/incubator-tvm/pull/5770 We may need to use Map to save the variable to constant/NDArray mapping. Should we move the `ModuleInitWrapper` out of runtime because it otherwise needs to have map in the runtime namespace? I used `SourceMetadataModule` as the name for the the packing module. Do we have any other better names? @tqchen --- [Visit Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/16) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/55c7472c0ea8ab4ba6d8593ddce7271c878b35ffc7697619975f0a56448442d3).
[TVM Discuss] [Development/RFC] [BYOC][runtime] JSON runtime for BYOC
cc @junrushao1994 as well --- [Visit Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/17) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/e9cc785a099199c3cb5fb7c375a7f4033cbe9bda1618316902a29b021b6e721c).
[TVM Discuss] [Development/RFC] [BYOC][runtime] JSON runtime for BYOC
I thought about array as well. Passing array to initialize is relatively simple. The more tricky part is packing the data and passing them around using packedfunc. --- [Visit Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/19) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/075715b3f31fa1d27d8815126351d429c95cd0c7175fd8ad5b7e5b821964231b).
[TVM Discuss] [Development/RFC] [BYOC][runtime] JSON runtime for BYOC
Yeah, I think I didn't make it very clear. The problem was because we may have multiple subgraphs, each of them may have "var_name: NDarray" pairs. I was trying to just have one `ModuleInitWrapper` to take charge of the initialization of engines for all subgraphs so that users don't need to orchestrate from the Python side. This means we need to have a field like `Map> metadata_`, to save the mapping of each subgraph symbol to the `var_name: ndarray` pairs. This makes passing by array a bit more complicated. Another alternative approach could be having multiple `ModuleInitWrapper`. Each of them only takes care of one engine. Then we would only have `Map`, and then we could pass this by two arrays `Array variables_` and `Array metadata_`. This would need users to be aware of that there would be multiple modules and they need to initiate multiple `ModuleInitWrapper` for different engines. This is okay for CSourceModule stuff because we would usually only have one such a module. For execution engines like TRT, there would be multiple of it depending on the number of subgraphs. Maybe I can give a try to the alternative approach. @tqchen @junrushao1994 how do you think about the effort from frontend? --- [Visit Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/23) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/62b3a4b3ae89cbb9fabe79ce18a3f50bd5f63d9d39bd48a56378f40d7436ebf2).
[TVM Discuss] [Development/RFC] [BYOC][runtime] JSON runtime for BYOC
Yeah, I would prefer C1 or C2. C2 was pretty much what I was doing. --- [Visit Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/26) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/a9cdbffce6e46468c96988b5bf358523d39959d6c5c5f17db146dc33db258601).
[TVM Discuss] [Development/RFC] [BYOC][runtime] JSON runtime for BYOC
Yeah, let me give it a try. --- [Visit Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/28) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/5b20458f6a56bd09d1ad313f6446917b6cdad41db5443486d1076ef7827d08e4).
[TVM Discuss] [Development/RFC] [BYOC][runtime] JSON runtime for BYOC
BTW, we will need to have the variables as well, i.e. %x1, %x2, %x3, something as I mentioned above. This is because we need to know which variable a ndarray should be assigned to. --- [Visit Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/29) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/c837f1917f27e624e47a8f06da6dcc4ac0a158982f4483955dc818c7ce6aa168).
[TVM Discuss] [Development/RFC] [BYOC][runtime] JSON runtime for BYOC
yeah, I thought about positional ordering as well. But it looks pass variables might be safer. For a CSourceModule external codegen we generate a wrapper like `float* a = const_0;` `const_0` would need to be produced by the initializer later. So we would anyway need a name for it. --- [Visit Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/31) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/ba9fa366a1440c54d77c7782b603b07a734aa8f7ead8f9af728ca0963ad457ef).
[TVM Discuss] [Development] [DISCUSS] The meaning of "float" in Relay
+1 for making fp32 as default as fp64 may not be that useful and it could possibly increase memory footprint and reduce performance (i.e. occupying more SIMD lanes). I also agree that we can make float more explicit. --- [Visit Topic](https://discuss.tvm.ai/t/discuss-the-meaning-of-float-in-relay/6949/9) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/467498af8e49c25da7609d552644b5f28d2ffee63ae60445e0eede80480597d4).
[TVM Discuss] [Development/RFC] [RFC][BYOC] Data Calibration Flow
cc @anijain2305 as well --- [Visit Topic](https://discuss.tvm.ai/t/rfc-byoc-data-calibration-flow/7099/3) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/da1ed6d19e3b968d5c3954ef9b22f544a17cd0054f96e607bb6697d3373d7c11).
[TVM Discuss] [Development/RFC] [RFC][BYOC] Runtime module to offload subgraph to edge server
@kazum Thanks for the effort. It is very interesting. It sounds that you only need BYOC to do annotation and partitioning as you don't really have a backend/library for it, right? I am wondering how you package the subgraphs, do you manually prepare them? Thanks. --- [Visit Topic](https://discuss.tvm.ai/t/rfc-byoc-runtime-module-to-offload-subgraph-to-edge-server/7141/2) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/a6f2bfe92f18f980631c47ea67b9e9e262f9f2e83b6915255ccca1949174b86b).
[TVM Discuss] [Development/RFC] [RFC][BYOC] Data Calibration Flow
Thanks for the discussion. I think we don't really need to tie this feature to the BYOC flow. The problem it tries to solve is providing calibration data to 3rd codegen with quantizers as @anijain2305 pointed out. This is not required by QNN or AutoQ. It is also optional to 3rd codegen or BYOC. I would say it is probably more appropriate to treat it is just a separate pass and users can optional invoke it when necessary. Examples can be provided to show how it can be used to help the 3rd quantizers. I think it is useful because the majority of hardware vendors actually have their own quantizer. In the long-term, I agree with @tqchen that it would be great to investigate how we can extend AutoQ to support opaque quantizers. I agree that we should definitely think about some mechanism to encapsulate the BYOC flow. We should have a separate RFC to list some possible options and move forward from there. --- [Visit Topic](https://discuss.tvm.ai/t/rfc-byoc-data-calibration-flow/7099/14) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/e4f0887ab6cf1ec9cb416ee14ea544fd57e9b488fd86b6f8b07d8471b8cffb30).
[TVM Discuss] [Development/RFC] [RFC] Composite Target
Glad to see this is proposed since we wanted to do it for a while. I also agree that P2 is better. Another use case of it is heterogeneous execution where we can have llvm and cuda targets in it. --- [Visit Topic](https://discuss.tvm.ai/t/rfc-composite-target/7744/4) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/80f3c0367ebea60017519ced0be80ca91e06c420d32c29ea832b0cee09ad018d).
[TVM Discuss] [Development/RFC] [RFC] SaveToFile(file_name, format) expected behavior
I think another situation where `SaveToFile` is hard is when we have multiple modules imported. For example, a `MetadataModule` could contain a DSOModule and one or more CSourceModule/JSONRuntimeModule. It seems a bit hard to save them out as one file for compilation though. I think this is not quite directly related to the problem you solve to but it could complicate the problem. --- [Visit Topic](https://discuss.tvm.ai/t/rfc-savetofile-file-name-format-expected-behavior/7741/3) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/e675d03af5332f0ab346c28a8040d9d861f71f8646153fad2b15e8289120d7de).
[Apache TVM Discuss] [Development/RFC] [RFC] TVM Object Schema DSL
Yeah, this could be a useful tool to generate the generic templates or the code with the fixed pattern which is actually the major part of a node. For some other members, e.g. SEqualReduce and SHashReduce, we may still need users to manually check/add since they are not always `Equal(this->a, other->a) && Equal(this->b, other->b)`; --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-tvm-object-schema-dsl/7930/6) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/5be9d9e8b1a8ef474962dc8efbd9e2845da0cd38360e43508f7c39696a21376f).
[Apache TVM Discuss] [Announcement] [COMMUNITY] junrushao1994 -> Committer
Please join us to welcome Junru Shao(@junrushao1994) as a new Committer. Junru has been actively contributing to various aspects of the TVM codebase. He reimplemented and refactored the Target system which greatly helped code lowering and code generation. Junru also largely contributed to the runtime and node objects to improve the object passing. From the frontend side, Junru has also implemented some Relay operators. In addition, Junru has been very actively sharing his thoughts in the discussion forum to help improve the design of new features and provide new developers with better understanding of TVM. He has been constantly providing suggestive comments to the pull request. - [Commits History](https://github.com/apache/incubator-tvm/commits?author=junrushao1994) - [Code Review](https://github.com/apache/incubator-tvm/issues?q=reviewed-by+%40junrushao1994) - [Community Forum Summary](https://discuss.tvm.apache.org/u/junrushao1994/summary) https://github.com/apache/incubator-tvm/pull/6719 --- [Visit Topic](https://discuss.tvm.apache.org/t/community-junrushao1994-committer/8220/1) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/6e48d553752086b5e2a91350cb2bd9b8a7eb730188af43042c2e2b2961a50bb7).
[Apache TVM Discuss] [Development/RFC] [RFC] A general task extraction mechanism for auto_scheduler
This looks okay to me. But I have one comment because this sounds like we need to add one more argument to the build interface which users may not need to know the details. Another possible option is that we can bake it into `PassContext` as a config. However, I understand that this configure is really not a pass config. @tqchen do you have any better suggestion? --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-a-general-task-extraction-mechanism-for-auto-scheduler/8444/4) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/627ecdecfda1f8035585a6c5e472b10858d8dbbc7f63c8469e65c03fd974ada0).
[Apache TVM Discuss] [Development/RFC] [RFC] Building a new reproducible benchmark for TVM
It is really nice to add the regression tests against a selected set of models, since the down streams users usually have to spend quite amount of time to find the root cause once there is a regression. Or they have to sync the upstream codebase as frequent as possible and test regression locally. cc @jroesch, you may have some comments about the output format or the UX of the test infra. --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-building-a-new-reproducible-benchmark-for-tvm/8496/6) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/d9e2f4ed854594c4fba8c0dd3c906a06e136e1a5863fa2a416184349b9f66e50).
[Apache TVM Discuss] [Development/RFC] [RFC] Rename TVMContext to TVMDevice
Yeah, there are different uses of context in the codebase. Device makes more sense to me as well. Would the change to DLPack break other projects that take it as a submodule? --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-rename-tvmcontext-to-tvmdevice/9090/7) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/277ddbe840f173cbe6fd42a09387f83841757836c4d9f5d5f9923a850d5e1542).
[Apache TVM Discuss] [Announcement] [COMMUNITY] @lunderberg -> Committer
Please join us to welcome @Lunderberg as a new committer to TVM. Eric has greatly contributed to the testing framework and CI, TIR buffer allocation, and Vulkan backend etc. He has also been actively participating the RFC and forum discussions around the related areas, where he has shared many constructive thoughts and incorporated various helpful suggestions from the others. In addition, Eric has reviewed many PRs, provided various valuable comments, and exchanged ideas with others frequently, which greatly helped the growth of the community. * [Commits History ](https://github.com/apache/incubator-tvm/commits?author=lunderberg) * [Code Review](https://github.com/apache/incubator-tvm/issues?q=reviewed-by+%40lunderberg) * [Community Forum Summary](https://discuss.tvm.apache.org/u/lunderberg/summary) https://github.com/apache/tvm/pull/9773 --- [Visit Topic](https://discuss.tvm.apache.org/t/community-lunderberg-committer/11733/1) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/3d8e25f79cba04a4dee093e96ad1e5f5056d6f7322b4def50e4ea8c481db4e66).
