For the F1, current design is simply to add multi model support (in the
previous pr I even implemented draft multi model support to verify current
design) , even on different ctxs. But the issue is the unique compiled name as
@tqchen described, we could evaluate and discuss whether we should d
On the parameter spliting etc. Take a look at module based runtime proposal in
F0 and see if it meet most of the need. The ideal is that the factory module
will create separate runtimes for each of the model. Rather than a single
executor for all models (so you don't have to differentiate betw
Thanks for splitting the proposal.
Replying about F1
Yes we can generate multiple libraries but the issue is linking them together.
Specifically, there is no way to differentiate inputs vs params/weights. There
is no way to know the name of the outputs as they have been mangled after
simplifi
I think we want to dissect these points a bit:
### F0: multi-model support
The support for multiple model loading is discussed and resolved as part of
module based runtime interface
https://discuss.tvm.apache.org/t/discuss-module-based-model-runtime-interface/5025/47.
Although the current
In scenarios where multiple models are used back to back, with multiple inputs
and outputs, TVM doesn't produce helpful native libraries to connect them:
- `get_num_inputs()` returns all tensors instead of only the inputs of the model
- `get_output(id)` has no support for strings. And since out
