Per offline discussions, here is a summary of the updated proposal:
* The original proposal uses a runtime module to maintain both json and
metadata (e.g., constant weights) together. As @tqchen pointed out, although
this is simple to be implemented, it is hard to debug and cannot be shared over
other runtimes such as VM and AOT.
* Accordingly, we now aim to separate code and metadata. In other words, while
the code can be in any formats (e.g., C++ for CSourceModule; JSON or other
forms for customized runtime), **we will use the same mechanism to deal with
the metadata over all kinds of runtime modules**.
In terms of the implementation and user inreface, the current API is:
```python
json, lib, params = relay.build(mod, target='llvm', params=params)
```
One question to solve with the current API is that we only have one fixed
output from `relay.build` for the built runtime module, `lib`, so it is not
straightforward to get both code and metadata via this API. As a result,
instead of returning a runtime module, we are thinking to return a
`PackingModule` for the Relay module with external functions, and use modular
approach to build pipeline on the Python side.
`PackingModule` is composed of a mapping from `symbol` to `(implement,
{var_name: NDArray})`. The `implement` would be the C++ code for
`CSourceModule`, or JSON string for `JSONRuntimeModule`. In this way, we could
have a unified inferface for both cases.
@tqchen do you think it's ok to use the `PackingModule`, or you have a better
idea?
---
[Visit
Topic](https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579/12) to
respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https://discuss.tvm.ai/email/unsubscribe/7106b5859e7649b14cde3043155a0a6a3e3bc9ec5e700a59cf8a93ffa386e3fb).
