In my case, these intermediate structs are strongly tied to our executor. They
are plain structs, so much easier to work with than full blown relay IR. So for
me they are not really overhead.
---
[Visit
Topic](https://discuss.tvm.ai/t/external-codegen-constant-tensors-in-c-codegen/5890/25
@masahi I see, thanks for sharing. I also thought about an approach like this.
It seems like a lot of additional overhead to create and maintain a whole new
IR for serialization. Since it seems to be the case that many external codegens
(not just TRT) will need to do something like this, I won
@trevor-m Thanks for confirming. I can't talk about specific, but let's just
say any cpp serialization lib should be able to serialize/deserialize structs
into a binary blob, and I am just using one of them.
Note that I'm not serializing Relay subgraph as it is, but some structs that
get con
Hi @masahi, that is correct. I am using [TVM's native json serialization
API](https://github.com/neo-ai/tvm/blob/dev/include/tvm/node/serialization.h#L39-L48)
to serialize the relay subgraphs during codegen and deserialize it in the
runtime for my TRT integration.
I am curiously what binary f
Thanks @masahi for chiming in. The CCompiler example is only used for
demonstration purpose. We intentionally made it simple to handle cases like
constant. For real external codegen tools/compilers, you may have your own ways
to handle the constant pool, and it is very backend compiler depende
I think TensorRT integration by AWS works in a similar way. If I remember
correctly, they use json instead of binary.
---
[Visit
Topic](https://discuss.tvm.ai/t/external-codegen-constant-tensors-in-c-codegen/5890/20)
to respond.
You are receiving this because you enabled mailing list mod
Ah, I understand now. We'll have a look at how viable that'll be for ACL.
Thanks for the suggestion!
---
[Visit
Topic](https://discuss.tvm.ai/t/external-codegen-constant-tensors-in-c-codegen/5890/19)
to respond.
You are receiving this because you enabled mailing list mode.
To unsubscrib
"The executor" part, including API calls to DNNL, is defined in another lib
that is built outside of TVM, and linked to my TVM build. My TVM external
runtime passes binary or deserialized graph rep together with arguements from
TVM to that lib, and this lib knows how to execute the graph. The
How are you compiling? We could serialize the graph, but we'd then need to
codegen the relevant ACL API calls on the remote and compile it into something
that can be executed. We can't do that without a toolchain though which can't
be guaranteed.
---
[Visit
Topic](https://discuss.tvm.ai/
hmm, for my use case, I simply serialize a Relay subgraph into some binary
format, pass the binary to runtime module and deserialize there. Then I can
execute this graph with arguments I recieve from TVM in whatever way I like,
including offloading to dnnl. This week I integrated upstream chan
@masahi Thanks for the suggestion, however since ACL is a c++ library ideally
we would want to be able to cross-compile our codegen before using it on the
remote device. I don't think we can assume the remote device has its own
toolchain to compile the codegen we receive.
---
[Visit
Topi
@matt-arm Have you considered using different codegen than CSource? To deal
with large constants, I think binary serialization based codegen is a good fit.
---
[Visit
Topic](https://discuss.tvm.ai/t/external-codegen-constant-tensors-in-c-codegen/5890/14)
to respond.
You are receiving thi
I've had a chance to look at this now and it seems like it's quite a
fundamental issue with C codegen, not just ACL. This will make a lot of
compile-time optimisations impossible as there's no reasonable way to handle
large constant tensors in the codegen. This we be especially prevalent when
I see, thanks for the help!
---
[Visit
Topic](https://discuss.tvm.ai/t/external-codegen-constant-tensors-in-c-codegen/5890/12)
to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https://discuss.tvm.ai/email/unsubscrib
The larger constant tensor issue was also considered before, but since we have
no idea how will developers deal with constant tensors, we transparent this
part to the developers. As a result, you can do anything you think that's
better, including writing them out to a separate file.
---
[
Thanks, this is ok for a small number of values but doing this for ~13,000
values is where I think things get interesting. Am I correct in thinking that
writing `a[0] = 2; a[1] = 1; ... ` for every value will cause the intermediate
c file that is generated to explode in size? If that's the cas
No I think you could just assign the value one-by-one. One optimization you
could consider is to make sure you assign values only at the first visit.
---
[Visit
Topic](https://discuss.tvm.ai/t/external-codegen-constant-tensors-in-c-codegen/5890/9)
to respond.
You are receiving this becau
Apologies for bringing up an old post for another question - I didn't think it
warranted a new one and is related to this topic.
Using the propagate constant to subgraphs PR
(https://github.com/apache/incubator-tvm/pull/5094) I've run into the stack
overflow issue mentioned in the comments tr
18 matches
Mail list logo
