I think main topic of interest here is the way we define function signatures,
not necessarily how to broaden the scope of the same function to support more
flexible inputs(e.g. `Any`)
I think the main goal of A1 concerns the semantics of the attribute. Since
attribute has always been considered as constant and can be used during shape
inference.
The main difference between A0 and A1 are not about how do we categorize
dynamic ops vs static ops, all the operator still will need to support the type
inference `Any`, as you suggested. It only has to do with the way we define
operators.
For example, in the context of the reshape operator, A1 means we will have two
variants of operators:
- A1:
- V0: relay.static.reshape(x, attr={shape: (1,2,3)})
- V1: relay.dyn.reshape(x, shape)
- relay.reshape alias to V0 for now for backward compact, once the dyn ->
static pass completes, relay.reshape alias to relay.dyn.reshape.
- A0: For a clear comparison this is the V2 variants.
- V2: relay.reshape(x, shape, optional_attr={shape: (1,2,3)})
Both V0 and V1 still need to be able to handle the case of `Any` in the
input(x), in the case of V1, it also need to handle `Any` in terms of the
shape. In the same case (A0) V2 also need to handle `Any` in all of its input.
There is a tension between frontend and the backends. For frontends, it is
easier to use operator signatures that are more "dynamic". For backend,
however, we want more information in the system, which favors more static
information as part of the function signature. The type inference can certainly
take benefit of the attribute information of the operator. Because attribute is
constant and directly available during type inference. So besides introducing
V1 style operator def, it is important to have a pass to be able to convert
dynamic variants of operators aka V1 into V0 when possible, so that even though
the input graph has a dynamic semantics, we will still be able to use static
information in the case of graph runtime and embedded devices.
The definition of V2 is trying to combine V0, V1 s a polymorphic function into
a single function signature. In terms of implementations, it means that during
the optimizations passes, a pass would have to deal with all possible
combinations of V2, (e.g. shape_attr exists, shape exists, both exists). It is
harder for pass writer to understands the semantics of the operator, as the
same piece of information is available in two possible places.
The advantage of A1 is the separation of concern. The frontend can always use
V1 to import the model. Of course A1 does not reduce the amount of effort,
since most effort done in V2 would need to be split into two natural part of
functions to support V0 and V1. It does makes the dyn to static conversion pass
easier, so that output of the pass contains as many ops in V0 style as
possible, and optimization passes that targets these static variants of ops can
be used to provide better optimization chances for the backend.
---
[Visit Topic](https://discuss.tvm.ai/t/dynamic-ops-in-relay/6909/6) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https://discuss.tvm.ai/email/unsubscribe/9dc5728971517b69ebc442a732bf2395600b7ac2ead4c121e78aec37ea346c91).
