Thanks @leandron and @ekalda for the comments. We all agree that we are trying to improve the graph-level IR of TVM while the controversial point is that if we can enhance relay to support features from relax. Let's discuss it directly and focus on the technical points themselves.
First of all, I'd like to list some most critical features that relax want to introduce: 1. Dynamic shape support, to be specific symbolic shape representation; 2. A representation for TVMUnity, i.e. a cross-layer abstraction for optimization; 3. Customizable compilation flow and operator support. In my opinion, it's hard to incrementally update relay to support them. ## G1: Dynamic shape support To be specific, relax can represent and *deduce* symbolic shape rather than use `Any`. However, if we introduce dynamic shapes to relay, there will be two competing repr for shapes (symbolic shape and `Any`), which makes it undesirable. ## G2: A representation for TVMUnity TVMUnity is an important feature for unified optimization for graph, tensor computation, and libraries. The build flow of relay is a one-way path: `relay->tir/libraries->runtime module`, while TVMUnity enables `IRModule(graph+tir+libraries)->IRModule` transformations, which gives users more flexibility to choose the backend (use codegen or call libraries) even after tuning. I'm not sure if it's possible for relay if we still keep the original workflow. ## G3: Customizable compilation flow and operator support. Customizing operators and backends are really common in production. There are [7 steps](https://tvm.apache.org/docs/dev/how_to/relay_add_op.html) to add a new operator to relay. However, we only need 2 steps in relax: 1. write how the op is computed (both tir or libraries are good), 2. use `call_tir` to represent it in IRModule Additionally, other compilation customization skills (e.g. BYOC, AOT, customized fusion) are also more straightforward with relax. Please see the [TVM Unity Connection](https://discuss.tvm.apache.org/t/establish-tvm-unity-connection-a-technical-strategy/13344#byoc-9) In short, a new IR is a reasonable way to support the above features IMO. And I'm open to hearing more ideas from the community. -- Reply to this email directly or view it on GitHub: https://github.com/apache/tvm-rfcs/pull/89#issuecomment-1220829699 You are receiving this because you are subscribed to this thread. Message ID: <apache/tvm-rfcs/pull/89/c1220829...@github.com>
