[Apache TVM Discuss] [Development] Relay Function virtual_device property
Hi Rafael, virtual device handling is unfortunately in a halfway-implemented state, and it's been on my backlog for a while to wrap that up. Sorry about that! I'm hoping I can work on it in a few weeks as a break between other tasks. There's a few things to be done: - Populate the virtual_device field on every expression node, and remove the use of 'on_device' annotations to record device information. - As you are noticing, propagate virtual_device info through the passes by migrating to WithFields (which should have the happy side effect of also propagating spans and other generic information.) - Update the existing passes that need to understand virtual devices. However I think consensus is this is better done in Relax, so another possibility is to backtrack. --- [Visit Topic](https://discuss.tvm.apache.org/t/relay-function-virtual-device-property/12958/2) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/2f9e9e009469b12d7bc479fab13226f81ab53a8c20d3f930b6dcda8c0599d685).
Re: [apache/tvm-rfcs] [RFC] TUNIP: TVMScript Unified Printer (PR #74)
thanks for summarizing @junrushao1994 . to provide some additional perspective: - I'd really like us to get to a place where we have 1 reliable and readable format to serialize TIR. i don't think we should build all of this infrastructure just to provide a "best-effort" serialization--anything expressable in TIR should roundtrip in any serialization format we provide. there's no reason we have to sacrifice reliability for readability, either--it's just a matter of putting effort into the unit tests. - right now i believe we have 3 TIR formats: `repr()`, TVMScript, and JSON. This RFC looks to provide infra that allows for generation of more formats e.g. so that Python doesn't have to be the frontend for TIR, or so we can leverage other APIs or frontend languages to write TIR. i'm okay with adopting this design to separate indentation from serialization, but if we do follow through and introduce an IRBuilder serialization, that would count as a fourth way to serialize TIR--doing that is only going to make it harder to achieve the previous goal because it splits our human and unit-test attention across multiple formats. i don't see a good reason for doing that laid out in the RFC. - this PR does provide considerations for Relax, although the language spec hasn't been RFC'd or introduced yet. i don't want to block the PR over that, but it's plain that the eventual goal is to leverage this infrastructure for Relax when it lands. then that gets us into a scenario where we have one parser/serializer for Relay and another for everything else. that's also more complex than it needs to be. admittedly, Relax and TIR benefit more from meta-programming than Relay, so it makes sense that the old Relay parser/serializer isn't a great fit here. but if we're going to discuss Relax here, it would be great to see this design considering how it could replace all of the printers in the codebase, not just overhaul the TIR one. anyhow, this discussion can be left for a future time when the Relax merge plan is clearer. -- Reply to this email directly or view it on GitHub: https://github.com/apache/tvm-rfcs/pull/74#issuecomment-1155312769 You are receiving this because you are subscribed to this thread. Message ID:
Re: [apache/tvm-rfcs] [RFC] TUNIP: TVMScript Unified Printer (PR #74)
Hey @areusch thanks for elaborating your points and these are all definitely great questions to me! > right now i believe we have 3 TIR formats: repr(), TVMScript, and JSON. This > RFC looks to provide infra that allows for generation of more formats e.g. so > that Python doesn't have to be the frontend for TIR, or so we can leverage > other APIs or frontend languages to write TIR Oh I think I get your concern here!! No, we are not building more and more and more frontends to cause fragmentation. Here is the thing - we only need two forms for serialization: - JSON for protobuf-like reliable serialization; - TVMScript python syntax. And we defragment the system by retiring `repr()` for TVMScript. Therefore, as a python user, the only interface you touch is TVMScript python syntax after the RFC landed. Imagine you are a rust user who doesn't want to use python, our proposal makes it possible to develop a frontend in pure rust. And of course, it's not going to be our priority, i'm just stating the possibility. > but if we do follow through and introduce an IRBuilder serialization, that > would count as a fourth way to serialize TIR Ah no, IRBuilder is not serialization, sorry for the confusion. Let's imagine a common usecases in company's prod env where python is strictly prohibited (e.g. on-device training), but our developers wrote most of the operators in python, which is the reality. How do we quickly migrate those operators to C++? Our RFC provides an answer to that - just write a codegen from Doc to C++ IRBuilder :-) Of course again, the RFC aims to open possibilities instead of implementing those functionalities. > this PR does provide considerations for Relax, although the language spec > hasn't been RFC'd or introduced yet. i don't want to block the PR over that, > but it's plain that the eventual goal is to leverage this infrastructure for > Relax when it lands Yeah it's definitely fair to say the PR considers Relax a lot. On the other hand, Relax doesn't need this RFC to use TVMScript. Take the current codebase as an example, it's not using the design of this proposal and it's totally fine given it's assuming there are only 2 IRs. Instead, I'm trying to convey is that we build an open infra for any vendors to integrate with their own IRs, which can be represented and compiled in the same IRModule with Relax, TIR, etc. -- Reply to this email directly or view it on GitHub: https://github.com/apache/tvm-rfcs/pull/74#issuecomment-1155589354 You are receiving this because you are subscribed to this thread. Message ID:
Re: [apache/tvm-rfcs] [RFC] TUNIP: TVMScript Unified Printer (PR #74)
@junrushao1994 ah thanks for clarifying! a few follow-ups then: > Imagine you are a rust user who doesn't want to use python, our proposal > makes it possible to develop a frontend in pure rust. And of course, it's not > going to be our priority, i'm just stating the possibility. Does this mean you can import the JSON-based serialization using only libtvm.so (e.g. rust could import JSON)? If so, then, in the hypothetical prod scenario: >Let's imagine a common usecases in company's prod env where python is strictly >prohibited (e.g. on-device training), but our developers wrote most of the >operators in python, which is the reality. How do we quickly migrate those >operators to C++? , what's the motivation for someone to use IRBuilder instead of just serializing the TVMScript to JSON via parse/print and then ingesting the JSON in their prod env using the JSON parser? > it's not using the design of this proposal and it's totally fine given it's > assuming there are only 2 IRs. Instead, I'm trying to convey is that we build > an open infra for any vendors to integrate with their own IRs, which can be > represented and compiled in the same IRModule with Relax, TIR, etc. That makes sense, thanks for clarifying :) -- Reply to this email directly or view it on GitHub: https://github.com/apache/tvm-rfcs/pull/74#issuecomment-1155759976 You are receiving this because you are subscribed to this thread. Message ID:
[Apache TVM Discuss] [Meetup] Next TVM Community Meeting June 15, 2022 [cancelled]
Hi all, I don't see an agenda topic for this week, so let's cancel this meeting. Andrew --- [Visit Topic](https://discuss.tvm.apache.org/t/next-tvm-community-meeting-june-15-2022-cancelled/12950/2) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/413bb5e1323282dca9ad156bb925e16e1c8cd36c438041ab917c7704c6946354).
Re: [apache/tvm-rfcs] [RFC] Buffer Layout Padding (PR #77)
Thanks @csullivan for providing the overview. I agree that non-local approaches 2-4 are necessary. From the examples in this RFC I can also see how the components C0-C2 can be used to support these non-local approaches. C0 + C1 allows to specify the constraints during scheduling, and propagate back to the graph. Besides them, I would also like to mention another component * C3: ability to specify constraints for each operator. It seems to me that C0, C1, C3 are actually choices of implementation as there are multiple ways that require a combination of them to achieve the goal of constraint flowing. * C0 + C1 (which imply C3 are satisfied) suggests implementing the constraints at TIR level using `BufferConstraint`. To propagate back the constraints to the graph, which is `Tensor` central, it seems the graph-level counterpart of `BufferContraints` is not clear, as @wrongtest mentioned. * C3 is also feasible purely in the graph, which requires some mechanism to register per-operator constraints. An example I came up with is each operator can have a list of supported layout, and the constraint solver can choose layout for each operator to approximate the global optimum for the graph. This satisfies the need for non-local approaches but doesn't need TIR level constraints. Padding, instead of explicitly inserting `transform` / `inv_transform`, is also achievable as graph-level constraint flowing. Back to the discussion of this RFC, I think the main comments about the proposed methods is IR changes required (which may have greater impacts on the existing TIR and scheduling), and the complexity involved using the new schedule primitive to reach the final desired state. From my understanding, the intention of these new primitives is to allow arithmetic simplification to perform graph rewriting like over-computation. If this can be achieved as graph-level rewriting rule (perhaps simpler as it doesn't need arithmetic manipulations), personally I think that would still be preferred for better maintainability. Also I'd like to mention that modeling such rewriting in the graph doesn't necessary tie the TIR operator with a specific graph IR implementation. As we are moving to S-TIR scheduling, it is easy to apply some preprocessing steps to derive the PrimFunc in specific layout from a standard `te.compute` definition. Finally, I would like to encourage us to focus on the e2e goals. It seems the current approaches, either implemented as A0 or A1 in graph-level, should suffice the use cases in the inference graph. Though the training graph is probably not an immediate need, if we would like to consider their use cases, probably having some concrete examples with desired result can guide us to make better decision. -- Reply to this email directly or view it on GitHub: https://github.com/apache/tvm-rfcs/pull/77#issuecomment-1155766862 You are receiving this because you are subscribed to this thread. Message ID:
Re: [apache/tvm-rfcs] [RFC] TUNIP: TVMScript Unified Printer (PR #74)
@areusch Thanks for following up! > what's the motivation for someone to use IRBuilder instead of just > serializing the TVMScript to JSON via parse/print JSON is mostly for serializing an IR after it's constructed (which users cannot manipulate), and the TVMScript format is for users to construct the IR themselves. For example, to implement a ReLU operator, TVMScript users could write a python/rust code, while it's pretty infeasible for them to write JSON with bare hand. -- Reply to this email directly or view it on GitHub: https://github.com/apache/tvm-rfcs/pull/74#issuecomment-1155853969 You are receiving this because you are subscribed to this thread. Message ID:
