jhlee525 opened a new issue, #328: URL: https://github.com/apache/tvm-ffi/issues/328
We are currently refactoring our runtime using `tvm-ffi`. Instead of building only `tvm-ffi`, we build the full `tvm-runtime`, which enables additional device APIs and global functions, allowing us to execute real LLM models. While enabling the device API, we encountered an issue and wanted to confirm our understanding. In `tvm-ffi`, the Tensor API provides an `NDAllocator`, which abstracts device tensor memory allocation and free. However, we cannot find any abstract API for data copy, either host → device or device → host. We are planning to add this in our ffi, but it would be more natural if `Tensor` exposed a dedicated data-copy interface, such as a Rust trait or a C++ mixin, rather than relying on raw `void*` buffer copy. We are wondering is there any roadmap in `tvm-ffi` providing an abstract interface for data-copy in `Tensor`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
