Ubospica commented on issue #264: URL: https://github.com/apache/tvm-ffi/issues/264#issuecomment-3569797349
Apologies for the late reply. Thanks @tqchen @yaoyaoding, and @junrushao for the detailed explanation and discussion! I agree we should use `TVMFFIEnvTensorAlloc` in most cases to avoid this. Especially, @yaoyaoding mentioned that the env allocator is more compatible with the target framework's memory pool. My concern is similar to @junrushao's, that sometimes we may not have the env allocator set, such as C++ environments. Also, for JAX the env allocator is not defined yet. In these cases, `FromNDAlloc` could be helpful. We can probably add CPU memory allocation defaults to it. Wondering if @yaoyaoding has further thoughts about this. About docs, one thing we could do is say the API is fine for the cases where the library is persistently loaded, but users need to be extra careful if the library is JIT loaded as a plugin. If these look fine, I can help with implementing these. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
