junrushao commented on code in PR #191: URL: https://github.com/apache/tvm-ffi/pull/191#discussion_r2464108373
########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` +~~~~~~~~~~~~~~~~~ + +TVM-FFI's ``Any`` is a fixed size (128-bit) tagged union that represents all supported types. + +- First 32 bits: type index indicating which value is stored (supports up to 2^32 types). +- Next 32 bits: null padding, or reserved for special flags in rare cases. +- Last 64 bits: payload that is either a 64-bit integer, a 64-bit floating-point number, or a pointer to a heap-allocated object. + +.. figure:: ../_static/tvm-ffi-layout-any.svg + :alt: Layout of the 128-bit Any tagged union + :name: fig:layout-any + + Figure 1. Layout spec for the ``Any`` type. + +The following conventions apply when representing values in ``Any``: + +- Primitive types: the last 64 bits directly store the value, for example: + + * Integers + * Floating-point numbers + +- Heap-allocated objects: the last 64 bits store a pointer to the actual object, for example: + + * `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ + * C-style strings + +- Arbitrary objects: the type index identifies the concrete type, and the last 64 bits store a pointer to a reference-counted object in TVM-FFI's object format, for example: + + * :py:class:`tvm_ffi.Function`, representing all functions, such as Python/C++ functions/lambdas, etc.; + * :py:class:`tvm_ffi.Array` and :py:class:`tvm_ffi.Map` as list/dictionary containers of all ``Any`` values; + * Up to 2^32 types can be represented in this type system. + +Function Calling Convention +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Function calls in TVM-FFI share the same calling convention, ``universal_c_abi``, as described above. + +- ``handle: void*``: a library handle that contains the function to be called. It is usually ``nullptr`` for most use cases, but some libraries (such as cuDNN) may use it to store global contexts. +- ``args: Any*``: a pointer to an array of ``Any`` values for input arguments, stored contiguously on stack or heap. +- ``num_args: int``: number of input arguments. +- ``result: Any*``: a pointer to a single ``Any`` that receives the output result. + +.. figure:: ../_static/tvm-ffi-layout-func.svg + :alt: Calling convention for universal_c_abi + :name: fig:layout-func + + Figure 2. Layout spec of ``universal_c_abi`` function. + + +Stability and Interoperability +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Stability.** The pure C layout and the calling convention are stable across compiler versions and independent of host languages or frameworks. + +**Cross-language.** TVM-FFI implements this calling convention in multiple languages (C, C++, Python, Rust, ...), enabling code written in one language—or generated by a DSL targeting the ABI—to be called from another language. + +**Cross-framework.** TVM-FFI uses standard data structures such as `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ to represent arrays, so compiled functions can be used from any array framework that implements the DLPack protocol (NumPy, PyTorch, TensorFlow, CuPy, JAX, and others). + + +Stable ABI in C Code +-------------------- + Review Comment: adjusted a little bit: > TVM FFI's :ref:`C ABI <tvm_ffi_c_abi>` is designed with DSL and ML compilers in mind. DSL codegen usually relies on MLIR, LLVM or low-level C as the compilation target, where no access to C++ features is available, and where stable C ABIs are preferred for simplicity and stability. > > This section shows how to write C code that follows the stable C ABI. Specifically, we provide two examples: > > - Callee side: A CPU ``add_one_cpu`` kernel in C that is equivalent to the :ref:`C++ example <cpp_add_one_kernel>`. > - Caller side: A loader and runner in C that invokes the kernel, a direct C translation of the :ref:`C++ example <cpp_load>`. > > The C code is minimal without any other dependencies, and thus can serve as a direct reference for DSL compilers to expose or invoke kernels or other functions following our C ABI standard. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
