huangbinghao opened a new issue, #20031: URL: https://github.com/apache/tvm/issues/20031
Thanks for participating in the TVM community! We use https://discuss.tvm.apache.org/ for any general usage questions and discussions. The issue tracker is used for actionable items such as feature proposals discussion, roadmaps, and bug tracking. You are always welcomed to post on the forum first :smile_cat: Issues that are inactive for a period of time may get closed. We adopt this policy so that we won't lose track of actionable issues that may fall at the bottom of the pile. Feel free to reopen a new one if you feel there is an additional problem that needs attention when an old one gets closed. ## Summary When TVM is built with both `USE_CUDA=ON` and `USE_CUTLASS=ON`, `tests/python/relax/test_codegen_cutlass.py` can still be skipped at module collection time because `tvm.testing.env.build_flag_enabled("USE_CUTLASS")` incorrectly returns `False`. This false-negative test gate masks at least three regressions in the Relax CUTLASS code generation path: 1. The regular GEMM template references the undefined substitution variable `${A_arg}`, while the template arguments provide `${lhs_arg}`. 2. `tvm/ffi/container/tensor.h`, which provides the required TVM-FFI tensor type conversion support. 3. The generated wrapper exports the old bare function symbol, while the current TVM-FFI library loader expects the `__tvm_ffi_<name>` symbol and the TVM-FFI SafeCall ABI. ### Expected behavior When TVM is built with `USE_CUTLASS=ON`: 1. `tvm.support.libinfo()["USE_CUTLASS"]` and `tvm.testing.env.build_flag_enabled("USE_CUTLASS")` should report that CUTLASS support is enabled. 2. `tests/python/relax/test_codegen_cutlass.py` should execute instead of being skipped. 3. A minimal Relax `float16` 2-D matmul partitioned for CUTLASS should: - generate valid CUDA/C++ source; - compile successfully; - be loadable by the Relax VM; - expose the external function using the current TVM-FFI ABI. ### Actual behavior #### 1. CUTLASS tests are skipped even though CUTLASS is available The module-level marker in `tests/python/relax/test_codegen_cutlass.py` is equivalent to: ```python python - <<'PY' import tvm libinfo = tvm.support.libinfo() print("TVM version =", tvm.__version__) print("libinfo USE_CUTLASS =", libinfo.get("USE_CUTLASS")) PY ``` output: ``` TVM version = 0.26.dev0 libinfo USE_CUTLASS = OFF ``` Running a CUTLASS test therefore reports it as skipped: ```python python -m pytest \ 'tests/python/relax/test_codegen_cutlass.py::test_matmul_offload[float16-x_shape0-y_shape0-False-none-none]' \ -qrs ``` Output: ``` SKIPPED [...] need cutlass ``` when I modify the `python/tvm/support/libinfo.py` to bypass the problem, the following mistakes were exposed #### 2. Bypassing the skip exposes invalid regular GEMM source When a minimal regular GEMM is passed through CUTLASS codegen, the generated source contains an unsubstituted ${A_arg} expression. The regular GEMM template currently contains code equivalent to: ``` cudaStream_t stream = ${lhs_arg}->device.device_id ``` This replacement should be scoped to the regular GEMM template because A_arg may still be valid in the specialized decode/quantized paths. #### 3. Generated code cannot cast TVM-FFI arguments to DLTensor* The generated CUTLASS source includes: `DLTensor* arg0 = (DLTensor*)(args[0].cast<DLTensor*>());` but its generated header list contains only: ```c++ headers = [ "tvm/ffi/function.h", "tvm/ffi/extra/c_env_api.h", ] ``` Without: ` #include <tvm/ffi/container/tensor.h>` the required TVM-FFI tensor type traits/conversion support is unavailable. Compilation then fails because no valid AnyView::cast<DLTensor*() instantiation can be found. FortunatelyI try to add `tvm/ffi/container/tensor.h` to the generated headers, and this compilation error was resolved. ### Environment The issue was reproduced with: Operating system: Ubuntu 20.04.5 LTS GPU: NVIDIA A100-SXM4-80GB GPU compute capability: 8.0 NVIDIA driver: 535.216.03 CUDA toolkit / NVCC: 12.2.140 ### Steps to reproduce #### 1. Build TVM with CUDA and CUTLASS enabled For example: ``` set(USE_CUDA ON) set(USE_CUTLASS ON) set(CMAKE_CUDA_ARCHITECTURES "80") ``` Build TVM and configure PYTHONPATH/LD_LIBRARY_PATH to use the resulting local build. #### 2. Temporarily, modify the `python/tvm/support/libinfo.py` to enable CUTLASS ```python ... "USE_CUTLASS": "ON", ... ``` #### 3. Run the existing CUTLASS test example: ``` python -m pytest \ 'tests/python/relax/test_codegen_cutlass.py::test_matmul_offload[float16-x_shape0-y_shape0-False-none-none]' \ -qrs ``` The problem will be reproduced ### Triage Please refer to the list of label tags [here](https://github.com/apache/tvm/labels) to find the relevant tags and add them below in a bullet format (example below). * needs-triage -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
