Source: clblast Version: 1.6.3-1 Severity: normal X-Debbugs-Cc: [email protected]
Dear Maintainer, *** Reporter, please consider answering these questions, where appropriate *** I am in an internship at Synchrotron-Soleil , my directors are Frederic- Emmanuel PICCA and Emmanuel FARHI. I implemented Rocm and Pocl autopkgtest for architecture amd64 and arm64. I implemented autopktests for rocm and pocl using Opencl on the package clblaston local. The aim is to have an overview of code compatibility with various AMD graphics cards, the codes on all the AMD boards available for CI rocm for GPU and Pocl for CPU. I implemented the POCL and ROCm tests in the control file. After running sbuild, it turns out that the tests involving the CPU, i.e. POCL, pass. However, the tests for the GPU part had accuracy problems, mainly due to the nature of the tests. The reason why these tests don't pass is that the card doesn't support, or supports poorly, FP16 calculations on ROCm. These libraries support these calculations, but this may not be enough to make them pass in terms of precision. We therefore need to increase the tolerance to see if this would be sufficient (rocBLAS, MIOpen and ROCm-Lib). This may be because the card has a kernel limitation. Apparently these are common problems with FP16 on ROCm, and intrinsic to the card you have. It's symptomatic if you don't have professional cards. Common problems with FP16 on ROCm, due to accuracy issues. CLBlast tests sometimes fail because tolerances are too tight for FP16. ROCm may use implicit FP16 to FP32 conversions (for compatibility), which may explain the numerical differences. Incomplete hardware support may be the cause. Some cards (such as the RX 6400) support FP16, but with limited extensions (e.g. no FP16_MMA on RDNA 2 consumer). The card we have would have partial FP16 support. This would explain our problem and any other problems we might have in the future. Sources : https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction- set-architectures/rdna2-shader-instruction-set-architecture.pdf https://cnugteren.github.io/clblast/clblast.html https://ar5iv.labs.arxiv.org/html/1705.05249 cf (section 3.1.Library Design et 3.2 ) https://localai.io/basics/build/ ( depedances nécessaires rocblas, miopen-hip) https://www.amd.com/en/developer.html Old version (ROCm 3.5.9) with known bugs on FP16 after some research... Consumer cards such as the RX 6400 are not a priority for FP16 optimisations in ROCm. ****Tests are laucnched but it supposed to pass, it's a precesion Error . The purpose is to pass Pocl and ROCm autopkgtest to tests compatibility with AMD's GPU. So it's very important to do it and to solve it. These tests are very important because we have to know if opencl tests are able to run on AMD's GPU or not. And on opencl tests on CPU. With these bug we couldn't know it because it does not go beyond the stage of these bugs. Especially when it comes to make precisions tests. * I implemented in debian/tests/control : Test-Command: debian/tests/tests-pocl.sh Depends: clinfo, clblast-tests, colorized-logs, gcc, ocl-icd-opencl-dev | opencl-dev, pocl-opencl-icd , libc6-dev Features: test-name=tests-pocl Architecture: amd64 arm64 i386 Restrictions: allow-stderr Test-Command: rocm-test-launcher debian/tests/tests-pocl.sh Features: test-name=tests-rocm Depends: clblast-tests, colorized-logs, gcc, ocl-icd-opencl-dev | opencl-dev, rocm-opencl-icd, pkg-rocm-tools, libc6-dev, rocm-device-libs-17, hipcc, libclang-common-17-dev, clinfo, mesa-utils, libclblast-dev, Architecture: amd64 arm64 i386 Restrictions: allow-stderr, skippable Log ERROR : Running test set xamax. -------------------------------------------------- * Options given/available: -platform 0 [=default] -device 0 [=default] -full_test [false] -verbose [false] -cblas 1 [=default] * Running on OpenCL device 'AMD Radeon RX 6400'. * Starting tests for the 'iSAMAX' routine. Legend: : -> Test produced correct results . -> Test returned the correct error code X -> Test produced incorrect results / -> Test returned an incorrect error code \ -> Test not executed: OpenCL-kernel compilation error o -> Test not executed: Unsupported precision - -> Test not completed: Reference CBLAS doesn't output error codes * Testing with error margins of 0.5% (relative) and 0.001 (absolute) * Testing 'regular behaviour' for 'default': :::::::::::: Pass rate 100.0%: 12 passed / 0 skipped / 0 failed * Completed all test-cases for this routine. Results: 12 test(s) passed 0 test(s) skipped 0 test(s) failed * Running on OpenCL device 'AMD Radeon RX 6400'. * Starting tests for the 'iDAMAX' routine. Legend: : -> Test produced correct results . -> Test returned the correct error code X -> Test produced incorrect results / -> Test returned an incorrect error code \ -> Test not executed: OpenCL-kernel compilation error o -> Test not executed: Unsupported precision - -> Test not completed: Reference CBLAS doesn't output error codes * Testing with error margins of 0.5% (relative) and 0.001 (absolute) * Testing 'regular behaviour' for 'default': :::::::::::: Pass rate 100.0%: 12 passed / 0 skipped / 0 failed * Completed all test-cases for this routine. Results: 12 test(s) passed 0 test(s) skipped 0 test(s) failed * Running on OpenCL device 'AMD Radeon RX 6400'. * Starting tests for the 'iCAMAX' routine. Legend: : -> Test produced correct results . -> Test returned the correct error code X -> Test produced incorrect results / -> Test returned an incorrect error code \ -> Test not executed: OpenCL-kernel compilation error o -> Test not executed: Unsupported precision - -> Test not completed: Reference CBLAS doesn't output error codes * Testing with error margins of 0.5% (relative) and 0.001 (absolute) * Testing 'regular behaviour' for 'default': :::::::::::: Pass rate 100.0%: 12 passed / 0 skipped / 0 failed * Completed all test-cases for this routine. Results: 12 test(s) passed 0 test(s) skipped 0 test(s) failed * Running on OpenCL device 'AMD Radeon RX 6400'. * Starting tests for the 'iZAMAX' routine. Legend: : -> Test produced correct results . -> Test returned the correct error code X -> Test produced incorrect results / -> Test returned an incorrect error code \ -> Test not executed: OpenCL-kernel compilation error o -> Test not executed: Unsupported precision - -> Test not completed: Reference CBLAS doesn't output error codes * Testing with error margins of 0.5% (relative) and 0.001 (absolute) * Testing 'regular behaviour' for 'default': :::::::::::: Pass rate 100.0%: 12 passed / 0 skipped / 0 failed * Completed all test-cases for this routine. Results: 12 test(s) passed 0 test(s) skipped 0 test(s) failed * Running on OpenCL device 'AMD Radeon RX 6400'. * Starting tests for the 'iHAMAX' routine. Legend: : -> Test produced correct results . -> Test returned the correct error code X -> Test produced incorrect results / -> Test returned an incorrect error code \ -> Test not executed: OpenCL-kernel compilation error o -> Test not executed: Unsupported precision - -> Test not completed: Reference CBLAS doesn't output error codes * Testing with error margins of 8.0% (relative) and 0.150 (absolute) * and a combined maximum allowed L2 error of 5.00e-02 * Testing 'regular behaviour' for 'default': XXXXXXXXX:X: Error rate 0.78%: n=7 incx=1 offx=0 Error rate 0.78%: n=7 incx=2 offx=0 Error rate 0.78%: n=7 incx=7 offx=0 Error rate 0.78%: n=93 incx=1 offx=0 Error rate 0.78%: n=93 incx=2 offx=0 Error rate 0.78%: n=93 incx=7 offx=0 Error rate 0.78%: n=144 incx=1 offx=0 Error rate 0.78%: n=144 incx=2 offx=0 Error rate 0.78%: n=144 incx=7 offx=0 Error rate 1.56%: n=4096 incx=2 offx=0 Pass rate 16.7%: 2 passed / 0 skipped / 10 failed * Completed all test-cases for this routine. Results: 2 test(s) passed 0 test(s) skipped 10 test(s) failed * Running on OpenCL device 'AMD Radeon RX 6400'. * Starting tests for the 'HAXPYBATCHED' routine. Legend: : -> Test produced correct results . -> Test returned the correct error code X -> Test produced incorrect results / -> Test returned an incorrect error code \ -> Test not executed: OpenCL-kernel compilation error o -> Test not executed: Unsupported precision - -> Test not completed: Reference CBLAS doesn't output error codes * Testing with error margins of 8.0% (relative) and 0.150 (absolute) * and a combined maximum allowed L2 error of 5.00e-02 * Testing 'regular behaviour' for 'default': ::X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:XXX:X:X:X:X:X:X:X:XXX:XXX:X:X: X:X:X:X: Error rate 0.75%: n=7 incx=1 incy=2 batch_num=1 alpha=3.138672 Error rate 0.75%: n=7 incx=1 incy=7 batch_num=1 alpha=3.138672 Error rate 0.75%: n=7 incx=2 incy=1 batch_num=1 alpha=3.138672 Error rate 0.75%: n=7 incx=2 incy=2 batch_num=1 alpha=3.138672 Error rate 0.75%: n=7 incx=2 incy=7 batch_num=1 alpha=3.138672 Error rate 0.75%: n=7 incx=7 incy=1 batch_num=1 alpha=3.138672 Error rate 0.75%: n=7 incx=7 incy=2 batch_num=1 alpha=3.138672 Error rate 0.75%: n=7 incx=7 incy=7 batch_num=1 alpha=3.138672 Error rate 0.45%: n=93 incx=1 incy=1 batch_num=1 alpha=3.138672 Error rate 0.45%: n=93 incx=1 incy=2 batch_num=1 alpha=3.138672 Error rate 0.45%: n=93 incx=1 incy=7 batch_num=1 alpha=3.138672 Error rate 0.45%: n=93 incx=2 incy=1 batch_num=1 alpha=3.138672 Error rate 0.45%: n=93 incx=2 incy=2 batch_num=1 alpha=3.138672 Error rate 0.45%: n=93 incx=2 incy=7 batch_num=1 alpha=3.138672 Error rate 0.45%: n=93 incx=7 incy=1 batch_num=1 alpha=3.138672 Error rate 0.45%: n=93 incx=7 incy=2 batch_num=1 alpha=3.138672 Error rate 0.45%: n=93 incx=7 incy=7 batch_num=1 alpha=3.138672 Error rate 0.25%: n=93 incx=7 incy=7 batch_num=3 alpha=3.138672 Error rate 0.37%: n=144 incx=1 incy=1 batch_num=1 alpha=3.138672 Error rate 0.37%: n=144 incx=1 incy=2 batch_num=1 alpha=3.138672 Error rate 0.37%: n=144 incx=1 incy=7 batch_num=1 alpha=3.138672 Error rate 0.37%: n=144 incx=2 incy=1 batch_num=1 alpha=3.138672 Error rate 0.37%: n=144 incx=2 incy=2 batch_num=1 alpha=3.138672 Error rate 0.37%: n=144 incx=2 incy=7 batch_num=1 alpha=3.138672 Error rate 0.37%: n=144 incx=7 incy=1 batch_num=1 alpha=3.138672 Error rate 0.37%: n=144 incx=7 incy=2 batch_num=1 alpha=3.138672 Error rate 0.37%: n=144 incx=7 incy=7 batch_num=1 alpha=3.138672 Error rate 0.18%: n=144 incx=7 incy=7 batch_num=3 alpha=3.138672 Error rate 0.02%: n=4096 incx=1 incy=1 batch_num=1 alpha=3.138672 Error rate 0.02%: n=4096 incx=1 incy=2 batch_num=1 alpha=3.138672 Error rate 0.01%: n=4096 incx=1 incy=2 batch_num=3 alpha=3.138672 Error rate 0.02%: n=4096 incx=1 incy=7 batch_num=1 alpha=3.138672 Error rate 0.02%: n=4096 incx=2 incy=1 batch_num=1 alpha=3.138672 Error rate 0.02%: n=4096 incx=2 incy=2 batch_num=1 alpha=3.138672 Error rate 0.02%: n=4096 incx=2 incy=7 batch_num=1 alpha=3.138672 Error rate 0.02%: n=4096 incx=7 incy=1 batch_num=1 alpha=3.138672 Error rate 0.02%: n=4096 incx=7 incy=2 batch_num=1 alpha=3.138672 Error rate 0.02%: n=4096 incx=7 incy=7 batch_num=1 alpha=3.138672 Pass rate 47.2%: 34 passed / 0 skipped / 38 failed * Completed all test-cases for this routine. Results: 34 test(s) passed 0 test(s) skipped 38 test(s) failed Running test set xgemmbatched. -------------------------------- * Running on OpenCL device 'AMD Radeon RX 6400'. * Starting tests for the 'HGEMMBATCHED' routine. Legend: : -> Test produced correct results . -> Test returned the correct error code X -> Test produced incorrect results / -> Test returned an incorrect error code \ -> Test not executed: OpenCL-kernel compilation error o -> Test not executed: Unsupported precision - -> Test not completed: Reference CBLAS doesn't output error codes * Testing with error margins of 8.0% (relative) and 0.150 (absolute) * and a combined maximum allowed L2 error of 5.00e-02 * Testing 'regular behaviour' for '102 (col-major) 112 (transposed) 111 (regular)': XXXXX:X:XXXXXXXX------------XXXXXXXXXXX:XXXXXXX:------------XXX: --XX--XX--XX--XX--------------XX--XX--X:--X:--X:--------------X: Error rate 0.57%: m=7 n=7 k=7 lda=7 ldb=7 ldc=7 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.36%: m=7 n=7 k=7 lda=7 ldb=7 ldc=7 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.57%: m=7 n=7 k=7 lda=7 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.36%: m=7 n=7 k=7 lda=7 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.57%: m=7 n=7 k=7 lda=7 ldb=64 ldc=7 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.57%: m=7 n=7 k=7 lda=7 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.57%: m=7 n=7 k=7 lda=64 ldb=7 ldc=7 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.36%: m=7 n=7 k=7 lda=64 ldb=7 ldc=7 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.57%: m=7 n=7 k=7 lda=64 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.36%: m=7 n=7 k=7 lda=64 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.57%: m=7 n=7 k=7 lda=64 ldb=64 ldc=7 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.36%: m=7 n=7 k=7 lda=64 ldb=64 ldc=7 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.57%: m=7 n=7 k=7 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.36%: m=7 n=7 k=7 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.57%: m=7 n=7 k=64 lda=64 ldb=64 ldc=7 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.36%: m=7 n=7 k=64 lda=64 ldb=64 ldc=7 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.57%: m=7 n=7 k=64 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.36%: m=7 n=7 k=64 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=7 n=64 k=7 lda=7 ldb=7 ldc=7 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=7 n=64 k=7 lda=7 ldb=7 ldc=7 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=7 n=64 k=7 lda=7 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=7 n=64 k=7 lda=7 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=7 n=64 k=7 lda=7 ldb=64 ldc=7 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=7 n=64 k=7 lda=7 ldb=64 ldc=7 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=7 n=64 k=7 lda=7 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=7 n=64 k=7 lda=64 ldb=7 ldc=7 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=7 n=64 k=7 lda=64 ldb=7 ldc=7 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=7 n=64 k=7 lda=64 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=7 n=64 k=7 lda=64 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=7 n=64 k=7 lda=64 ldb=64 ldc=7 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=7 n=64 k=7 lda=64 ldb=64 ldc=7 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=7 n=64 k=7 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=7 n=64 k=64 lda=64 ldb=64 ldc=7 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=7 n=64 k=64 lda=64 ldb=64 ldc=7 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=7 n=64 k=64 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=64 n=7 k=7 lda=7 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=64 n=7 k=7 lda=7 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=64 n=7 k=7 lda=7 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=64 n=7 k=7 lda=7 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=64 n=7 k=7 lda=64 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=64 n=7 k=7 lda=64 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=64 n=7 k=7 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=64 n=7 k=7 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.17%: m=64 n=7 k=64 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.07%: m=64 n=7 k=64 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.02%: m=64 n=64 k=7 lda=7 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.01%: m=64 n=64 k=7 lda=7 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=3 alpha=3.138672 beta=3.138672 Error rate 0.02%: m=64 n=64 k=7 lda=7 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.02%: m=64 n=64 k=7 lda=64 ldb=7 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.02%: m=64 n=64 k=7 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Error rate 0.02%: m=64 n=64 k=64 lda=64 ldb=64 ldc=64 offa=0 offb=0 offc=0 batch_num=1 alpha=3.138672 beta=3.138672 Pass rate 7.0%: 9 passed / 68 skipped / 51 failed * Completed all test-cases for this routine. Results: 56 test(s) passed 556 test(s) skipped 412 test(s) failed * Running on OpenCL device 'AMD Radeon RX 6400'. * Starting tests for the 'HAXPYBATCHED' routine. Legend: : -> Test produced correct results . -> Test returned the correct error code X -> Test produced incorrect results / -> Test returned an incorrect error code \ -> Test not executed: OpenCL-kernel compilation error o -> Test not executed: Unsupported precision - -> Test not completed: Reference CBLAS doesn't output error codes * Testing with error margins of 8.0% (relative) and 0.150 (absolute) * and a combined maximum allowed L2 error of 5.00e-02 * Testing 'regular behaviour' for 'default': ::X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:XXX:X:X:X:X:X:X:X:XXX:XXX:X:X: X:X:X:X: Thank you -- System Information: Debian Release: 13.0 APT prefers testing-security APT policy: (500, 'testing-security'), (500, 'testing') Architecture: amd64 (x86_64) Kernel: Linux 6.12.32-amd64 (SMP w/128 CPU threads; PREEMPT) Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled
