On Wed, Jul 22, 2026 at 10:16 AM Priscila Gutierres via devel <[email protected]> wrote: > > Hello! > > I wanted to bring up something I noticed about the current Ollama package > in Fedora (0.12.11 / 0.24.0 in rawhide). The package ships with ROCm and > Vulkan GPU backends, but there is no CUDA backend at all. This means users > with NVIDIA GPUs can only use the Vulkan path > for GPU acceleration, which is noticeably slower than CUDA for LLM > inference workloads. > > I'm running a laptop with an RTX 3060 (6 GB VRAM) and 64 GB of RAM. Many of > the models I run don't fit entirely in VRAM, so some layers get offloaded to > the CPU. > In this scenario, both the GPU and CPU inference paths matter a lot. On the > CPU side, the package already does a great job by shipping multiple > architecture-specific builds (Alder Lake, Haswell, Ice Lake, etc.) with the > right SIMD flags for each, so the layers that run in RAM are well optimized. > But on the GPU side, the layers that should be running fast in VRAM are > going through Vulkan instead of CUDA. The result is that half of the > inference pipeline is properly optimized and the other half is not, which > becomes especially noticeable with partially offloaded models where both > paths are active at the same time. > I know CUDA is proprietary and cannot live in Fedora's official > repositories, which is why the package only ships ROCm and Vulkan today. > But the package already splits GPU backends into separate subpackages > (ollama-rocm, ollama-vulkan), so the structure for an ollama-cudasubpackage > is already in place. Since the NVIDIA drivers and CUDA toolkit are already > available through RPM Fusion, it seems like a natural fit to offer > ollama-cuda there as well, following the same pattern. This would save > NVIDIA users from having to compile from source or fall back to the upstream > install script (which bypasses the package manager entirely and breaks on > every kernel or driver update, as discussed in the Getting ollama to use GPU > thread (Fedora Discussion). > > It is worth noting that a community COPR already solves this, the > fachep/ollama repository packages an ollama-cuda subpackage built against > CUDA 12.8, covering a wide range of GPU architectures from Maxwell to > Blackwell. > Given that this work already exists and is functional, would it be possible > to incorporate it into the official Fedora package as an RPM Fusion > dependency, or coordinate with the COPR maintainer to move it into RPM > Fusion proper? > Thanks for all the work on the Ollama package. The CPU variant builds and > the overall packaging quality are really solid. >
RPM Fusion is not part of the Fedora Project. You'll need to talk to them if you want ollama-cuda. From the Fedora point of view, I would hope that we'd push harder for everything to use Vulkan Compute or OpenCL instead of these disparate proprietary APIs. And while ROCm is open source, it is a vendor-specific API, and I would love for us to not rely on that in the future. -- 真実はいつも一つ！/ Always, there's only one truth! -- _______________________________________________ devel mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected] Do not reply to spam, report it: https://forge.fedoraproject.org/infra/tickets/issues/new
