By the end of this guide your Fedora machine will run language models on its NVIDIA card: driver installed and signed, a first model in Ollama, and the CUDA toolkit, PyTorch and GPU containers if you need them. The cheapest card in our catalogue that this applies to is an RTX 3060 with 12 GB.
We did not run the Fedora steps. We do not own a Fedora machine. Ours are NVIDIA DGX Sparks on Ubuntu 24.04 (aarch64, CUDA 13). Every Fedora command below comes from the linked source, read on 16 November 2026, and is labelled with it. The outputs we show are from our Ubuntu machine and say so.
Where each step comes from
| Source (all read 16 Nov 2026) | Used for | Driver route it recommends |
|---|---|---|
| RPM Fusion NVIDIA how-to, edited 14 Nov 2026 | driver, build wait, legacy cards, Atomic desktops | its own akmod-nvidia |
| RPM Fusion Secure Boot how-to | key generation and MOK enrolment | n/a |
| RPM Fusion CUDA how-to, edited 25 Sep 2026 | toolkit next to the RPM Fusion driver | RPM Fusion driver, NVIDIA toolkit |
| NVIDIA driver installation guide, Fedora | NVIDIA’s own route | NVIDIA’s nvidia-open |
| NVIDIA CUDA installation guide for Linux, version 13.5 | toolkit, supported Fedora and GCC | n/a since 13.4: the toolkit package installs no driver |
| NVIDIA Container Toolkit install guide, 1.20.1 | Podman and Docker | any working driver |
| Ollama on Linux, GPU page, troubleshooting | first model, error codes | n/a |
Fedora’s documentation site refused our automated reader, so nothing here rests on Fedora’s own pages. The Secure Boot steps are RPM Fusion’s. Read the Fedora 45 release notes yourself before upgrading; the link is the one in Fedora Magazine’s announcement.
What you need
- Fedora Workstation 44 or 45 on x86_64, or an Atomic desktop (the differences are in the FAQ).
- An NVIDIA card. RPM Fusion’s default package covers current cards; GTX 800, 900 and 10 series use
akmod-nvidia-580xxinstead, with CUDA up to 12.x. - Ollama asks for driver 550 or newer, and 570 or newer for cards with compute capability 5.0 to 6.2, per its GPU page.
- A new card if you have none: the RTX 3060 12 GB was EUR 509.00 at PcComponentes on 16 November 2026 in our hardware catalogue.
- Root access and one or two reboots.
Step 1: choose one driver route
| RPM Fusion | NVIDIA’s repository | |
|---|---|---|
| Package | akmod-nvidia | nvidia-open |
| Module build | akmods, on your machine | DKMS, on your machine |
| Fedora releases | follows Fedora; the setup command reads your release number | Fedora 44 is the one its guide names |
| Driver branch (16 Nov) | 595 or newer on Fedora 44 and later, per its page | up to 620.70.06 in the fedora44 repository |
| Secure Boot | documented, with akmods signing | not mentioned on NVIDIA’s Fedora page |
The trade-off is plain. NVIDIA’s repository gives you the newest branch and NVIDIA’s support, on the one release it has qualified. RPM Fusion tracks Fedora’s kernels and tells you to use its driver “in order to receive fixes needed by Fedora kernel update”.
The rest of this guide follows RPM Fusion. NVIDIA’s route is one short section, kept separate. Do not combine them.
Step 2: check the card and Secure Boot
RPM Fusion’s detection commands, then the Secure Boot state:
/sbin/lspci | grep -e VGA
/sbin/lspci | grep -e 3D # a second line here means an Optimus laptop
mokutil --sb-stateOn our Ubuntu machine the last command prints SecureBoot enabled. If yours says the same, do step 3 before installing the driver. If it is disabled, skip to step 4.
Step 3: enrol a signing key (Secure Boot on)
A kernel with Secure Boot enforced loads only modules signed by a key the firmware trusts. RPM Fusion’s page is direct: “you must sign the nvidia kmod”. You do not have to turn Secure Boot off. From its Secure Boot how-to:
sudo dnf install kmodtool akmods mokutil openssl
sudo kmodgenca -a
sudo mokutil --import /etc/pki/akmods/certs/public_key.der
systemctl rebootmokutil asks you to invent a password. On the next boot a blue MOK screen appears: choose Enroll MOK, Continue, Yes, and type that password. The how-to warns that this screen uses a QWERTY layout, which matters on a Spanish keyboard.
Two cautions from the same page. After a firmware update you may have to import the key again. And the private key sits in /etc/pki/akmods, so the page treats an encrypted root disk as required.
Step 4: install the driver from RPM Fusion
Enable the repositories with the command on RPM Fusion’s configuration page, then install:
sudo dnf install \
https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm \
https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm
sudo dnf update -y
sudo dnf install akmod-nvidia
sudo dnf install xorg-x11-drv-nvidia-cudaRPM Fusion marks the second package as optional, for CUDA and video encoding support. For this guide it is not optional.
Now wait. The module is compiled on your machine after the transaction ends, and the how-to allows up to five minutes for it. Check before you reboot:
modinfo -F version nvidiaA version number means the module is built. On our Ubuntu machine the same command prints 580.173.02. If you get modinfo: ERROR: Module nvidia not found., it is still building. Wait and repeat.
The how-to notes that Turing and later cards get NVIDIA’s open kernel module by default. You have nothing to choose.
NVIDIA’s own route instead (Fedora 44 only)
From NVIDIA’s driver guide, as root, and only if you skipped the RPM Fusion driver:
dnf install kernel-devel-matched kernel-headers
dnf config-manager addrepo --from-repofile=https://developer.download.nvidia.com/compute/cuda/repos/fedora44/x86_64/cuda-fedora44.repo
dnf clean expire-cache
dnf install nvidia-open
rebootThat page does not cover Secure Boot. NVIDIA’s driver README says only that many distributions require signed modules under Secure Boot and that the key should be trusted before you install.
Step 5: reboot and verify
sudo reboot
lsmod | grep nouveau # must print nothing (RPM Fusion)
cat /proc/driver/nvidia/version # NVIDIA's own check
nvidia-smiThis is the top of nvidia-smi on our Ubuntu machine, 16 November 2026. On Fedora the layout is the same, with your card and your versions:
| NVIDIA-SMI 580.173.02 Driver Version: 580.173.02 CUDA Version: 13.0 |
| 0 NVIDIA GB10 On | 0000000F:01:00.0 Off | N/A |
| N/A 48C P0 12W / N/A | Not Supported | 0% Default |Read CUDA Version correctly. It is the newest CUDA the driver can serve, not an installed toolkit. A machine with no toolkit still shows it.
Our memory column reads Not Supported because the GB10 shares one memory pool between CPU and GPU. A discrete card shows used and total MiB there.
Step 6: a first model with Ollama
curl -fsSL https://ollama.com/install.sh | sh
ollama -v
ollama run llama3.1:8b "Summarise in one sentence: the driver is in, the model runs."
ollama psollama ps is the proof. This is ours, on Ubuntu, with Ollama 0.30.10:
NAME ID SIZE PROCESSOR CONTEXT UNTIL
llama3.1:8b 4fc21a39517b 9.2 GB 100% GPU 32768 48 minutes from nowYou want 100% GPU. A CPU share in that column means the model plus its context did not fit in the card. Our 9.2 GB is this model at a 32k context; the VRAM guide shows the sum.
Logs are at journalctl -e -u ollama. If you have never used Ollama, our install guide covers the basics.
Step 7: the CUDA toolkit, only if you compile
You do not need it for Ollama, which ships its own CUDA libraries: our install has cuda_v12 and cuda_v13 folders under /usr/local/lib/ollama. You do not need it for PyTorch from pip either: on 16 November the default torch 2.15.0 on PyPI declared pip-packaged CUDA 13.2 runtime libraries as dependencies on Linux. You need the toolkit to build llama.cpp or any other CUDA code yourself.
RPM Fusion’s CUDA how-to keeps its driver and takes only the toolkit from NVIDIA. Its section for Fedora 44 and later:
sudo dnf config-manager addrepo --from-repofile=https://developer.download.nvidia.com/compute/cuda/repos/fedora44/$(uname -m)/cuda-fedora44.repo
sudo dnf clean all
sudo dnf config-manager setopt cuda-fedora44-$(uname -m).exclude=nvidia-driver\*,cuda-drivers\*,nvidia-modprobe,nvidia-persistenced,nvidia-settings,nvidia-libXNVCtrl\*,nvidia-xconfig,nvidia-open,nvidia-imex,nvidia-kmod-common,libnvidia-fbc,kmod-nvidia-\*-dkms,nvidia-fs\*,nvidia-gds\*,xorg-x11-nvidia\*
sudo dnf -y install cuda-toolkit xorg-x11-drv-nvidia-cuda
export PATH=${PATH}:/usr/local/cuda-13.5/bin
nvcc --versionThe exclude line is the important one. It stops NVIDIA’s driver packages from replacing RPM Fusion’s. The PATH line is from NVIDIA’s guide for 13.5; adjust the number to what you installed.
On our Ubuntu machine nvcc --version ends with Cuda compilation tools, release 13.0, V13.0.88. On 16 November the fedora44 repository offered toolkit 13.5.0.
Then llama.cpp, from its build guide:
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config ReleaseTo choose a model file for it, see which GGUF quantisation to pick.
Step 8: PyTorch with the matching wheel
Pick the build in PyTorch’s install selector. On 16 November the wheel index carried cu126, cu130, cu132 and cu134 builds, and 2.15.0 existed for the last two. Inside a virtual environment:
python3 -m venv ~/venvs/torch && source ~/venvs/torch/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/cu132
python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available())"On our Ubuntu machine that last line prints 2.12.1+cu130 13.0 True. You want True and a CUDA version your driver supports. We could not read the selector’s own text with our tools, so the command above follows the index, not the selector.
Step 9: GPU containers with Podman or Docker
From NVIDIA’s Container Toolkit guide, with the driver already working:
curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo dnf install -y nvidia-container-toolkit
nvidia-ctk cdi list
podman run --rm --security-opt=label=disable --device=nvidia.com/gpu=all ubuntu nvidia-smiFor Podman, NVIDIA recommends CDI. Since toolkit 1.18.0 a service called nvidia-cdi-refresh writes the device file to /var/run/cdi/nvidia.yaml on install, on driver upgrade and at boot. The --security-opt=label=disable flag turns off SELinux separation for that container; NVIDIA’s own example uses it.
Docker needs one more step:
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smiThe guide pins a version (1.20.1-1 on the day we read it). We show the unpinned install for brevity.
What we measured, on Ubuntu
None of this is a Fedora measurement. The driver and the model are the same software on both systems; the card is not.
| Hardware | System | Model | Speed | Source |
|---|---|---|---|---|
| NVIDIA DGX Spark (GB10, 128 GB unified) | Ubuntu 24.04, driver 580.173.02, Ollama 0.30.10 | llama3.1:8b, Q4_K_M, 32k context | 40.2 tok/s, 0.365 s to first token, 9.21 GB loaded | ours, 17 Oct 2026, result file |
| RTX 3060 12 GB, Core i7-12700F | Debian 13, llama.cpp built from source | Llama 3.1 8B Instruct, Q4_K_M, context not stated | 64.5 tok/s | tyolab, May 2026, their own run |
The second row is not ours and is not Fedora. It is the nearest first-hand figure we found for the card most readers will buy. Measure your own with our 30-minute benchmark.
What our CUDA 13 machines taught us
Three things from our operations notes that are not specific to our hardware.
- A wheel built for another CUDA major installs without complaint and fails later. pip does not check the CUDA ABI. Our notes record the failure mode for a CUDA 12 wheel on a CUDA 13 host: an undefined symbol at import, or a crash at the first GPU call. Our rule: one virtual environment per tool, torch from the matching index, and the one-line check from step 8 after every install.
- When
torch.cuda.is_available()isFalse, check in this order.nvidia-smiinside the container; an empty or staleCUDA_VISIBLE_DEVICES; permissions on/dev/nvidia*in rootless containers; a stuck context from a crashed process; and only then the wheel. Reinstalling the wheel fixes only the last one. - One card, one memory pool, many tenants. Our fine-tuning runs were killed while another model server sat resident on the same memory. On a 12 GB card the same thing happens sooner: Ollama, a notebook and a container all draw from one pool.
nvidia-smi --query-compute-apps=pid,used_memory --format=csvshows who holds what.
Failures the sources document
| What you see | Cause, per the source | Documented fix |
|---|---|---|
modinfo: ERROR: Module nvidia not found. | the module is still building (RPM Fusion) | wait up to 5 minutes, repeat, then reboot |
| Driver does not load with Secure Boot on | the kernel accepts only modules signed by a key it trusts (NVIDIA README) | step 3; re-import the key after a firmware update |
lsmod | grep nouveau prints a line | nouveau holds the card. The README’s installer message: Nouveau, or another driver, is already using the GPU. | reboot once the module is built; on Atomic, add RPM Fusion’s rpm-ostree kargs blacklist line |
| No driver after a kernel update | the module is built per kernel; the README has an entry for exactly this | akmods checks for missing modules at startup; test with modinfo -F version nvidia -k <kernel>, force with sudo akmods --kernels $(uname -r) --rebuild |
nvcc rejects your compiler | CUDA 13.5 supports GCC 6.x to 16.x and checks the major version only (NVIDIA) | compare gcc --version; on Fedora 44 NVIDIA documents dnf install gcc15-c++ and export NVCC_CCBIN='g++-15' |
| Ollama answers on CPU; the log shows code 3, 46, 100 or 999 | GPU initialisation failed (Ollama) | sudo nvidia-modprobe -u, reload nvidia_uvm, reboot, newer driver, in that order |
| Ollama loses the GPU after suspend | a driver bug, says Ollama | sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm |
Failed to initialize NVML: Insufficient Permissions in a container | SELinux relabelling (NVIDIA) | add --security-opt=label=disable |
Failed to initialize NVML: Unknown Error in a running container | the container lost its device access | recreate the container; CDI avoids it |
nvcc warning : Cannot find valid GPU for '-arch=native', default arch is used | llama.cpp’s build could not detect the card | add -DGGML_NATIVE=OFF or set -DCMAKE_CUDA_ARCHITECTURES |
We did not find the exact nvcc compiler error at a source we could open, so the table does not quote one.
What fits on the card
We will not repeat the sum here. The VRAM guide derives it from weights, context and parallel slots, and the VRAM explorer does it for your card. The short version from our catalogue: at 8k of context a 12 GB card holds an 8B model and a 16 GB card a 14B.
Where this fits, and when Fedora is the wrong choice
A Fedora desktop with a 12 GB card is a sound base for one person or a small team running 8B models, with the documents staying on the machine. Add Open WebUI when more than one person needs it.
Fedora is the wrong choice in these cases:
- You need a configuration NVIDIA supports. Its guides name one Fedora release. On 16 November that was Fedora 44, while Fedora 45 was two weeks old and its NVIDIA repository held only monitoring packages. Fedora 43 reaches end of life on 1 December 2026, per endoflife.date. NVIDIA’s same table lists three Ubuntu LTS releases and RHEL 8, 9 and 10.
- The machine must not surprise you. Fedora ships new kernels often and each one needs a module rebuild. RPM Fusion documents a version lock, and warns that it also blocks kernel-compatibility fixes.
- You upgrade on release day. Wait until the driver for the new kernel exists, and the toolkit repository if you compile.
- You have no NVIDIA card. On Intel Arc, see llama.cpp with SYCL.
Limits
- We ran none of the Fedora commands. A typo in a source would be a typo here.
- We could not read Fedora’s own documentation or PyTorch’s selector with our tools. Both are linked for you to check.
- We did not confirm that RPM Fusion had built
akmod-nvidiafor Fedora 45’s kernel on the day. Check withdnf info akmod-nvidiabefore you upgrade. - llama.cpp’s own Fedora guide builds in a Fedora 41 toolbox, a release past end of life. We left it out.
- Optimus laptops, multiple GPUs and Wayland-specific problems are not covered.
FAQ
Do I need the CUDA toolkit to run Ollama? No. The driver is enough. Install the toolkit only to compile CUDA code.
Can I keep Secure Boot on? Yes. RPM Fusion’s how-to says disabling it is not required. Enrol the key first (step 3).
Does NVIDIA support Fedora 45? Not on 16 November 2026. Its driver and CUDA guides name Fedora 44. RPM Fusion’s CUDA section covers Fedora 44 and later with the fedora44 repository.
Can I install NVIDIA’s driver over RPM Fusion’s to get a newer one? Not as a normal install. RPM Fusion says the two packagings conflict, which is why its toolkit commands exclude NVIDIA’s driver packages.
Does it work under Wayland? RPM Fusion says the driver works under Wayland and Xwayland from Fedora 35, with known limitations.
What about Silverblue or Kinoite? RPM Fusion documents sudo rpm-ostree install akmod-nvidia xorg-x11-drv-nvidia-cuda plus a kernel-argument line that blacklists nouveau. Secure Boot there needs a separate key package, which its page links.
Which CUDA version does my PyTorch need? The one in the wheel’s name. The wheel carries its runtime, so the installed toolkit does not have to match. The driver has to be new enough.
My GTX 1060 is not in the default driver. Is it finished? No. RPM Fusion keeps akmod-nvidia-580xx for the GTX 10 series, and Ollama accepts driver 570 or newer for those cards.
Next steps
- Size the model before you download it: how much VRAM a local LLM needs.
- Measure your card: benchmark a local model in 30 minutes.
- Plan the rest of the stack: how to deploy AI locally in your business.
Work with us
We set up local inference on hardware clients already own before recommending anything new, and we leave a written runbook behind. If you want that on your machines, get in touch or read how our consulting works.