# CUDA on Fedora: driver, Secure Boot, toolkit, first model

> The sourced path to local AI on Fedora 44 and 45: NVIDIA driver, Secure Boot, CUDA toolkit, GPU containers, Ollama and PyTorch. Sources read 16 Nov 2026.

- Published: 2026-09-09
- Updated: 2026-11-16
- Author: Jacobo González Jaspe
- Language: en
- Tags: Fedora, CUDA, Ollama, llama.cpp, Guide
- Canonical URL: https://vorluxai.com/en/blog/cuda-on-fedora-set-up-a-local-ai-workstation-en/
- Source: VORLUX AI (J4SGON S.L.), https://vorluxai.com/

---

By the end of this guide your Fedora machine will run language models on its NVIDIA card: driver installed and signed, a first model in Ollama, and the CUDA toolkit, PyTorch and GPU containers if you need them. The cheapest card in our catalogue that this applies to is an RTX 3060 with 12 GB.

> **We did not run the Fedora steps.** We do not own a Fedora machine. Ours are NVIDIA DGX Sparks on Ubuntu 24.04 (aarch64, CUDA 13). Every Fedora command below comes from the linked source, read on 16 November 2026, and is labelled with it. The outputs we show are from our Ubuntu machine and say so.

> **TL;DR**
> - Two driver routes exist: RPM Fusion's packages, or NVIDIA's own repository. Pick one. RPM Fusion and NVIDIA's packaging conflict, and RPM Fusion's page says so.
> - NVIDIA's guides name Fedora 44. Fedora 45 came out on 3 November 2026 and NVIDIA's `fedora45` repository held no driver and no toolkit when we listed it on 16 November.
> - Ollama and the prebuilt PyTorch wheels bring their own CUDA runtime. You need the toolkit only to compile.
> - Verdict: on Fedora 44 or 45, use RPM Fusion for the driver, enrol the signing key before you install, and stop at Ollama unless you have a reason to go further.

## Where each step comes from

| Source (all read 16 Nov 2026) | Used for | Driver route it recommends |
|---|---|---|
| [RPM Fusion NVIDIA how-to](https://rpmfusion.org/Howto/NVIDIA), edited 14 Nov 2026 | driver, build wait, legacy cards, Atomic desktops | its own `akmod-nvidia` |
| [RPM Fusion Secure Boot how-to](https://rpmfusion.org/Howto/Secure%20Boot) | key generation and MOK enrolment | n/a |
| [RPM Fusion CUDA how-to](https://rpmfusion.org/Howto/CUDA), edited 25 Sep 2026 | toolkit next to the RPM Fusion driver | RPM Fusion driver, NVIDIA toolkit |
| [NVIDIA driver installation guide, Fedora](https://docs.nvidia.com/datacenter/tesla/driver-installation-guide/fedora.html) | NVIDIA's own route | NVIDIA's `nvidia-open` |
| [NVIDIA CUDA installation guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/), version 13.5 | toolkit, supported Fedora and GCC | n/a since 13.4: the toolkit package installs no driver |
| [NVIDIA Container Toolkit install guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html), 1.20.1 | Podman and Docker | any working driver |
| [Ollama on Linux](https://docs.ollama.com/linux), [GPU page](https://docs.ollama.com/gpu), [troubleshooting](https://docs.ollama.com/troubleshooting) | first model, error codes | n/a |

Fedora's documentation site refused our automated reader, so nothing here rests on Fedora's own pages. The Secure Boot steps are RPM Fusion's. Read the [Fedora 45 release notes](https://docs.fedoraproject.org/en-US/fedora/f45/release-notes/) yourself before upgrading; the link is the one in [Fedora Magazine's announcement](https://fedoramagazine.org/announcing-fedora-linux-45/).

## What you need

- Fedora Workstation 44 or 45 on x86_64, or an Atomic desktop (the differences are in the FAQ).
- An NVIDIA card. RPM Fusion's default package covers current cards; GTX 800, 900 and 10 series use `akmod-nvidia-580xx` instead, with CUDA up to 12.x.
- Ollama asks for driver 550 or newer, and 570 or newer for cards with compute capability 5.0 to 6.2, per its GPU page.
- A new card if you have none: the RTX 3060 12 GB was EUR 509.00 at PcComponentes on 16 November 2026 in our [hardware catalogue](https://vorluxai.com/en/blog/local-ai-hardware-catalog-2026-what-to-buy-at-each-budget-en/).
- Root access and one or two reboots.

## Step 1: choose one driver route

| | RPM Fusion | NVIDIA's repository |
|---|---|---|
| Package | `akmod-nvidia` | `nvidia-open` |
| Module build | akmods, on your machine | DKMS, on your machine |
| Fedora releases | follows Fedora; the setup command reads your release number | Fedora 44 is the one its guide names |
| Driver branch (16 Nov) | 595 or newer on Fedora 44 and later, per its page | up to 620.70.06 in the `fedora44` repository |
| Secure Boot | documented, with akmods signing | not mentioned on NVIDIA's Fedora page |

The trade-off is plain. NVIDIA's repository gives you the newest branch and NVIDIA's support, on the one release it has qualified. RPM Fusion tracks Fedora's kernels and tells you to use its driver "in order to receive fixes needed by Fedora kernel update".

The rest of this guide follows RPM Fusion. NVIDIA's route is one short section, kept separate. Do not combine them.

## Step 2: check the card and Secure Boot

RPM Fusion's detection commands, then the Secure Boot state:

```bash
/sbin/lspci | grep -e VGA
/sbin/lspci | grep -e 3D      # a second line here means an Optimus laptop
mokutil --sb-state
```

On our Ubuntu machine the last command prints `SecureBoot enabled`. If yours says the same, do step 3 before installing the driver. If it is disabled, skip to step 4.

## Step 3: enrol a signing key (Secure Boot on)

A kernel with Secure Boot enforced loads only modules signed by a key the firmware trusts. RPM Fusion's page is direct: "you must sign the nvidia kmod". You do not have to turn Secure Boot off. From its Secure Boot how-to:

```bash
sudo dnf install kmodtool akmods mokutil openssl
sudo kmodgenca -a
sudo mokutil --import /etc/pki/akmods/certs/public_key.der
systemctl reboot
```

`mokutil` asks you to invent a password. On the next boot a blue MOK screen appears: choose Enroll MOK, Continue, Yes, and type that password. The how-to warns that this screen uses a QWERTY layout, which matters on a Spanish keyboard.

Two cautions from the same page. After a firmware update you may have to import the key again. And the private key sits in `/etc/pki/akmods`, so the page treats an encrypted root disk as required.

## Step 4: install the driver from RPM Fusion

Enable the repositories with the command on [RPM Fusion's configuration page](https://rpmfusion.org/Configuration), then install:

```bash
sudo dnf install \
  https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm \
  https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm
sudo dnf update -y
sudo dnf install akmod-nvidia
sudo dnf install xorg-x11-drv-nvidia-cuda
```

RPM Fusion marks the second package as optional, for CUDA and video encoding support. For this guide it is not optional.

Now wait. The module is compiled on your machine after the transaction ends, and the how-to allows up to five minutes for it. Check before you reboot:

```bash
modinfo -F version nvidia
```

A version number means the module is built. On our Ubuntu machine the same command prints `580.173.02`. If you get `modinfo: ERROR: Module nvidia not found.`, it is still building. Wait and repeat.

The how-to notes that Turing and later cards get NVIDIA's open kernel module by default. You have nothing to choose.

### NVIDIA's own route instead (Fedora 44 only)

From NVIDIA's driver guide, as root, and only if you skipped the RPM Fusion driver:

```bash
dnf install kernel-devel-matched kernel-headers
dnf config-manager addrepo --from-repofile=https://developer.download.nvidia.com/compute/cuda/repos/fedora44/x86_64/cuda-fedora44.repo
dnf clean expire-cache
dnf install nvidia-open
reboot
```

That page does not cover Secure Boot. NVIDIA's driver README says only that many distributions require signed modules under Secure Boot and that the key should be trusted before you install.

## Step 5: reboot and verify

```bash
sudo reboot
lsmod | grep nouveau          # must print nothing (RPM Fusion)
cat /proc/driver/nvidia/version   # NVIDIA's own check
nvidia-smi
```

This is the top of `nvidia-smi` on our Ubuntu machine, 16 November 2026. On Fedora the layout is the same, with your card and your versions:

```text
| NVIDIA-SMI 580.173.02             Driver Version: 580.173.02     CUDA Version: 13.0     |
|   0  NVIDIA GB10                    On  |   0000000F:01:00.0 Off |                  N/A |
| N/A   48C    P0             12W /  N/A  | Not Supported          |      0%      Default |
```

Read `CUDA Version` correctly. It is the newest CUDA the driver can serve, not an installed toolkit. A machine with no toolkit still shows it.

Our memory column reads `Not Supported` because the GB10 shares one memory pool between CPU and GPU. A discrete card shows used and total MiB there.

## Step 6: a first model with Ollama

```bash
curl -fsSL https://ollama.com/install.sh | sh
ollama -v
ollama run llama3.1:8b "Summarise in one sentence: the driver is in, the model runs."
ollama ps
```

`ollama ps` is the proof. This is ours, on Ubuntu, with Ollama 0.30.10:

```text
NAME           ID              SIZE      PROCESSOR    CONTEXT    UNTIL
llama3.1:8b    4fc21a39517b    9.2 GB    100% GPU     32768      48 minutes from now
```

You want `100% GPU`. A CPU share in that column means the model plus its context did not fit in the card. Our 9.2 GB is this model at a 32k context; the [VRAM guide](https://vorluxai.com/en/blog/how-much-vram-local-llm-en/) shows the sum.

Logs are at `journalctl -e -u ollama`. If you have never used Ollama, [our install guide](https://vorluxai.com/en/blog/install-ollama-first-local-llm-en/) covers the basics.

## Step 7: the CUDA toolkit, only if you compile

You do not need it for Ollama, which ships its own CUDA libraries: our install has `cuda_v12` and `cuda_v13` folders under `/usr/local/lib/ollama`. You do not need it for PyTorch from pip either: on 16 November the default [`torch` 2.15.0 on PyPI](https://pypi.org/project/torch/) declared pip-packaged CUDA 13.2 runtime libraries as dependencies on Linux. You need the toolkit to build llama.cpp or any other CUDA code yourself.

RPM Fusion's CUDA how-to keeps its driver and takes only the toolkit from NVIDIA. Its section for Fedora 44 and later:

```bash
sudo dnf config-manager addrepo --from-repofile=https://developer.download.nvidia.com/compute/cuda/repos/fedora44/$(uname -m)/cuda-fedora44.repo
sudo dnf clean all
sudo dnf config-manager setopt cuda-fedora44-$(uname -m).exclude=nvidia-driver\*,cuda-drivers\*,nvidia-modprobe,nvidia-persistenced,nvidia-settings,nvidia-libXNVCtrl\*,nvidia-xconfig,nvidia-open,nvidia-imex,nvidia-kmod-common,libnvidia-fbc,kmod-nvidia-\*-dkms,nvidia-fs\*,nvidia-gds\*,xorg-x11-nvidia\*
sudo dnf -y install cuda-toolkit xorg-x11-drv-nvidia-cuda
export PATH=${PATH}:/usr/local/cuda-13.5/bin
nvcc --version
```

The `exclude` line is the important one. It stops NVIDIA's driver packages from replacing RPM Fusion's. The `PATH` line is from NVIDIA's guide for 13.5; adjust the number to what you installed.

On our Ubuntu machine `nvcc --version` ends with `Cuda compilation tools, release 13.0, V13.0.88`. On 16 November the `fedora44` repository offered toolkit 13.5.0.

Then llama.cpp, from its [build guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md):

```bash
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release
```

To choose a model file for it, see [which GGUF quantisation to pick](https://vorluxai.com/en/blog/gguf-quantization-pick-the-right-file-for-your-hardware-en/).

## Step 8: PyTorch with the matching wheel

Pick the build in [PyTorch's install selector](https://pytorch.org/get-started/locally/). On 16 November the wheel index carried `cu126`, `cu130`, `cu132` and `cu134` builds, and 2.15.0 existed for the last two. Inside a virtual environment:

```bash
python3 -m venv ~/venvs/torch && source ~/venvs/torch/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/cu132
python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available())"
```

On our Ubuntu machine that last line prints `2.12.1+cu130 13.0 True`. You want `True` and a CUDA version your driver supports. We could not read the selector's own text with our tools, so the command above follows the index, not the selector.

## Step 9: GPU containers with Podman or Docker

From NVIDIA's Container Toolkit guide, with the driver already working:

```bash
curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo dnf install -y nvidia-container-toolkit
nvidia-ctk cdi list
podman run --rm --security-opt=label=disable --device=nvidia.com/gpu=all ubuntu nvidia-smi
```

For Podman, NVIDIA recommends CDI. Since toolkit 1.18.0 a service called `nvidia-cdi-refresh` writes the device file to `/var/run/cdi/nvidia.yaml` on install, on driver upgrade and at boot. The `--security-opt=label=disable` flag turns off SELinux separation for that container; NVIDIA's own example uses it.

Docker needs one more step:

```bash
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
```

The guide pins a version (1.20.1-1 on the day we read it). We show the unpinned install for brevity.

## What we measured, on Ubuntu

None of this is a Fedora measurement. The driver and the model are the same software on both systems; the card is not.

| Hardware | System | Model | Speed | Source |
|---|---|---|---|---|
| NVIDIA DGX Spark (GB10, 128 GB unified) | Ubuntu 24.04, driver 580.173.02, Ollama 0.30.10 | llama3.1:8b, Q4_K_M, 32k context | 40.2 tok/s, 0.365 s to first token, 9.21 GB loaded | ours, 17 Oct 2026, [result file](https://vorluxai.com/data/measurements/local-model-benchmark-2026-10-17.json) |
| RTX 3060 12 GB, Core i7-12700F | Debian 13, llama.cpp built from source | Llama 3.1 8B Instruct, Q4_K_M, context not stated | 64.5 tok/s | [tyolab](https://www.tyolab.com/blog/2026/05/11-64gb-ram-12gb-vram-the-honest-local-llm-benchmark/), May 2026, their own run |

The second row is not ours and is not Fedora. It is the nearest first-hand figure we found for the card most readers will buy. Measure your own with our [30-minute benchmark](https://vorluxai.com/en/blog/benchmark-local-model-own-machine-30-minutes-en/).

## What our CUDA 13 machines taught us

Three things from our operations notes that are not specific to our hardware.

- **A wheel built for another CUDA major installs without complaint and fails later.** pip does not check the CUDA ABI. Our notes record the failure mode for a CUDA 12 wheel on a CUDA 13 host: an undefined symbol at import, or a crash at the first GPU call. Our rule: one virtual environment per tool, torch from the matching index, and the one-line check from step 8 after every install.
- **When `torch.cuda.is_available()` is `False`, check in this order.** `nvidia-smi` inside the container; an empty or stale `CUDA_VISIBLE_DEVICES`; permissions on `/dev/nvidia*` in rootless containers; a stuck context from a crashed process; and only then the wheel. Reinstalling the wheel fixes only the last one.
- **One card, one memory pool, many tenants.** Our fine-tuning runs were killed while another model server sat resident on the same memory. On a 12 GB card the same thing happens sooner: Ollama, a notebook and a container all draw from one pool. `nvidia-smi --query-compute-apps=pid,used_memory --format=csv` shows who holds what.

## Failures the sources document

| What you see | Cause, per the source | Documented fix |
|---|---|---|
| `modinfo: ERROR: Module nvidia not found.` | the module is still building (RPM Fusion) | wait up to 5 minutes, repeat, then reboot |
| Driver does not load with Secure Boot on | the kernel accepts only modules signed by a key it trusts ([NVIDIA README](https://download.nvidia.com/XFree86/Linux-x86_64/595.58.03/README/commonproblems.html)) | step 3; re-import the key after a firmware update |
| `lsmod \| grep nouveau` prints a line | nouveau holds the card. The README's installer message: `Nouveau, or another driver, is already using the GPU.` | reboot once the module is built; on Atomic, add RPM Fusion's `rpm-ostree kargs` blacklist line |
| No driver after a kernel update | the module is built per kernel; the README has an entry for exactly this | akmods checks for missing modules at startup; test with `modinfo -F version nvidia -k <kernel>`, force with `sudo akmods --kernels $(uname -r) --rebuild` |
| `nvcc` rejects your compiler | CUDA 13.5 supports GCC 6.x to 16.x and checks the major version only (NVIDIA) | compare `gcc --version`; on Fedora 44 NVIDIA documents `dnf install gcc15-c++` and `export NVCC_CCBIN='g++-15'` |
| Ollama answers on CPU; the log shows code 3, 46, 100 or 999 | GPU initialisation failed (Ollama) | `sudo nvidia-modprobe -u`, reload `nvidia_uvm`, reboot, newer driver, in that order |
| Ollama loses the GPU after suspend | a driver bug, says Ollama | `sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm` |
| `Failed to initialize NVML: Insufficient Permissions` in a container | SELinux relabelling ([NVIDIA](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/troubleshooting.html)) | add `--security-opt=label=disable` |
| `Failed to initialize NVML: Unknown Error` in a running container | the container lost its device access | recreate the container; CDI avoids it |
| `nvcc warning : Cannot find valid GPU for '-arch=native', default arch is used` | llama.cpp's build could not detect the card | add `-DGGML_NATIVE=OFF` or set `-DCMAKE_CUDA_ARCHITECTURES` |

We did not find the exact `nvcc` compiler error at a source we could open, so the table does not quote one.

## What fits on the card

We will not repeat the sum here. The [VRAM guide](https://vorluxai.com/en/blog/how-much-vram-local-llm-en/) derives it from weights, context and parallel slots, and the [VRAM explorer](https://vorluxai.com/en/tools/vram-explorer/) does it for your card. The short version from our catalogue: at 8k of context a 12 GB card holds an 8B model and a 16 GB card a 14B.

## Where this fits, and when Fedora is the wrong choice

A Fedora desktop with a 12 GB card is a sound base for one person or a small team running 8B models, with the documents staying on the machine. Add [Open WebUI](https://vorluxai.com/en/blog/open-webui-office-accounts-models-logs-en/) when more than one person needs it.

Fedora is the wrong choice in these cases:

- **You need a configuration NVIDIA supports.** Its guides name one Fedora release. On 16 November that was Fedora 44, while Fedora 45 was two weeks old and its NVIDIA repository held only monitoring packages. Fedora 43 reaches end of life on 1 December 2026, per [endoflife.date](https://endoflife.date/fedora). NVIDIA's same table lists three Ubuntu LTS releases and RHEL 8, 9 and 10.
- **The machine must not surprise you.** Fedora ships new kernels often and each one needs a module rebuild. RPM Fusion documents a version lock, and warns that it also blocks kernel-compatibility fixes.
- **You upgrade on release day.** Wait until the driver for the new kernel exists, and the toolkit repository if you compile.
- **You have no NVIDIA card.** On Intel Arc, see [llama.cpp with SYCL](https://vorluxai.com/en/blog/llama-cpp-intel-arc-sycl-local-ai-on-a-budget-gpu-en/).

## Limits

- We ran none of the Fedora commands. A typo in a source would be a typo here.
- We could not read Fedora's own documentation or PyTorch's selector with our tools. Both are linked for you to check.
- We did not confirm that RPM Fusion had built `akmod-nvidia` for Fedora 45's kernel on the day. Check with `dnf info akmod-nvidia` before you upgrade.
- llama.cpp's own Fedora guide builds in a Fedora 41 toolbox, a release past end of life. We left it out.
- Optimus laptops, multiple GPUs and Wayland-specific problems are not covered.

## FAQ

**Do I need the CUDA toolkit to run Ollama?** No. The driver is enough. Install the toolkit only to compile CUDA code.

**Can I keep Secure Boot on?** Yes. RPM Fusion's how-to says disabling it is not required. Enrol the key first (step 3).

**Does NVIDIA support Fedora 45?** Not on 16 November 2026. Its driver and CUDA guides name Fedora 44. RPM Fusion's CUDA section covers Fedora 44 and later with the `fedora44` repository.

**Can I install NVIDIA's driver over RPM Fusion's to get a newer one?** Not as a normal install. RPM Fusion says the two packagings conflict, which is why its toolkit commands exclude NVIDIA's driver packages.

**Does it work under Wayland?** RPM Fusion says the driver works under Wayland and Xwayland from Fedora 35, with known limitations.

**What about Silverblue or Kinoite?** RPM Fusion documents `sudo rpm-ostree install akmod-nvidia xorg-x11-drv-nvidia-cuda` plus a kernel-argument line that blacklists nouveau. Secure Boot there needs a separate key package, which its page links.

**Which CUDA version does my PyTorch need?** The one in the wheel's name. The wheel carries its runtime, so the installed toolkit does not have to match. The driver has to be new enough.

**My GTX 1060 is not in the default driver. Is it finished?** No. RPM Fusion keeps `akmod-nvidia-580xx` for the GTX 10 series, and Ollama accepts driver 570 or newer for those cards.

## Next steps

- Size the model before you download it: [how much VRAM a local LLM needs](https://vorluxai.com/en/blog/how-much-vram-local-llm-en/).
- Measure your card: [benchmark a local model in 30 minutes](https://vorluxai.com/en/blog/benchmark-local-model-own-machine-30-minutes-en/).
- Plan the rest of the stack: [how to deploy AI locally in your business](https://vorluxai.com/en/blog/how-to-deploy-ai-locally-business-guide/).

## Work with us

We set up local inference on hardware clients already own before recommending anything new, and we leave a written runbook behind. If you want that on your machines, [get in touch](https://vorluxai.com/en/contact/) or read [how our consulting works](https://vorluxai.com/en/consulting/).
