# Ollama num_ctx Memory Usage: 4k to 128k, Measured

> We loaded three models at 4k to 128k context on two DGX Sparks. The 8B went from 5.3 to 22.8 GB, and parallel slots multiply it. How to set num_ctx.

- Published: 2026-11-02
- Author: Jacobo González Jaspe
- Language: en
- Tags: Ollama, Memory, Benchmarks, Self-hosted, Local AI
- Canonical URL: https://vorluxai.com/en/blog/ollama-context-length-memory-measured-en/
- Source: VORLUX AI (J4SGON S.L.), https://vorluxai.com/

---

By the end of this post you will know what each context size costs in memory, why the same model can take 5 GB on one server and 39 GB on another, and where to set `num_ctx` so a load does not fail. It works on any machine that runs Ollama, a laptop included. We learned it on a box with 128 GB, which turned out not to be enough.

> **TL;DR**
> - You get measured tables: loaded size against `num_ctx` for three models, and what parallel slots do to it.
> - It costs nothing: one line in a Modelfile, or one environment variable.
> - Verdict: give every model the context its real task needs. A Llama 3.1 8B loaded at 5.3 GB with 4k and 22.8 GB with 128k; eight slots at 32k took 39.3 GB.

## The day our models stopped loading

On 2026-10-11 every model load on our second DGX Spark hung or was killed for lack of memory, unless the model carried its own context size. The box has 128 GB of memory.

The log said why. Ollama 0.34 picks a default context from the memory it sees, and on that box it chose 262,144 tokens (`default_num_ctx=262144`). The server also runs 8 parallel slots, and each slot reserves its own context. `llama3.1:8b`, a 4.9 GB file, had reached 36 GB of resident memory when we stopped it.

Two models kept working. One had 32k written into its Modelfile; the other was our news-desk judge, which sends `num_ctx` with every request. That gave us the fix before we opened any documentation. The same day we wrote a `num_ctx` into every model on that box (16384 for almost all of them), and later set `OLLAMA_CONTEXT_LENGTH=32768` in the server's unit.

Then we measured what each choice costs, so you can skip the bad afternoon.

## What you need

- Ollama on any machine. Our [install guide](https://vorluxai.com/en/blog/install-ollama-first-local-llm-en/) gets you there on a laptop.
- A model you already use.
- `curl`, and `sudo` only for step 4.
- Optional: the same table for your machine. Our script `scripts/bench/ollama_num_ctx.py` is in our internal repository and not public yet; loading each size and reading `ollama ps` (step 1) gives you the same numbers by hand.

## Step 1: see what context you are paying for

```bash
ollama ps
# NAME           SIZE      PROCESSOR    CONTEXT
# llama3.1:8b    9.2 GB    100% GPU     32768

ollama show llama3.1:8b --parameters
# num_ctx    32768
```

`SIZE` is what the loaded model holds in memory. `CONTEXT` is the context of one slot, while `SIZE` pays for all of them. If `show` lists no `num_ctx`, the model takes the server default, and that default depends on your version and your memory. On a Linux install, look at the server's settings and at what it decided:

```bash
systemctl show ollama --property=Environment
journalctl -u ollama | grep -i num_ctx | tail -3
```

## Step 2: set it per request

Any call to Ollama's own API can carry the size:

```bash
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.1:8b",
  "messages": [{"role": "user", "content": "Summarise this email: ..."}],
  "options": {"num_ctx": 8192}
}'
```

Keep the value the same across a client. A request with a different size makes Ollama load the model again.

The OpenAI-style endpoint (`/v1/chat/completions`) has no field for it. [Ollama's compatibility page](https://github.com/ollama/ollama/blob/main/docs/api/openai-compatibility.mdx) says to use a Modelfile instead, which is step 3.

## Step 3: write it into the model

This is the fix that worked for us. A Modelfile of two lines gives the model a context of its own:

```bash
cat > Modelfile <<'EOF'
FROM qwen3.6:35b
PARAMETER num_ctx 16384
EOF
ollama create qwen3.6-16k -f Modelfile
ollama show qwen3.6-16k --parameters | grep num_ctx
```

Every client that asks for `qwen3.6-16k` now gets 16k, including those on the `/v1` endpoint. On our second box we overwrote the original names instead, so no client had to change:

```bash
curl http://localhost:11434/api/create -d '{"model": "qwen3.6:35b", "from": "qwen3.6:35b", "parameters": {"num_ctx": 16384}}'
```

A model you pull later arrives without that line. Add it the same day.

## Step 4: set a server default

The default catches any model you forgot. On a systemd install:

```bash
sudo systemctl edit ollama
# [Service]
# Environment="OLLAMA_CONTEXT_LENGTH=16384"
# Environment="OLLAMA_NUM_PARALLEL=4"
sudo systemctl restart ollama
```

The order is: the request's `num_ctx` first, then the model's, then the server default.

## What we measured

Measured 2026-11-02 on two NVIDIA DGX Sparks (GB10, 128 GB unified memory each). First box: Ollama 0.30.10, two temporary instances with 1 and 4 parallel slots. Second box: our production Ollama 0.34.0, 8 slots, another model resident throughout. For each row the script unloads the model, loads it with one 8-token request that carries `num_ctx`, reads `size` from `/api/ps`, and unloads it again. All models are Q4_K_M. Raw file: [ollama-num-ctx-2026-11-02.json](https://vorluxai.com/data/measurements/ollama-num-ctx-2026-11-02.json).

```bash
python3 scripts/bench/ollama_num_ctx.py --ollama http://127.0.0.1:11437 --num-parallel 1 \
  --models llama3.1:8b,gemma4:26b --ctx 4096,32768,131072 --append \
  --out docs/measurements/ollama-num-ctx-2026-11-02.json
python3 scripts/bench/ollama_num_ctx.py --ollama http://127.0.0.1:11438 --num-parallel 4 \
  --models llama3.1:8b,gemma4:26b --ctx 4096,32768 --append \
  --out docs/measurements/ollama-num-ctx-2026-11-02.json
python3 scripts/bench/ollama_num_ctx.py --ollama http://<second-box>:11434 --num-parallel 8 \
  --mem-host <second-box> --models llama3.1:8b,qwen3.6:35b --ctx 4096,16384,32768 --append \
  --out docs/measurements/ollama-num-ctx-2026-11-02.json
```

**One slot** (first box, `OLLAMA_NUM_PARALLEL=1`):

| num_ctx | llama3.1:8b (4.9 GB file) | gemma4:26b (18.0 GB file) |
|---|---|---|
| 4,096 | 5.27 GB | 17.20 GB |
| 32,768 | 9.21 GB | 17.47 GB |
| 131,072 | 22.79 GB | 18.36 GB |

**llama3.1:8b across slots** (reserved tokens = slots × num_ctx):

| Box, slots | num_ctx | Reserved tokens | Loaded size |
|---|---|---|---|
| first, 1 | 4,096 | 4,096 | 5.27 GB |
| first, 4 | 4,096 | 16,384 | 7.00 GB |
| second, 8 | 4,096 | 32,768 | 9.15 GB |
| first, 1 | 32,768 | 32,768 | 9.21 GB |
| second, 8 | 16,384 | 131,072 | 22.32 GB |
| first, 4 | 32,768 | 131,072 | 22.38 GB |
| first, 1 | 131,072 | 131,072 | 22.79 GB |
| second, 8 | 32,768 | 262,144 | 39.27 GB |

**The two that barely grow:** gemma4:26b with 4 slots loaded at 18.67 GB (4k) and 19.84 GB (32k). qwen3.6:35b (23.9 GB file) with 8 slots loaded at 22.72 GB (4k), 23.10 GB (16k) and 23.47 GB (32k).

Three things the tables say.

**Slots times context is the number that counts.** 32,768 reserved tokens cost about 9.2 GB as one slot of 32k or as eight slots of 4k. 131,072 cost 22.3 to 22.8 GB in three layouts, on two boxes and two Ollama versions. [Ollama's FAQ](https://github.com/ollama/ollama/blob/main/docs/faq.mdx) says memory scales with `OLLAMA_NUM_PARALLEL` × `OLLAMA_CONTEXT_LENGTH` (read 2026-11-02). Now you have the numbers.

**On a classic model each reserved token costs about 0.14 MB.** From 32,768 to 131,072 tokens, llama3.1:8b grew 13.58 GB: 0.138 MB per token. The formula in [How much VRAM do I need?](https://vorluxai.com/en/blog/how-much-vram-local-llm-en/) gives 0.131 MB for this model, close enough to plan with.

**Not every model pays like that.** gemma4:26b gained 1.16 GB from 4k to 128k; Llama gained 17.52 GB over the same range. We did not measure why. Our reading is that both newer families keep most layers' cache at a fixed size, so only a few layers pay per token. Treat that as a hypothesis and measure your own model.

## What the spec sheet won't tell you

- **The size is reserved at load, not when you use it.** Our loads were 8-token requests and already showed the full size. A long context you never fill still costs memory from the first message.
- **The documentation and your build can disagree.** The FAQ says the default context is 4,096 tokens (read 2026-11-02). Our 0.34.0 chose 262,144 on a 128 GB box. Trust the log line from step 1.
- **A server default is a safety net, not a fit.** With `OLLAMA_CONTEXT_LENGTH=32768` and 8 slots, a Llama 3.1 8B without its own value loads at 39.27 GB. That is why every model on our second box also carries 16k (22.32 GB).
- **We did not re-run the 262,144 case.** We know which loads failed, not which buffer failed first in each one. Breaking a production box once was enough.

## Verdict

- **Write it into every model.** `PARAMETER num_ctx` sized to the longest real task, not to the number on the model card.
- **Then set a server default**, lower than feels comfortable, for the model you will forget.
- **Fewer slots before more memory.** Four slots at 32k cost what one slot at 128k costs. Pick the one your users need. What each slot buys in speed: num_parallel and keep_alive, measured.

## When not to do this

If one person runs one model on a laptop, check `ollama ps` once; your default may already be fine. If your task needs the long context (a full contract, a long transcript), a small `num_ctx` is worse than a slow load, because Ollama cuts the start of the prompt and only says so in its log. And if that long job happens twice a month, a hosted model with a large window may cost less than the memory.

## Next steps

- Turn these sizes into a buying decision: [How much VRAM do I need?](https://vorluxai.com/en/blog/how-much-vram-local-llm-en/).
- Before you set `num_ctx`, count how many tokens your longest task takes with the [tokenizer](https://vorluxai.com/en/tools/tokenizer/).
- When several people ask at once, slots matter more: [Ollama vs vLLM for 1 to 10 users](https://vorluxai.com/en/blog/ollama-vs-vllm-small-office-measured-en/).
- Put the model behind accounts for the office: [Open WebUI for a 10-person office](https://vorluxai.com/en/blog/open-webui-office-accounts-models-logs-en/).
- Get told when a load fails instead of finding out from a colleague: [keep a local AI server running](https://vorluxai.com/en/blog/keep-a-local-ai-server-running-monitoring-with-uptime-kuma-and-prometheus-en/).

## Work with us

We size the model and the machine by measuring, not by guessing. If you want to see your own task running on real hardware, [book a free 15-minute call](https://vorluxai.com/en/contact/) or [see how we work in consulting](https://vorluxai.com/en/consulting/).
