# Claude Code on a Local Ollama Model: Setup, Limits, Tests

> Point Claude Code at a local Ollama model: the documented variables, what Anthropic does not support, the context trap, and the tool-call tests we ran.

- Published: 2026-09-09
- Updated: 2026-11-15
- Author: Jacobo González Jaspe
- Language: en
- Tags: Claude Code, Ollama, Local AI, Developer Tools
- Canonical URL: https://vorluxai.com/en/blog/claude-code-local-models-ollama-web-search-subagents-en/
- Source: VORLUX AI (J4SGON S.L.), https://vorluxai.com/

---

By the end of this guide, Claude Code will talk to a model served by Ollama on your own machine. You will know which variables do the work, what still leaves your network, and where a local model falls short.

Every variable, flag and version below was checked against the Claude Code and Ollama documentation on 15 November 2026. The numbers are ours, with the result files linked.

> **TL;DR**
> - **No proxy needed.** Ollama serves the Anthropic Messages format at `/v1/messages`. Two variables connect Claude Code to it; a handful more make it behave.
> - **It is a workaround.** Ollama documents the setup. Anthropic does not support running Claude Code on non-Claude models.
> - **The trap is context.** In our test, a prompt longer than the loaded context came back as HTTP 200 with the start silently dropped.
> - **Model size matters for tools.** Our 8B model called a shell tool to answer plain questions. Our 26B model did not.
> - **Not everything is local.** Web search runs on Ollama's hosted service, and Claude Code sends usage metrics to Anthropic unless you turn them off.

## Who this is for, and when not to do it

This is for you if you have code that cannot go to a hosted API, you work offline, or you want to learn how a coding agent drives a model.

It is the wrong choice in three cases:

- **You need the product as Anthropic supports it.** Their gateway documentation says Anthropic "doesn't support routing Claude Code to non-Claude models through any gateway" ([LLM gateways](https://code.claude.com/docs/en/llm-gateway), read 15 November 2026). If something breaks, the fix is yours to find.
- **You have less than 16 GB of memory for the model.** A coding agent needs a long context, and context costs memory. See Step 2.
- **The task is a long, multi-file change.** A local 26B model needs more turns and more supervision than a hosted frontier model. We keep a hosted model for that work.

## What you need

- **Ollama v0.14.0 or later.** That release added the Anthropic-compatible endpoint, according to [Ollama's announcement](https://ollama.com/blog/claude) of 16 January 2026. Check with `ollama --version`. The latest stable release on 15 November 2026 was [v0.40.16](https://github.com/ollama/ollama/releases). Our tests ran on 0.34.0, and we have not tested 0.40.x.
- **Claude Code.** Install with `curl -fsSL https://claude.ai/install.sh | bash` on macOS or Linux, or `irm https://claude.ai/install.ps1 | iex` in Windows PowerShell. Both commands appear in [Ollama's integration page](https://docs.ollama.com/integrations/claude-code).
- **A model with the `tools` capability.** `ollama show <model>` lists capabilities. New to Ollama? Start with [installing Ollama and your first model](https://vorluxai.com/en/blog/install-ollama-first-local-llm-en/).
- **Memory for the model plus its context.** The table in Step 2 has our loaded sizes.

## How the connection works today

Claude Code speaks the Anthropic Messages format. Its [gateway compatibility guide](https://code.claude.com/docs/en/llm-gateway-protocol) lists the endpoints it needs: `/v1/messages`, plus an optional `/v1/messages/count_tokens`. Any server that answers those can sit behind `ANTHROPIC_BASE_URL`.

Ollama answers `/v1/messages` natively. Its [Anthropic compatibility page](https://docs.ollama.com/api/anthropic-compatibility) lists streaming, system prompts, tool calls, tool results, base64 images and thinking blocks as supported. It lists `count_tokens`, `tool_choice`, prompt caching, batches, citations and PDF input as unsupported.

So a translating proxy such as LiteLLM is no longer required for this. A proxy still earns its place when you want one address in front of several servers, per-user keys or usage logs. LiteLLM documents that setup in its [Claude Code tutorial](https://docs.litellm.ai/docs/tutorials/claude_non_anthropic_models). We run LiteLLM in front of our own machines for those reasons, not for format translation.

Ollama's OpenAI-compatible endpoint is no use here. The same guide lists the formats Claude Code speaks, and the OpenAI one is not among them.

## Step 1: check the endpoint before you open Claude Code

Send one tool-calling request by hand. If this fails, Claude Code will fail too, with a less clear message.

```bash
curl -s http://localhost:11434/v1/messages \
  -H "content-type: application/json" -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "gemma4:26b", "max_tokens": 400,
    "tools": [{"name": "Read", "description": "Read a file from the repository.",
      "input_schema": {"type": "object",
        "properties": {"file_path": {"type": "string"}}, "required": ["file_path"]}}],
    "messages": [{"role": "user", "content": "Show me what is in the file src/parser.py."}]
  }'
```

What you should see is a `tool_use` block and `"stop_reason": "tool_use"`. Our run on 15 November 2026 returned this, trimmed to the fields that matter:

```json
{"type": "tool_use", "name": "Read", "input": {"file_path": "src/parser.py"}}
```

A text answer that describes the file, or a JSON object printed as text, means the model is not calling tools properly. Pick another model.

## Step 2: give the model a real context window

This is the step people skip, and it causes the worst failure in this guide.

Ollama's [context length page](https://docs.ollama.com/context-length) (read 15 November 2026) says the default depends on your VRAM: 4k tokens under 24 GiB, 32k from 24 to 48 GiB, and 256k from 48 GiB. The same page recommends at least 64,000 tokens for agents and coding tools. A 4k window cannot hold a coding agent's system prompt and tool definitions.

The Anthropic endpoint has no `num_ctx` field. The window comes from the server or from the model:

```bash
# Option A: for the whole server
OLLAMA_CONTEXT_LENGTH=65536 ollama serve

# Option B: bake it into a named copy of the model (what we do; see Ollama's Modelfile reference)
printf 'FROM gemma4:26b\nPARAMETER num_ctx 32768\n' > Modelfile
ollama create gemma4-32k -f Modelfile

ollama ps    # the CONTEXT column shows what is really loaded
```

We checked that the baked value holds. We loaded a model with 6,144 tokens, sent it twelve requests through `/v1/messages`, and `ollama ps` still reported 6,144.

**Context costs memory, and parallel slots multiply it.** These are loaded sizes from `ollama ps` on our NVIDIA DGX Spark machines (GB10, 128 GB unified memory), measured on 2 November 2026 ([result file](https://vorluxai.com/data/measurements/ollama-num-ctx-2026-11-02.json)):

| Model (Q4_K_M) | File | Context | Slots | Loaded |
|---|---|---|---|---|
| `llama3.1:8b` | 4.9 GB | 32,768 | 1 | 9.21 GB |
| `llama3.1:8b` | 4.9 GB | 32,768 | 4 | 22.38 GB |
| `llama3.1:8b` | 4.9 GB | 32,768 | 8 | 39.27 GB |
| `gemma4:26b` | 18.0 GB | 32,768 | 1 | 17.47 GB |
| `gemma4:26b` | 18.0 GB | 131,072 | 1 | 18.36 GB |
| `qwen3.6:35b` | 23.9 GB | 32,768 | 8 | 23.47 GB |

The cost of context varies a lot by model. The 8B gains 13 GB going from one slot to four. `gemma4:26b` gains under 1 GB going from 32k to 131k. Ollama starts its engine with a total context of `num_ctx` times the number of slots, which we read from its load log ([receipt](https://vorluxai.com/data/measurements/llama-load-log-buffers-2026-11-15.json)). On a one-person machine, leave `OLLAMA_NUM_PARALLEL` at 1 unless you use subagents (Step 4).

Our machines have 128 GB. We have not measured a 16 GB laptop or a 24 GB card. To work out your own case, use [how much VRAM a local LLM needs](https://vorluxai.com/en/blog/how-much-vram-local-llm-en/) and [picking the GGUF quantization](https://vorluxai.com/en/blog/gguf-quantization-pick-the-right-file-for-your-hardware-en/).

## Step 3: point Claude Code at Ollama

Two routes. Both are documented by Ollama, not by Anthropic.

**The launcher.** `ollama launch claude --model gemma4:26b` sets the variables for that one process and starts Claude Code. The [CLI reference](https://docs.ollama.com/cli) also documents `--config`, which configures without launching.

**By hand.** We prefer this because you can see every setting:

```bash
export LOCAL_MODEL=gemma4-32k     # the copy from Step 2

# From Ollama's integration page
export ANTHROPIC_BASE_URL=http://localhost:11434
export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""

# Send every model alias and every subagent to the local model
export ANTHROPIC_DEFAULT_OPUS_MODEL="$LOCAL_MODEL"
export ANTHROPIC_DEFAULT_SONNET_MODEL="$LOCAL_MODEL"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="$LOCAL_MODEL"
export CLAUDE_CODE_SUBAGENT_MODEL="$LOCAL_MODEL"

# Tell Claude Code the real window, and stop background traffic
export CLAUDE_CODE_MAX_CONTEXT_TOKENS=32768
export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1

claude --model "$LOCAL_MODEL"
```

What each group does, and where it is documented:

- **The first three** come from [Ollama's integration page](https://docs.ollama.com/integrations/claude-code). Ollama notes that the local server ignores the token's value. Claude Code's [environment reference](https://code.claude.com/docs/en/env-vars) says `ANTHROPIC_AUTH_TOKEN` is sent as a bearer header and takes precedence over a saved login.
- **The four model variables** are the ones Ollama's launcher sets, which we read in its [source code](https://github.com/ollama/ollama/blob/main/cmd/launch/claude.go) on 15 November 2026. Each is documented in Claude Code's [model configuration](https://code.claude.com/docs/en/model-config). Without them, a subagent that asks for `haiku` requests a Claude model name that Ollama does not have.
- **`CLAUDE_CODE_MAX_CONTEXT_TOKENS`** matters most. The model configuration page says Claude Code assumes a 1M-token window for a model ID it does not recognise, and compacts the conversation to fit that assumption. Set it to the window you gave Ollama. The launcher's source sets it only for Ollama's cloud models, so with a local model you set it yourself.
- **`CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC`** turns off auto-updates, telemetry and error reporting. It is optional. Update Claude Code by hand if you set it.

Then check inside Claude Code with `/status`. The `Anthropic base URL` line should show your Ollama address.

**We did not run this Claude Code session for this update.** The machine we write from runs Claude Code in production, and we do not change its configuration for a test. Step 3 is assembled from the two vendors' documentation. What we ran ourselves is Steps 1 and 2 and the tests below.

## What we ran: tool calls through the endpoint

On 15 November 2026 we sent Ollama 0.34.0 six requests per model through `/v1/messages`. Each carried a short system prompt and three tools shaped like a coding agent's (`Read`, `Bash`, `Grep`). Three prompts needed a tool, two needed a plain answer, and one returned a tool result. Temperature 0, one run per case, checked by script. Full requests and answers: [result file](https://vorluxai.com/data/measurements/ollama-anthropic-messages-probe-2026-11-15.json).

| Case | `llama3.1:8b` | `gemma4:26b` (32k) |
|---|---|---|
| Read a file | `Read` call, correct path | `Read` call, correct path |
| Run the tests with pytest | `Bash`: `pytest && pytest --version` | `Bash`: `ls -R \| grep test` |
| Find uses of a function | `Grep` call, correct pattern | `Grep` call, correct pattern |
| "Reply with exactly three words" | `Bash`: `echo 'local model ready'` | Plain text, correct |
| "What does JSON stand for?" | `Bash`: `echo '...'` | Plain text, correct |
| Answer from a tool result | Correct | Correct |

Every tool call from both models was well formed: a known tool name and valid arguments. The difference is judgement. The 8B model reached for the shell to answer questions that needed no tool, in both cases. In a real session, each of those is a permission prompt and a wasted turn. The 26B model answered in text. It also chose to look for the tests before running them, which our script marked as a miss and a person would call reasonable.

This is a small probe, not a Claude Code session. Claude Code sends a far longer system prompt and many more tools, which is harder for a small model, not easier.

Two more results from the same run:

- **`tool_choice` had no visible effect.** We sent one prompt four ways: no `tool_choice`, forced `Bash`, forced `Read` and `none`. All four answers were identical. That matches Ollama's page, which lists it as unsupported.
- **Structured output points the same way.** In our separate test of 40 Spanish invoices on 5 November 2026, `gemma4:26b` got every field right in all 40. `qwen2.5-coder:7b` managed 85% and `llama3.1:8b` 62.5% ([result file](https://vorluxai.com/data/measurements/structured-extraction-2026-11-05.json), and [the method](https://vorluxai.com/en/blog/ollama-structured-output-invoices-measured-en/)).

## Which local model, and how fast

All four models below list the `tools` capability in Ollama. Speeds use one prompt, 200 output tokens, on our DGX Spark machines. They are single-request speeds, not Claude Code sessions.

| Model (Q4_K_M) | File | Generation | Measured |
|---|---|---|---|
| `qwen2.5-coder:7b` | 4.68 GB | 42.1 tok/s | 17 Oct 2026, Ollama 0.30.10 |
| `llama3.1:8b` | 4.92 GB | 40.2 tok/s | 17 Oct 2026, Ollama 0.30.10 |
| `gemma4:26b` | 17.99 GB | 62.0 tok/s | 17 Oct 2026, Ollama 0.30.10 |
| `gemma4:26b` | 17.99 GB | 68.2 tok/s | 2 Nov 2026, Ollama 0.34.0 |
| `qwen3.6:35b` | 23.94 GB | 75.8 tok/s | 2 Nov 2026, Ollama 0.34.0 |

Sources: [October file](https://vorluxai.com/data/measurements/local-model-benchmark-2026-10-17.json) and [November file](https://vorluxai.com/data/measurements/dgx-spark-review-spark2-2026-11-02.json).

Our reading:

- **7B to 8B:** fine for learning the setup and for single-file questions. Expect stray tool calls.
- **Around 26B:** the smallest size we would use for agent work. It is what we run daily.
- **`qwen3-coder:30b`:** Ollama's own examples use it. Its [library page](https://ollama.com/library/qwen3-coder/tags) lists 19 GB, a 256K window and tools. We have not measured it.

## Step 4: subagents

A subagent is a side task with its own context window. It returns a summary, so the main conversation stays short. Claude Code's [subagent page](https://code.claude.com/docs/en/sub-agents) says subagents work on every provider.

Three things change with a local model:

1. **Pin the subagent model.** A subagent can ask for an alias such as `haiku`. The four model variables in Step 3 send those requests to your local model.
2. **Ask for them.** Ollama's [post on subagents](https://ollama.com/blog/web-search-subagents-claude-code) (16 February 2026) names three of its cloud models that start subagents on their own. It makes no such claim for local models. Say it in the prompt: "use subagents to check the tests and the docs".
3. **Parallel needs slots.** With `OLLAMA_NUM_PARALLEL=1`, parallel subagents wait in a queue. More slots cost memory, as the table in Step 2 shows. On one GPU they also share its speed. Use subagents to keep context clean, not to go faster.

We have not measured a subagent run on a local model.

## What stays on your machine, and what does not

| Traffic | Where it goes | How to stop it |
|---|---|---|
| Prompts, code, tool calls, subagents | Your Ollama server | Nothing to stop |
| Web search | Ollama's hosted search API | Deny the `WebSearch` tool |
| WebFetch page downloads | The site you fetch | Deny the `WebFetch` tool |
| WebFetch hostname check | `api.anthropic.com`, hostname only | `skipWebFetchPreflight: true` in settings |
| Usage metrics, update checks | Anthropic and its providers | `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` |

Rows three to five come from Claude Code's [data usage page](https://code.claude.com/docs/en/data-usage). It lists metrics as on by default for the Claude API route, which is how Claude Code treats a custom base URL. It also says metrics never include code, prompts or file paths.

**Web search is not local.** Claude Code's `WebSearch` tool normally runs on Anthropic's servers. Ollama's post says its compatibility layer runs the search instead and hands the results to the model. Ollama's [web search page](https://docs.ollama.com/capabilities/web-search) documents that API at `ollama.com`, and says it needs a free Ollama account. The same compatibility page says hosted web search is "not fully supported". We have not tested it with a local model.

If your rule is that nothing leaves, skip it. Claude Code's tools reference says the search backend cannot be changed, and points to an MCP server as the way to add another one. We self-host [SearXNG](https://github.com/searxng/searxng) for that. See [connecting agents to tools with MCP](https://vorluxai.com/en/blog/n8n-mcp-connect-ai-agents-business-tools-2026-en/).

## Failures you will hit

**1. The model forgets the start of the conversation.** We sent a 64,507-character prompt to a model loaded with 6,144 tokens of context. A code word sat on the first line and the question on the last. Ollama returned HTTP 200, reported 3,070 input tokens, and the model answered that it saw no code word. No error, no warning. The same question with a short prompt was answered correctly. Fix: Step 2, then `CLAUDE_CODE_MAX_CONTEXT_TOKENS` in Step 3. If it still drifts, run `/compact` earlier.

**2. `model 'claude-haiku-4-5' not found`.** We reproduced this: Ollama answers HTTP 404 with a `not_found_error` when asked for a Claude model name. It means some part of Claude Code is still asking for a Claude model. Fix: the four model variables in Step 3. Ollama's compatibility page offers another route, copying your model under a Claude name with `ollama cp`.

**3. Your variables are ignored.** Claude Code's [connection page](https://code.claude.com/docs/en/llm-gateway-connect) says a value in a settings file's `env` block wins over a shell export. Run `/status` and read the base URL line.

**4. Token counts look approximate.** On our Ollama 0.34.0, `/v1/messages/count_tokens` answered `404 page not found`. Claude Code's gateway guide says it then falls back to a character-based estimate, so `/context` shows approximate figures. Nothing to fix.

**5. The first answer takes a long time.** The model is loading from disk. In our context test, `gemma4:26b` took 32 seconds on its first load and 9 to 10 seconds on the next two. Claude Code waits up to 10 minutes for a response to start by default (`API_TIMEOUT_MS`).

**6. `400` errors that name an unknown field.** Claude Code's troubleshooting table gives `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1` for servers that reject pre-release fields. We did not see this error.

## Limits against the hosted models

- **Context.** Claude Code's model page gives the hosted Claude models windows of 200K to 1M tokens. Our daily local model runs at 32k. Long sessions compact sooner and lose detail.
- **No prompt caching.** Ollama lists `cache_control` as unsupported. In our probe Ollama did report reused prompt tokens on repeated prefixes, but that is its own mechanism.
- **Judgement.** The 26B model formed tool calls correctly in our probe. We have no measurement of it on long, multi-file work, and we do not claim it matches a frontier model there.
- **Features tied to an Anthropic login.** Remote Control, voice dictation and claude.ai connectors are unavailable with a custom base URL or token, according to the Claude Code documentation.
- **Support.** None from Anthropic for this setup.

**What we did not test:** a full Claude Code session on today's versions, Ollama 0.40.x, web search with a local model, subagents on a local model, a 16 GB or 24 GB machine, `qwen3-coder`, and whether Claude Code accepts a declared window as small as 32k (its documentation gives no minimum). An earlier version of this article quoted session timings from September 2026. We had no saved result file for them, so we removed them.

## Frequent questions

**Do I still need LiteLLM or another proxy?** No. Ollama has served the Anthropic format since v0.14.0. Use a proxy only if you want routing, keys or logs.

**Is this supported by Anthropic?** No. Their documentation says so directly. Ollama documents and maintains its side.

**Do I need an Anthropic account or key?** Ollama's setup uses a placeholder token, and Claude Code's documentation says that token takes the place of a login. We have not tried it on a machine that never signed in.

**Does web search work offline?** No. It calls Ollama's hosted API.

**Can I keep my normal Claude Code login?** Yes. Shell exports last for that terminal only. Close it and Claude Code is back to normal.

**Which model first?** The largest tools-capable model that fits your memory with a 32k window. On our machines that is `gemma4:26b`.

## Next steps

- Size the memory first: [how much VRAM a local LLM needs](https://vorluxai.com/en/blog/how-much-vram-local-llm-en/).
- See how context changes memory, model by model: [Ollama num_ctx, measured](https://vorluxai.com/en/blog/ollama-context-length-memory-measured-en/).
- Choose a model for your machine: [best local LLMs by hardware and task](https://vorluxai.com/en/blog/best-local-llms-by-hardware-and-task-en/).
- Decide what may leave the building: [hybrid routing](https://vorluxai.com/en/blog/hybrid-ai-routing-pii-stays-local-en/).

## Work with us

We run coding agents on local models every day and keep a hosted model for the hard cases. If you want that split designed for your team, sized to the machines you already own, [get in touch](https://vorluxai.com/en/contact/) or see [how our consulting works](https://vorluxai.com/en/consulting/).
