View all articles
Claude CodeOllamaLocal AIDeveloper Tools

Claude Code on a Local Ollama Model: Setup, Limits, Tests

JG
Jacobo González Jaspe
|

Reviewed:

Abstract illustration: ribbons of amber and white light weaving in a gentle curve. Models
Illustration generated with AI on our own machine.
This article is also available in Spanish:Claude Code con un modelo local de Ollama: guía y límites

By the end of this guide, Claude Code will talk to a model served by Ollama on your own machine. You will know which variables do the work, what still leaves your network, and where a local model falls short.

Every variable, flag and version below was checked against the Claude Code and Ollama documentation on 15 November 2026. The numbers are ours, with the result files linked.

Who this is for, and when not to do it

This is for you if you have code that cannot go to a hosted API, you work offline, or you want to learn how a coding agent drives a model.

It is the wrong choice in three cases:

  • You need the product as Anthropic supports it. Their gateway documentation says Anthropic “doesn’t support routing Claude Code to non-Claude models through any gateway” (LLM gateways, read 15 November 2026). If something breaks, the fix is yours to find.
  • You have less than 16 GB of memory for the model. A coding agent needs a long context, and context costs memory. See Step 2.
  • The task is a long, multi-file change. A local 26B model needs more turns and more supervision than a hosted frontier model. We keep a hosted model for that work.

What you need

  • Ollama v0.14.0 or later. That release added the Anthropic-compatible endpoint, according to Ollama’s announcement of 16 January 2026. Check with ollama --version. The latest stable release on 15 November 2026 was v0.40.16. Our tests ran on 0.34.0, and we have not tested 0.40.x.
  • Claude Code. Install with curl -fsSL https://claude.ai/install.sh | bash on macOS or Linux, or irm https://claude.ai/install.ps1 | iex in Windows PowerShell. Both commands appear in Ollama’s integration page.
  • A model with the tools capability. ollama show <model> lists capabilities. New to Ollama? Start with installing Ollama and your first model.
  • Memory for the model plus its context. The table in Step 2 has our loaded sizes.

How the connection works today

Claude Code speaks the Anthropic Messages format. Its gateway compatibility guide lists the endpoints it needs: /v1/messages, plus an optional /v1/messages/count_tokens. Any server that answers those can sit behind ANTHROPIC_BASE_URL.

Ollama answers /v1/messages natively. Its Anthropic compatibility page lists streaming, system prompts, tool calls, tool results, base64 images and thinking blocks as supported. It lists count_tokens, tool_choice, prompt caching, batches, citations and PDF input as unsupported.

So a translating proxy such as LiteLLM is no longer required for this. A proxy still earns its place when you want one address in front of several servers, per-user keys or usage logs. LiteLLM documents that setup in its Claude Code tutorial. We run LiteLLM in front of our own machines for those reasons, not for format translation.

Ollama’s OpenAI-compatible endpoint is no use here. The same guide lists the formats Claude Code speaks, and the OpenAI one is not among them.

Step 1: check the endpoint before you open Claude Code

Send one tool-calling request by hand. If this fails, Claude Code will fail too, with a less clear message.

bash
curl -s http://localhost:11434/v1/messages \
  -H "content-type: application/json" -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "gemma4:26b", "max_tokens": 400,
    "tools": [{"name": "Read", "description": "Read a file from the repository.",
      "input_schema": {"type": "object",
        "properties": {"file_path": {"type": "string"}}, "required": ["file_path"]}}],
    "messages": [{"role": "user", "content": "Show me what is in the file src/parser.py."}]
  }'

What you should see is a tool_use block and "stop_reason": "tool_use". Our run on 15 November 2026 returned this, trimmed to the fields that matter:

JSON
{"type": "tool_use", "name": "Read", "input": {"file_path": "src/parser.py"}}

A text answer that describes the file, or a JSON object printed as text, means the model is not calling tools properly. Pick another model.

Step 2: give the model a real context window

This is the step people skip, and it causes the worst failure in this guide.

Ollama’s context length page (read 15 November 2026) says the default depends on your VRAM: 4k tokens under 24 GiB, 32k from 24 to 48 GiB, and 256k from 48 GiB. The same page recommends at least 64,000 tokens for agents and coding tools. A 4k window cannot hold a coding agent’s system prompt and tool definitions.

The Anthropic endpoint has no num_ctx field. The window comes from the server or from the model:

bash
# Option A: for the whole server
OLLAMA_CONTEXT_LENGTH=65536 ollama serve

# Option B: bake it into a named copy of the model (what we do; see Ollama's Modelfile reference)
printf 'FROM gemma4:26b\nPARAMETER num_ctx 32768\n' > Modelfile
ollama create gemma4-32k -f Modelfile

ollama ps    # the CONTEXT column shows what is really loaded

We checked that the baked value holds. We loaded a model with 6,144 tokens, sent it twelve requests through /v1/messages, and ollama ps still reported 6,144.

Context costs memory, and parallel slots multiply it. These are loaded sizes from ollama ps on our NVIDIA DGX Spark machines (GB10, 128 GB unified memory), measured on 2 November 2026 (result file):

Model (Q4_K_M)FileContextSlotsLoaded
llama3.1:8b4.9 GB32,76819.21 GB
llama3.1:8b4.9 GB32,768422.38 GB
llama3.1:8b4.9 GB32,768839.27 GB
gemma4:26b18.0 GB32,768117.47 GB
gemma4:26b18.0 GB131,072118.36 GB
qwen3.6:35b23.9 GB32,768823.47 GB

The cost of context varies a lot by model. The 8B gains 13 GB going from one slot to four. gemma4:26b gains under 1 GB going from 32k to 131k. Ollama starts its engine with a total context of num_ctx times the number of slots, which we read from its load log (receipt). On a one-person machine, leave OLLAMA_NUM_PARALLEL at 1 unless you use subagents (Step 4).

Our machines have 128 GB. We have not measured a 16 GB laptop or a 24 GB card. To work out your own case, use how much VRAM a local LLM needs and picking the GGUF quantization.

Step 3: point Claude Code at Ollama

Two routes. Both are documented by Ollama, not by Anthropic.

The launcher. ollama launch claude --model gemma4:26b sets the variables for that one process and starts Claude Code. The CLI reference also documents --config, which configures without launching.

By hand. We prefer this because you can see every setting:

bash
export LOCAL_MODEL=gemma4-32k     # the copy from Step 2

# From Ollama's integration page
export ANTHROPIC_BASE_URL=http://localhost:11434
export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""

# Send every model alias and every subagent to the local model
export ANTHROPIC_DEFAULT_OPUS_MODEL="$LOCAL_MODEL"
export ANTHROPIC_DEFAULT_SONNET_MODEL="$LOCAL_MODEL"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="$LOCAL_MODEL"
export CLAUDE_CODE_SUBAGENT_MODEL="$LOCAL_MODEL"

# Tell Claude Code the real window, and stop background traffic
export CLAUDE_CODE_MAX_CONTEXT_TOKENS=32768
export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1

claude --model "$LOCAL_MODEL"

What each group does, and where it is documented:

  • The first three come from Ollama’s integration page. Ollama notes that the local server ignores the token’s value. Claude Code’s environment reference says ANTHROPIC_AUTH_TOKEN is sent as a bearer header and takes precedence over a saved login.
  • The four model variables are the ones Ollama’s launcher sets, which we read in its source code on 15 November 2026. Each is documented in Claude Code’s model configuration. Without them, a subagent that asks for haiku requests a Claude model name that Ollama does not have.
  • CLAUDE_CODE_MAX_CONTEXT_TOKENS matters most. The model configuration page says Claude Code assumes a 1M-token window for a model ID it does not recognise, and compacts the conversation to fit that assumption. Set it to the window you gave Ollama. The launcher’s source sets it only for Ollama’s cloud models, so with a local model you set it yourself.
  • CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC turns off auto-updates, telemetry and error reporting. It is optional. Update Claude Code by hand if you set it.

Then check inside Claude Code with /status. The Anthropic base URL line should show your Ollama address.

We did not run this Claude Code session for this update. The machine we write from runs Claude Code in production, and we do not change its configuration for a test. Step 3 is assembled from the two vendors’ documentation. What we ran ourselves is Steps 1 and 2 and the tests below.

What we ran: tool calls through the endpoint

On 15 November 2026 we sent Ollama 0.34.0 six requests per model through /v1/messages. Each carried a short system prompt and three tools shaped like a coding agent’s (Read, Bash, Grep). Three prompts needed a tool, two needed a plain answer, and one returned a tool result. Temperature 0, one run per case, checked by script. Full requests and answers: result file.

Casellama3.1:8bgemma4:26b (32k)
Read a fileRead call, correct pathRead call, correct path
Run the tests with pytestBash: pytest && pytest --versionBash: ls -R | grep test
Find uses of a functionGrep call, correct patternGrep call, correct pattern
”Reply with exactly three words”Bash: echo 'local model ready'Plain text, correct
”What does JSON stand for?”Bash: echo '...'Plain text, correct
Answer from a tool resultCorrectCorrect

Every tool call from both models was well formed: a known tool name and valid arguments. The difference is judgement. The 8B model reached for the shell to answer questions that needed no tool, in both cases. In a real session, each of those is a permission prompt and a wasted turn. The 26B model answered in text. It also chose to look for the tests before running them, which our script marked as a miss and a person would call reasonable.

This is a small probe, not a Claude Code session. Claude Code sends a far longer system prompt and many more tools, which is harder for a small model, not easier.

Two more results from the same run:

  • tool_choice had no visible effect. We sent one prompt four ways: no tool_choice, forced Bash, forced Read and none. All four answers were identical. That matches Ollama’s page, which lists it as unsupported.
  • Structured output points the same way. In our separate test of 40 Spanish invoices on 5 November 2026, gemma4:26b got every field right in all 40. qwen2.5-coder:7b managed 85% and llama3.1:8b 62.5% (result file, and the method).

Which local model, and how fast

All four models below list the tools capability in Ollama. Speeds use one prompt, 200 output tokens, on our DGX Spark machines. They are single-request speeds, not Claude Code sessions.

Model (Q4_K_M)FileGenerationMeasured
qwen2.5-coder:7b4.68 GB42.1 tok/s17 Oct 2026, Ollama 0.30.10
llama3.1:8b4.92 GB40.2 tok/s17 Oct 2026, Ollama 0.30.10
gemma4:26b17.99 GB62.0 tok/s17 Oct 2026, Ollama 0.30.10
gemma4:26b17.99 GB68.2 tok/s2 Nov 2026, Ollama 0.34.0
qwen3.6:35b23.94 GB75.8 tok/s2 Nov 2026, Ollama 0.34.0

Sources: October file and November file.

Our reading:

  • 7B to 8B: fine for learning the setup and for single-file questions. Expect stray tool calls.
  • Around 26B: the smallest size we would use for agent work. It is what we run daily.
  • qwen3-coder:30b: Ollama’s own examples use it. Its library page lists 19 GB, a 256K window and tools. We have not measured it.

Step 4: subagents

A subagent is a side task with its own context window. It returns a summary, so the main conversation stays short. Claude Code’s subagent page says subagents work on every provider.

Three things change with a local model:

  1. Pin the subagent model. A subagent can ask for an alias such as haiku. The four model variables in Step 3 send those requests to your local model.
  2. Ask for them. Ollama’s post on subagents (16 February 2026) names three of its cloud models that start subagents on their own. It makes no such claim for local models. Say it in the prompt: “use subagents to check the tests and the docs”.
  3. Parallel needs slots. With OLLAMA_NUM_PARALLEL=1, parallel subagents wait in a queue. More slots cost memory, as the table in Step 2 shows. On one GPU they also share its speed. Use subagents to keep context clean, not to go faster.

We have not measured a subagent run on a local model.

What stays on your machine, and what does not

TrafficWhere it goesHow to stop it
Prompts, code, tool calls, subagentsYour Ollama serverNothing to stop
Web searchOllama’s hosted search APIDeny the WebSearch tool
WebFetch page downloadsThe site you fetchDeny the WebFetch tool
WebFetch hostname checkapi.anthropic.com, hostname onlyskipWebFetchPreflight: true in settings
Usage metrics, update checksAnthropic and its providersCLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1

Rows three to five come from Claude Code’s data usage page. It lists metrics as on by default for the Claude API route, which is how Claude Code treats a custom base URL. It also says metrics never include code, prompts or file paths.

Web search is not local. Claude Code’s WebSearch tool normally runs on Anthropic’s servers. Ollama’s post says its compatibility layer runs the search instead and hands the results to the model. Ollama’s web search page documents that API at ollama.com, and says it needs a free Ollama account. The same compatibility page says hosted web search is “not fully supported”. We have not tested it with a local model.

If your rule is that nothing leaves, skip it. Claude Code’s tools reference says the search backend cannot be changed, and points to an MCP server as the way to add another one. We self-host SearXNG for that. See connecting agents to tools with MCP.

Failures you will hit

1. The model forgets the start of the conversation. We sent a 64,507-character prompt to a model loaded with 6,144 tokens of context. A code word sat on the first line and the question on the last. Ollama returned HTTP 200, reported 3,070 input tokens, and the model answered that it saw no code word. No error, no warning. The same question with a short prompt was answered correctly. Fix: Step 2, then CLAUDE_CODE_MAX_CONTEXT_TOKENS in Step 3. If it still drifts, run /compact earlier.

2. model 'claude-haiku-4-5' not found. We reproduced this: Ollama answers HTTP 404 with a not_found_error when asked for a Claude model name. It means some part of Claude Code is still asking for a Claude model. Fix: the four model variables in Step 3. Ollama’s compatibility page offers another route, copying your model under a Claude name with ollama cp.

3. Your variables are ignored. Claude Code’s connection page says a value in a settings file’s env block wins over a shell export. Run /status and read the base URL line.

4. Token counts look approximate. On our Ollama 0.34.0, /v1/messages/count_tokens answered 404 page not found. Claude Code’s gateway guide says it then falls back to a character-based estimate, so /context shows approximate figures. Nothing to fix.

5. The first answer takes a long time. The model is loading from disk. In our context test, gemma4:26b took 32 seconds on its first load and 9 to 10 seconds on the next two. Claude Code waits up to 10 minutes for a response to start by default (API_TIMEOUT_MS).

6. 400 errors that name an unknown field. Claude Code’s troubleshooting table gives CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1 for servers that reject pre-release fields. We did not see this error.

Limits against the hosted models

  • Context. Claude Code’s model page gives the hosted Claude models windows of 200K to 1M tokens. Our daily local model runs at 32k. Long sessions compact sooner and lose detail.
  • No prompt caching. Ollama lists cache_control as unsupported. In our probe Ollama did report reused prompt tokens on repeated prefixes, but that is its own mechanism.
  • Judgement. The 26B model formed tool calls correctly in our probe. We have no measurement of it on long, multi-file work, and we do not claim it matches a frontier model there.
  • Features tied to an Anthropic login. Remote Control, voice dictation and claude.ai connectors are unavailable with a custom base URL or token, according to the Claude Code documentation.
  • Support. None from Anthropic for this setup.

What we did not test: a full Claude Code session on today’s versions, Ollama 0.40.x, web search with a local model, subagents on a local model, a 16 GB or 24 GB machine, qwen3-coder, and whether Claude Code accepts a declared window as small as 32k (its documentation gives no minimum). An earlier version of this article quoted session timings from September 2026. We had no saved result file for them, so we removed them.

Frequent questions

Do I still need LiteLLM or another proxy? No. Ollama has served the Anthropic format since v0.14.0. Use a proxy only if you want routing, keys or logs.

Is this supported by Anthropic? No. Their documentation says so directly. Ollama documents and maintains its side.

Do I need an Anthropic account or key? Ollama’s setup uses a placeholder token, and Claude Code’s documentation says that token takes the place of a login. We have not tried it on a machine that never signed in.

Does web search work offline? No. It calls Ollama’s hosted API.

Can I keep my normal Claude Code login? Yes. Shell exports last for that terminal only. Close it and Claude Code is back to normal.

Which model first? The largest tools-capable model that fits your memory with a 32k window. On our machines that is gemma4:26b.

Next steps

Work with us

We run coding agents on local models every day and keep a hosted model for the hard cases. If you want that split designed for your team, sized to the machines you already own, get in touch or see how our consulting works.

Diagram
Share: LinkedIn X
Veredicto semanal

Get new guides before anyone else

Subscribe and we tell you when new guides, templates and workflows go up. One email a week, no spam.

Already published: 102 guides and 25 templates. All free, no signup.

Bonus: the local-AI starter pack PDF when you subscribe
Once a week No spam Unsubscribe anytime

See what you get