Qwen2.5-72B-Instruct Reviewed Locally: The 72B That Speaks Spanish, and the 35B That Runs 15x Faster
By the end of this post you will know what Qwen2.5-72B-Instruct needs to run, what it does well in Spanish, and why a newer 35B mixture-of-experts model on the same machine answered the same Spanish prompt fifteen times faster. Cheapest hardware that runs the 72B properly: a 64 GB Mac. The 35B alternative runs on a 32 GB machine.
What you need
- 64 GB of GPU-reachable memory for the 72B at 4 bits (the file is 47 GB), or 32 GB for the
qwen3.6:35bcomparison model (23 GB resident at 8k context on our box). - Ollama, free, no account.
- 50 GB of disk and a patient download.
- One Spanish prompt you actually use at work, to repeat our test on your own hardware.
Step 1: Know the model
Qwen2.5-72B-Instruct, from Alibaba’s Qwen team, has 72.7 billion parameters (70.0 billion without embeddings), 80 layers, grouped-query attention with 64 query and 8 key-value heads, a 131,072-token input context and up to 8,192 tokens of generation (model card). The card lists support for more than 29 languages, with Spanish named explicitly alongside Portuguese, Italian, French and German. That multilingual training is the reason it has been a favourite for Spanish-language work since late 2024.
Two details to note before you deploy it. The licence on the card is “qwen”, a custom licence rather than Apache 2.0, so read it once. And contexts beyond 32k tokens need YaRN scaling (factor 4) enabled in the runtime, which Ollama handles for you but some servers do not.
Step 2: Choose the quantization
Sizes from bartowski’s GGUF repository and the Ollama tags page. The 72B is about 10 percent bigger than Llama 3.3 70B at every level (47 GB vs 43 GB at Q4_K_M), so it needs the same hardware with a little less headroom.
| Quantization | File size | Ollama tag | Fits comfortably on |
|---|---|---|---|
| Q8_0 | 77 GB | qwen2.5:72b-instruct-q8_0 | 128 GB unified memory |
| Q6_K | 64 GB | qwen2.5:72b-instruct-q6_K | 96 GB Mac |
| Q5_K_M | 54 GB | qwen2.5:72b-instruct-q5_K_M | 64 GB Mac (tight, short context) |
| Q4_K_M (default) | 47 GB | qwen2.5:72b | 64 GB Mac, 2x24 GB GPUs with offload |
| IQ4_XS | 40 GB | (GGUF only) | 2x24 GB GPUs fully on GPU |
| Q3_K_M | 38 GB | qwen2.5:72b-instruct-q3_K_M | 48 GB, visible quality loss |
| IQ2_M | 29 GB | (GGUF only) | one 32 GB machine, noticeably worse |
Step 3: Run the same Spanish test we ran
ollama pull qwen2.5:72b # 47 GB
ollama pull qwen3.6:35b # 23 GB, the comparison model
cat > es_prompt.json <<'EOF'
{"model":"qwen2.5:72b","stream":false,"think":false,
"prompt":"Redacta en español, en unas 300 palabras y sin listas, un correo profesional a un cliente de una asesoría de Valencia explicándole qué documentación debe preparar para el cierre contable del tercer trimestre y por qué conviene entregarla antes del día 15.",
"options":{"num_predict":450,"temperature":0.3,"num_ctx":8192}}
EOF
curl -s localhost:11434/api/generate -d @es_prompt.json \
| python3 -c "import json,sys; d=json.load(sys.stdin); print(round(d['eval_count']/d['eval_duration']*1e9,1),'tok/s'); print(d['response'][:400])"
ollama ps # resident memory of what just ranChange "model" to qwen3.6:35b and run it again. The think:false flag matters for qwen3.6: without it the model spends its whole token budget reasoning silently and returns an empty response, which is exactly what happened to us on the first attempt.
What we measured
All runs on 2026-09-09, on our NVIDIA GB10 workstation (128 GB unified memory, Ollama 0.30.10), a machine shared with other services, so treat the numbers as a floor rather than a lab result. For the 72B we used qwen2.5vl:72b, the vision-language release whose language model is the Qwen2.5-72B backbone (Q4_K_M, 73.4B parameters including the vision tower, ollama show). It is a proxy for the text-only Instruct model, not the same file, and we say so.
| Model | Architecture | Speed on the Spanish prompt | Resident memory (ollama ps) | Prompt processing |
|---|---|---|---|---|
| qwen2.5vl:72b, Q4_K_M | dense, 73.4B | 3.2 tok/s cold, 2.7 tok/s second run (450 tokens) | 49 GB at 8k context | 136 to 141 tok/s |
| qwen3.6:35b, Q4_K_M | mixture of experts, 36B total | 60.4 / 36.6 / 39.1 tok/s across three runs | 23 GB at 8k context (29 GB at its 262k default) | 155 to 166 tok/s |
| llama3.1:8b, Q4_K_M | dense, 8B | 29.3 tok/s | 5.3 GB at 4k context | 759 tok/s |
For reference, Ollama measured llama3.1:70b Q4_K_M at 4.4 tok/s on a DGX Spark, which uses the same GB10 chip as our workstation (Ollama blog, October 2025). Our 3 tok/s on a slightly larger model, on a busy machine, is consistent with that. On faster-memory hardware the 70B class does better: 17.2 tok/s for Llama 3.3 70B Q4_K_M on a 64 GB M4 Max (Markaicode) and 21.0 tok/s for Llama 3 70B on two RTX 4090 (Sergey Nog). We found no published measurement of Qwen2.5-72B specifically on those two setups; scaling by file size (47 vs 43 GB) suggests roughly 10 percent slower, so expect about 15 tok/s on the Mac and about 19 tok/s on the GPU pair. That is our estimate, not a measurement.
Why 36B parameters ran 15 times faster than 72B
A dense model uses every parameter for every token: the whole 49 GB streams through memory each time. A mixture-of-experts (MoE) model like qwen3.6:35b keeps all 36B parameters in memory but activates only a few “expert” blocks per token, so it reads a fraction of the weights and generates at 8B-class speed with far more knowledge than an 8B model. That is the trade that has made the older dense 72B a harder sell in 2026.
The Spanish quality of both was good on our test. The 72B opened with a correct formal register: “Espero que este correo le encuentre bien. Como asesoría especializada en Valencia, nos complace recordarle la importancia de prepararse para el cierre contable del tercer trimestre.” The 35B produced an equally natural, slightly longer letter in the same register, and finished in 12 seconds instead of 166. The 8B model was also fluent but generic, and used the same template phrases a Spanish reader has seen a thousand times.
Where the 72B still earns its place
- Long Spanish and Portuguese documents where nuance matters. Contracts, tenders, regulatory text: the dense 72B misses fewer details than the 8B and is on par with the 35B in our small test, so it is a fair choice if you already have it deployed and validated.
- Structured output at scale. Qwen2.5’s strengths on the card are long-text generation, table understanding and JSON output; a nightly batch that turns 2,000 PDFs into rows does not care about 3 tok/s.
- Validated pipelines. If your prompts were tuned on Qwen2.5-72B in 2025 and the outputs pass review, changing models has a cost too. Measure before you switch.
For everything interactive, run qwen3.6:35b or an 8B model and keep the 72B for queued work.
Honest limits
- 3 tok/s on unified-memory workstations. A 450-token answer takes two and a half minutes on our box. Only a 64 GB Mac or a GPU pair makes it conversational.
- Memory. 47 GB of weights plus context: no 32 GB machine, no laptop.
- Licence is not Apache 2.0. The qwen3.6 model is Apache 2.0; the 2.5-72B is not. Check before redistributing.
- Knowledge stops in 2024. The model was released in September 2024 and the card states no cutoff date, so pair it with retrieval over your own documents for anything current.
Next steps
- What Q4 and Q5 cost in quality, with numbers: Quantization explained.
- If you need step-by-step reasoning more than languages: DeepSeek-R1 local review.
- Whether a 64 GB machine pays for itself: Cloud vs local break-even.
Work with us
We run this exact Spanish-prompt test on a client’s own documents before recommending a model, and we publish what we measure. If you want it run on your workload, get in touch or see how our consulting works.