Llama 3.3 70B on Your Own Hardware: What It Takes to Run It and When It Earns Its Keep
By the end of this post you will know whether Llama 3.3 70B fits on the machine you have, what speed to expect from it, and which three business tasks justify it over an 8B model that runs ten times faster. The cheapest entry point that works is a 64 GB Mac; a 16 GB laptop is not enough, and we say so up front.
What you need
- A machine with at least 48 GB of memory that the GPU can reach: a 64 GB Apple Silicon Mac, a PC with two 24 GB graphics cards, or a unified-memory workstation. The 4-bit file alone is 43 GB.
- Ollama installed. No account, no API key.
- About 45 GB of free disk and a download window: the default tag is 43 GB.
- Twenty minutes for the steps below once the download finishes.
Step 1: Know what you are downloading
Llama 3.3 70B Instruct was released by Meta on 6 December 2024 under the Llama 3.3 Community License. It is a dense 70-billion-parameter text model with a 128k-token context, trained on more than 15 trillion tokens, with official support for eight languages including Spanish (model card). Its knowledge cutoff is December 2023, so it does not know anything that happened after that unless you put it in the prompt.
The reason people still care about it in 2026: on Meta’s own table it matches or beats the 405B model on instruction following (IFEval 92.1 vs 88.6) and maths (MATH 77.0 vs 73.8), and it leaves Llama 3.1 8B far behind on every hard benchmark (MMLU Pro 68.9 vs 48.3, GPQA Diamond 50.5 vs 31.8). Those are Meta’s numbers, not ours.
Step 2: Pick a quantization that fits your memory
Quantization stores each weight in fewer bits so the file shrinks. Sizes below are from bartowski’s GGUF files and the Ollama library tags. Add 2 to 6 GB on top for the context cache and runtime.
| Quantization | File size | Ollama tag | Fits comfortably on |
|---|---|---|---|
| Q8_0 | 75 GB | llama3.3:70b-instruct-q8_0 | 128 GB unified memory |
| Q6_K | 58 GB | llama3.3:70b-instruct-q6_K | 96 GB Mac, 128 GB workstation |
| Q5_K_M | 50 GB | llama3.3:70b-instruct-q5_K_M | 64 GB Mac (tight) |
| Q4_K_M (default) | 43 GB | llama3.3:70b | 64 GB Mac, 2x24 GB GPUs |
| IQ4_XS | 38 GB | (GGUF only) | 2x24 GB GPUs with more context room |
| Q3_K_M | 34 GB | llama3.3:70b-instruct-q3_K_M | 48 GB Mac, visible quality loss |
| IQ2_M | 24 GB | (GGUF only) | one 24 GB GPU, noticeably worse answers |
Our rule: stay at Q4_K_M or above for anything a client will read. Below Q4 the model still talks fluently, but it starts to miss details in long documents, which is exactly the job you bought a 70B for.
Step 3: Run it
ollama pull llama3.3:70b # 43 GB, Q4_K_M
ollama run llama3.3:70b "Summarise the three main obligations in this contract: ..."
ollama ps # shows resident memory and whether it is 100% on the GPUIf ollama ps shows a CPU percentage instead of 100% GPU, the model did not fit in graphics memory and you will get one or two tokens per second. Drop to a smaller quantization or reduce the context with OLLAMA_CONTEXT_LENGTH=8192.
To time a run yourself, ask the API for the counters:
curl -s localhost:11434/api/generate -d '{"model":"llama3.3:70b","prompt":"Explain IVA deferral for a Spanish SME in 200 words.","stream":false}' \
| python3 -c "import json,sys; d=json.load(sys.stdin); print(round(d['eval_count']/d['eval_duration']*1e9,1),'tok/s')"What we measured and what we cite
We did not download the 43 GB Llama 3.3 file for this post. Instead we ran the 72B-class model that was already on our machine, qwen2.5vl:72b (Q4_K_M, 49 GB resident), which tells you what a dense 70B-class model does on our hardware. Every other row is a third-party measurement with a link.
| Hardware | Model | Speed | Source and date |
|---|---|---|---|
| Our NVIDIA GB10 workstation, 128 GB unified memory, Ollama 0.30.10 | qwen2.5vl:72b Q4_K_M, 450-token Spanish answer, 8k context | 3.2 tok/s cold, 2.7 tok/s second run; 49 GB resident | measured 2026-09-09, machine shared with other services |
| Same workstation | llama3.1:8b Q4_K_M, same Spanish prompt | 29.3 tok/s | measured 2026-09-09 |
| NVIDIA DGX Spark (same GB10 chip) | llama3.1:70b Q4_K_M | 4.4 tok/s decode, 1,911 tok/s prefill | Ollama blog, 23 Oct 2025 |
| MacBook Pro M4 Max, 64 GB | Llama 3.3 70B Q4_K_M, Ollama | 17.2 tok/s | Markaicode |
| MacBook Pro M4 Max, 64 GB | Llama 3.1 70B Q4_K_M, llama.cpp | 12.5 tok/s | Markaicode |
| 2x RTX 4090, 48 GB total | Llama 3 70B, 4-bit | 21.0 tok/s | Sergey Nog |
| 1x RTX 3090, 24 GB, rest on CPU | Llama 3 70B Q4_K_M | 5.2 tok/s | GigaGPU |
Read the table honestly. A 70B model is memory-bandwidth bound: the whole 43 GB has to stream through the processor for every token. That is why two 4090s or an M4 Max beat our 128 GB workstation by a factor of five on generation, while the workstation wins where memory capacity matters (bigger quantizations, long context, several models loaded at once). It is also why our 8B model answered the same Spanish prompt at 29 tok/s: nine times less data per token.
Which SME tasks justify 70B over 8B
An 8B model handles classification, short summaries, first-draft emails and structured extraction from clean documents. Reach for 70B when the task looks like one of these:
- Long Spanish contracts and tenders. Reading 40 pages and answering “which clauses shift liability to us” needs the reasoning depth and the 128k context. The 8B model paraphrases; the 70B model cites the clause.
- Multi-step analysis. “Compare these three supplier quotes against our purchasing policy and flag exceptions” chains several judgements. The MMLU Pro gap (68.9 vs 48.3) shows up here as fewer silent mistakes.
- Code review of business logic. HumanEval 88.4 vs 72.6 is the difference between spotting an off-by-one in an invoicing script and complimenting it.
If your daily volume is high and the tasks are simple, run the 8B model at 30 tok/s and keep the 70B for a nightly batch. At 3 to 20 tok/s, a 500-token answer takes between 25 seconds and three minutes, which is fine for a queue and frustrating for a chat window.
Honest limits
- Memory is the hard wall. No 16 GB or 32 GB machine runs this model usefully. Do not try to make a laptop do it.
- Electricity. A two-GPU tower draws several times what a 64 GB Mac draws under load; check the TDP on the spec sheet of the cards you are considering. Factor it into the break-even calculation.
- December 2023 knowledge. Pair it with retrieval over your own documents; do not ask it about 2026 regulation.
- Licence. The Llama 3.3 Community License allows commercial use with conditions; read it once before deploying for clients.
Next steps
- Understand what the Q4 and Q5 labels cost you in quality: Quantization explained.
- Need reasoning rather than raw size? Compare with DeepSeek-R1 for local reasoning.
- Work out whether the hardware pays for itself: Cloud vs local break-even in 15 minutes.
Work with us
We size 70B deployments against the client’s real documents and token counts before recommending a Mac, a GPU pair or a shared workstation. If you want a second opinion on yours, get in touch or read how our consulting works.