# Llama 3.3 70B locally: the memory sum, hardware and speed

> What it takes to run Llama 3.3 70B on your own machine: file sizes, the memory sum by context, hardware that holds it and published speeds. Not run by us.

- Published: 2026-09-09
- Updated: 2026-11-15
- Author: Jacobo González Jaspe
- Language: en
- Tags: Model Review, Llama 3.3, Local AI, Hardware, Quantization
- Canonical URL: https://vorluxai.com/en/blog/llama-3-3-70b-local-review-what-it-takes-to-run-en/
- Source: VORLUX AI (J4SGON S.L.), https://vorluxai.com/

---

By the end of this guide you will know whether Llama 3.3 70B fits the machine you have, what you would need if it does not, what speed to expect and whether a smaller model would serve you better. A calculator is enough to follow it. A 16 GB laptop is not enough to run it, and nothing below changes that.

One thing first. **We have not run Llama 3.3 70B.** Our machines are shared, and a 43 GB load on them was a risk we chose not to take. So this is not a test report. The model's facts come from Meta and the file repositories, the memory figures from a sum checked against real loads of other models, and the 70B speeds from other people, named. What we measured ourselves is the largest models we do run.

> **TL;DR**
> - The usual file (Q4_K_M) is 42.52 GB. Add 2.68 GB of context cache at 8k and about 1 GB of buffers: plan for 46 GB.
> - It fits in 64 GB of unified memory. On 48 GB (one card, or two of 24 GB) it fits only at a short context. On one 24 GB card it does not fit at a usable quality.
> - Published speeds for a 70B at Q4_K_M: 4.4 tokens per second on a DGX Spark, 16 to 19 on two 24 GB cards. On our Sparks we measured 68 to 76 with newer 26B and 35B models.
> - Verdict: starting today, try a newer mid-size model first. Buy for a 70B only when your own test says the smaller one is not enough.

## The model's facts

From Meta's [model card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct), read on 2026-11-15:

| Fact | Value |
|---|---|
| Released | 6 December 2024 |
| Type | Dense text model: text in, text and code out. No images. |
| Parameters | 70 billion (the 16-bit file is 141.12 GB: 70.6 billion at two bytes each) |
| Context | 128k tokens (131,072 in its `config.json`) |
| Languages supported | English, German, French, Italian, Portuguese, Hindi, Spanish, Thai |
| Knowledge cutoff | December 2023 |
| Licence | Llama 3.3 Community License |

Two things follow for a Spanish office. Spanish is on the supported list; for languages outside it, the card discourages use without fine-tuning and controls of your own. And the model knows nothing after 2023, so current rules or prices have to come from your documents.

Meta's table on the card gives it 68.9 on MMLU Pro against 48.3 for Llama 3.1 8B, and 92.1 on instruction following (IFEval) against 80.4. Those are Meta's numbers under Meta's conditions.

## What "open" means here

This section is information, not legal advice. We read the [licence](https://dev.meta.ai/llama/llama3_3/license) and the [acceptable use policy](https://dev.meta.ai/llama/llama3_3/use-policy) on 2026-11-15; read them yourself before a client depends on the model.

The weights are free to download and you may use them commercially. That is the "open" part. But the model card calls the licence "a custom commercial license": it is Meta's own agreement, not Apache or MIT. What it asks:

- **If you redistribute the model or a product containing it** (section 1.b): pass on a copy of the agreement, show "Built with Llama" somewhere prominent, and keep the attribution notice.
- **If you use it to train or improve another model that you distribute**: that model's name must begin with "Llama".
- **Above 700 million monthly active users** (section 2) you need a separate licence from Meta.
- **The acceptable use policy binds you** (section 1.b.iv), and Meta may end the licence if you breach the agreement (section 6). California law governs it (section 7).

From the use policy, an office should know these: no unlicensed practice of a profession (1.d), no military or critical-infrastructure use (2.a and 2.d), no presenting the output as written by a person (3.e), and a duty to tell end users about known dangers of your system (4).

The policy also withholds the licence from people and companies based in the EU for any *multimodal* models. Llama 3.3 70B is text only, so as we read it that clause does not reach this model.

Meta's repository on Hugging Face is gated: you accept the licence and share contact details before downloading.

## File sizes by quantisation

Sizes from [bartowski's GGUF repository](https://huggingface.co/bartowski/Llama-3.3-70B-Instruct-GGUF), read from its file list on 2026-11-15. The quality notes are that repository's, in our words. Tags are from [Ollama's list](https://ollama.com/library/llama3.3/tags), same day.

| Quantisation | File | Ollama tag | The repository's note |
|---|---|---|---|
| Q8_0 | 74.98 GB | `llama3.3:70b-instruct-q8_0` | Highest quality, usually more than needed |
| Q6_K | 57.89 GB | `llama3.3:70b-instruct-q6_K` | Near perfect, recommended |
| Q5_K_M | 49.95 GB | `llama3.3:70b-instruct-q5_K_M` | High quality, recommended |
| Q4_K_M | 42.52 GB | `llama3.3:70b` (default) | Good quality, the default for most uses |
| IQ4_XS | 37.90 GB | none | Decent, recommended |
| Q2_K | 26.38 GB | `llama3.3:70b-instruct-q2_K` | Very low quality but usable |
| IQ2_XXS | 19.10 GB | none | Very low quality |

Ollama rounds the default to 43 GB. How to choose between these labels is in our guide to [picking the right GGUF file](https://vorluxai.com/en/blog/gguf-quantization-pick-the-right-file-for-your-hardware-en/). We have no quality measurement of our own for any of them.

## The memory sum

The file is not the whole bill. The method is the one in our [VRAM guide](https://vorluxai.com/en/blog/how-much-vram-local-llm-en/), where the sum matched 13 real loads of smaller models to the megabyte ([load-log receipt](https://vorluxai.com/data/measurements/llama-load-log-buffers-2026-11-15.json)):

```
memory = weights (the file) + context cache × parallel slots + compute buffers
```

### Step 1: read the three values

The cache depends on layers, KV heads and head dimension, all in the model's `config.json`. Meta's repository is gated, so this reads an ungated copy:

```bash
curl -sL https://huggingface.co/unsloth/Llama-3.3-70B-Instruct/raw/main/config.json | python3 -c "
import sys, json
c = json.load(sys.stdin)
for k in ('num_hidden_layers', 'num_key_value_heads', 'head_dim', 'hidden_size', 'vocab_size'): print(k, '=', c[k])"
```

What it printed for us on 2026-11-15:

```
num_hidden_layers = 80
num_key_value_heads = 8
head_dim = 128
hidden_size = 8192
vocab_size = 128256
```

### Step 2: the cache, by context

Each token stores a K and a V vector per layer, at 16 bits: 80 × 8 × 128 × 2 × 2 = 327,680 bytes per token. That is two and a half times what Llama 3.1 8B needs, and the file's quantisation does not change it.

```python
FILE_GB = 42.52                                    # Q4_K_M file
per_token = 80 * 8 * 128 * 2 * 2                   # bytes
for ctx, slots in [(4096, 1), (8192, 1), (16384, 1), (32768, 1), (131072, 1), (8192, 8)]:
    cache = per_token * ctx * slots / 1e9
    print(f"{ctx:>7} x {slots}: cache {cache:5.2f} GB, weights + cache {FILE_GB + cache:6.2f} GB")
```

We ran it (it loads nothing):

```
   4096 x 1: cache  1.34 GB, weights + cache  43.86 GB
   8192 x 1: cache  2.68 GB, weights + cache  45.20 GB
  16384 x 1: cache  5.37 GB, weights + cache  47.89 GB
  32768 x 1: cache 10.74 GB, weights + cache  53.26 GB
 131072 x 1: cache 42.95 GB, weights + cache  85.47 GB
   8192 x 8: cache 21.47 GB, weights + cache  63.99 GB
```

At the full 128k the cache is as large as the model.

### Step 3: the slots

The last line is the same model serving eight people at once. Ollama's [FAQ](https://docs.ollama.com/faq) says required memory scales with `OLLAMA_NUM_PARALLEL` times the context length; the default is 1. Each extra user at 8k costs this model 2.68 GB. Our second machine runs eight slots, and there a 24B model went from 19.79 GB to 25.64 GB loaded just by doubling the context from 4k to 8k ([4k](https://vorluxai.com/data/measurements/mistral-small-24b-speed-ctx4k-2026-11-15.json), [8k](https://vorluxai.com/data/measurements/mistral-small-24b-speed-ctx8k-2026-11-15.json)).

### Step 4: buffers, and what `ollama ps` leaves out

llama.cpp also reserves working memory: 112 to 912 MiB in our logs, depending on batch size and slots. We allow 1 GB for a 70B. That is an allowance, not a measurement of this model.

`ollama ps` may then show *less* than the file size. It reports the GPU buffers, and the token embedding table stays on the host side. In our Llama 3.1 8B load that table is 281.81 MiB: its 128,256 × 4,096 values at 4.5 bits each. The 70B has 128,256 × 8,192. Stored the same way, that is 0.59 GB missing from `ollama ps` and still in memory. We have not checked it on a real load.

## What fits in each machine

Planning sum for Q4_K_M with one slot (file + cache + 1 GB): 44.86 GB at 4k, 46.20 GB at 8k, 54.26 GB at 32k and 86.47 GB at 128k. As in the VRAM guide, a card keeps 0.5 GB back and unified memory above 32 GB keeps 6 GB for the system. "Tight" means over 85% of what is left.

| Machine memory | Usable | 4k | 8k | 32k | 128k |
|---|---|---|---|---|---|
| One 24 GB card | 23.5 GB | no | no | no | no |
| One 32 GB card | 31.5 GB | no | no | no | no |
| Two 24 GB cards | 47 GB | tight | tight | no | no |
| One 48 GB card | 47.5 GB | tight | tight | no | no |
| 64 GB unified | 58 GB | yes | yes | tight | no |
| 96 GB unified | 90 GB | yes | yes | yes | tight |
| 128 GB unified | 122 GB | yes | yes | yes | yes |

How to read it:

- **24 and 32 GB.** Only IQ2_XXS fits in 24 GB (21.44 GB at 4k) and Q2_K in 32 GB (30.06 GB at 8k, tight). The repository rates both very low quality. A good 24B at Q4 is a better use of those cards.
- **48 GB.** Q4_K_M works up to 8k with almost nothing spare; at 16k the sum is 48.89 GB and it is out. IQ4_XS buys room: 44.27 GB at 16k. Two cards are not one pool, since whole layers go to one or the other, so expect less headroom. We have not run a split load.
- **64 GB unified.** The first size where it is comfortable. Q5_K_M also fits up to 16k (56.32 GB, tight).
- **128 GB unified.** Q8_0 at 32k is 86.72 GB. But "128 GB" is not 128 free: our own machines report 121.7 GB in total, and with their usual services had 65.4 to 75.1 GB available before the loads in our receipts ([first machine](https://vorluxai.com/data/measurements/dgx-spark-review-prime-2026-11-02.json), [second](https://vorluxai.com/data/measurements/mistral-small-24b-speed-ctx4k-2026-11-15.json)). That is why we did not load this model.

## Hardware that can hold it

Vendor facts, each read on 2026-11-15:

- **Two 24 GB cards.** [NVIDIA's comparison page](https://www.nvidia.com/en-us/geforce/graphics-cards/compare/) lists the RTX 3090 and RTX 4090 at 24 GB, drawing 350 W and 450 W per card, and the RTX 5090 at 32 GB and 575 W. Ollama's FAQ says a model that does not fit on one GPU "will be spread across all the available GPUs".
- **One 48 GB card.** NVIDIA's [professional line](https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/) has the RTX A6000 and RTX 6000 Ada at 48 GB and 300 W, the RTX PRO 5000 Blackwell at 48 or 72 GB, and the RTX PRO 6000 Blackwell at 96 GB.
- **Unified memory, Apple.** The [MacBook Pro](https://www.apple.com/macbook-pro/specs/) with M5 Max (40-core GPU) comes with 48, 64 or 128 GB and 614 GB/s of memory bandwidth. The [Mac Studio](https://www.apple.com/mac-studio/specs/) with M5 Ultra goes from 96 to 512 GB, at 1.2 TB/s. macOS limits how much the GPU may hold: see the VRAM guide.
- **Unified memory, NVIDIA.** The [DGX Spark](https://www.nvidia.com/en-us/products/workstations/dgx-spark/), the machine we own, has 128 GB and 273 GB/s.

Bandwidth is in that list for a reason. A dense model reads all its weights for every token, so bandwidth divided by file size caps the speed for one user: 273 ÷ 42.52 is 6.4 tokens per second on a Spark, however much memory is free. What to buy at each budget is in our [hardware catalogue](https://vorluxai.com/en/blog/local-ai-hardware-catalog-2026-what-to-buy-at-each-budget-en/).

## What speed to expect

**Published by others.** None of these is Llama 3.3 itself: they are the earlier 70B models of the same family at Q4_K_M.

| Hardware | Generation | Source and conditions |
|---|---|---|
| DGX Spark, 128 GB unified | 4.42 tok/s | [Ollama's blog](https://ollama.com/blog/nvidia-spark-performance), 23 Oct 2025: llama3.1:70b, Ollama 0.12.6, 500 tokens out |
| Two RTX 3090 | 16.29 tok/s | [XiongjieDai's benchmark repository](https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference): Llama 3 70B, `llama-bench`, 1,024 tokens, May 2024 |
| Two RTX 4090 | 19.06 tok/s | same |
| RTX A6000, 48 GB | 14.58 tok/s | same |
| RTX 6000 Ada, 48 GB | 18.36 tok/s | same |
| M3 Max (40-core GPU), 64 GB | 7.53 tok/s | same |
| One RTX 3090 or one RTX 4090 | out of memory | same |

The repository's figures are from 2024 software, the cards on rented machines. Treat them as an order of magnitude.

**Measured by us**, with the protocol of our [30-minute benchmark guide](https://vorluxai.com/en/blog/benchmark-local-model-own-machine-30-minutes-en/): one 278-word prompt, 200 tokens out, second run reported. The 8B ran on our first Spark (Ollama 0.30.10), the other three on the second (Ollama 0.34.0, eight slots).

| Model (Q4_K_M) | File | Generation | First token | Receipt |
|---|---|---|---|---|
| llama3.1:8b, dense | 4.92 GB | 43.7 tok/s | 0.385 s | [2026-11-02](https://vorluxai.com/data/measurements/dgx-spark-review-prime-2026-11-02.json) |
| mistral-small:24b, dense | 14.33 GB | 14.4 tok/s | 0.36 s | [2026-11-15](https://vorluxai.com/data/measurements/mistral-small-24b-speed-ctx4k-2026-11-15.json) |
| gemma4:26b, mixture of experts | 17.99 GB | 68.2 tok/s | 0.212 s | [2026-11-02](https://vorluxai.com/data/measurements/dgx-spark-review-spark2-2026-11-02.json) |
| qwen3.6:35b, mixture of experts | 23.94 GB | 75.8 tok/s | 0.243 s | same file |

[Mistral Small 24B](https://vorluxai.com/en/blog/mistral-small-24b-instruct-2501-en/) is the closest thing we have to an anchor: dense, like the 70B, and measured today. If speed fell in proportion to file size, its 14.4 tokens per second would become 4.9 for a 42.52 GB file. That is a sum, not a measurement, and it lands beside Ollama's published 4.42.

The two fastest rows are the largest files we run. They are mixture-of-experts models, which read only part of their weights per token. A 500-token answer takes about 113 seconds at 4.42 tokens per second, 26 at 19.06 and under 7 at 75.8.

## If you run it: the commands and the checks

We did not run these with this model. They are the commands we use daily with others, on [Ollama](https://vorluxai.com/en/blog/install-ollama-first-local-llm-en/).

1. Do the sum for your context and slots. Stop here if it does not fit.
2. Fix the context before the first load. Ollama's [context length page](https://docs.ollama.com/context-length) sets the default by memory: 4k under 24 GiB, 32k from 24 to 48, 256k from 48 up.

```bash
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama pull llama3.3:70b
ollama run llama3.3:70b "Summarise this clause in three bullet points: ..."
```

3. Check where it landed:

```bash
ollama ps
```

The `PROCESSOR` column should read `100% GPU` and `CONTEXT` should be the value you set. The FAQ's example of a model that did not fit is `48%/52% CPU/GPU`.

4. Compare the load log with your sum (the command is in the VRAM guide) and time it with your own prompt, discarding the first run.

## What goes wrong

- **The default context.** On a 48 GB setup a 32k default means 10.74 GB of cache: the sum is 54.26 GB and part of the model goes to the CPU. At the full 131,072 tokens the cache alone is 42.95 GB per slot. Set it yourself.
- **A CPU/GPU split in `ollama ps`.** The CPU layers set the pace. Shorten the context, cut the slots or take a smaller file, in that order. We have no number of our own for a split.
- **Unified memory that runs out.** On our machines the symptom was a frozen desktop and `NV_ERR_NO_MEMORY` in the driver log (see the DGX Spark review). Check available memory first, not total.

## When a smaller model is the better choice

More often than the benchmark tables suggest. Three things we measured:

- **Extraction.** On 40 synthetic Spanish invoices, gemma4:26b got 360 of 360 fields right; the 7B and 8B models got 321 to 344 ([receipt](https://vorluxai.com/data/measurements/structured-extraction-2026-11-05.json), [method](https://vorluxai.com/en/blog/ollama-structured-output-invoices-measured-en/)). A 17.99 GB file left nothing for a 70B to improve on that test.
- **Spanish writing.** Mistral Small 24B handled all three of our office tasks (an email, a summary, an invoice to JSON), which no 7B or 8B model did. Its email took 20.3 seconds against 7.7 for llama3.1:8b ([answers](https://vorluxai.com/data/measurements/mistral-small-24b-spanish-2026-11-15.json), [scores](https://vorluxai.com/data/measurements/mistral-small-24b-spanish-2026-11-15-scores.json)). One reader scored them, not a panel.
- **Waiting.** The speed table above.

So test an 8B, then a mid-size model, on your own documents. Move to a 70B only if the failure you see is one that size fixes and the wait is acceptable. A 70B earns its place in a nightly queue more easily than in a chat window.

## Has it been overtaken?

In releases, yes. What we can source, read on 2026-11-15:

- **Meta itself.** [Llama 4](https://ai.meta.com/blog/llama-4-multimodal-intelligence/) arrived on 5 April 2025: Scout (109 billion parameters in total, 17 billion active) and Maverick (400 billion in total). Both are multimodal and larger, so not an easier local option. Meta's post describes Maverick as high quality at a lower price than Llama 3.3 70B.
- **A smaller Meta model under Apache 2.0.** [Muse Glimmer 30B](https://huggingface.co/meta-models/Muse-Glimmer-30B) appeared in August 2026. Its card names Meta's Superintelligence Lab as author and gives about 29.6 billion parameters, a January 2026 knowledge cutoff and a 4-bit version under 20 GB. [Artificial Analysis](https://artificialanalysis.ai/articles/muse-glimmer) calls it Meta's first open-weights release since Llama 4. The card does not compare it with Llama 3.3 and does not name Spanish among its languages.
- **Other makers.** The [Gemma 4 page](https://ollama.com/library/gemma4) in Ollama's library reports 82.6% on MMLU Pro for the 26B; Meta's card gives Llama 3.3 70B 68.9. Different laboratories and conditions: an indication, not a measurement.

We cannot say how Llama 3.3 70B compares with these on your work: we have not run it. We can say that the 26B and 35B models we run have files under 24 GB and are many times faster than the published 70B figure on our class of machine.

Reasons to still choose it: a workflow or fine-tune already validated on it, a need for a text-only model with Spanish on its supported list, or hardware you already own.

## Frequent questions

**Can I run it on one RTX 4090?** Not at a quality worth having. The benchmark repository above records out of memory for Q4_K_M on one 24 GB card, and the only file that fits is rated very low quality.

**Is 64 GB of RAM without a GPU enough?** The sum is the same, so it should load, but everything runs on the CPU. We have no figure for that and would not plan on it.

**Does Q5 or Q8 make it slower?** By the bandwidth argument, yes. In Ollama's Spark table, Q8 files of smaller dense models ran at about two thirds of the Q4 speed. We have no 70B figure.

## Limits

We did not run the model: no speed, loaded size or quality judgement of ours for it. The memory sum is verified on models of 7 to 24 billion parameters, not on this one, and the 1 GB of buffers is an allowance. Published speeds are for Llama 3 and 3.1 70B on older software. We have no Mac, dedicated card or split load of our own.

## Next steps

- Do the full sum for any model with the [VRAM guide](https://vorluxai.com/en/blog/how-much-vram-local-llm-en/).
- See the models we would try first in [best local LLMs by hardware and task](https://vorluxai.com/en/blog/best-local-llms-by-hardware-and-task-en/).
- Read what the largest models we run do in our [DGX Spark review](https://vorluxai.com/en/blog/dgx-spark-review-7-weeks-measured-en/).

## Let's talk for 15 minutes

We do this sum with your real documents and number of users, and test the smaller models first, before recommending any hardware. If you want it for your company, [ask for a 15-minute call](https://vorluxai.com/en/contact/?ref=llama-3-3-70b-local-review-what-it-takes-to-run-en) or see [how we work in consulting](https://vorluxai.com/en/consulting/).
