View all articles
DGX SparkHardwareOllamaBenchmarksLocal AI

NVIDIA DGX Spark After 7 Weeks: What 128 GB Runs, Measured

JG
Jacobo González Jaspe
|

Reviewed:

Abstract illustration: two paths of light diverging; one faint white, one bright amber. Merece la pena
Illustration generated with AI on our own machine.

By the end of this post you will know what an NVIDIA DGX Spark does with real models, how fast, with how much memory left over, and whether you are the person it was built for. Most readers are not, and the cheapest answer is the laptop you already own or a far cheaper desktop. We say that first because we run two of these machines and like them.

What you need

To reproduce our table you need no Spark at all. Any machine running Ollama works:

  • Ollama installed (we ran 0.30.10 on the first box and 0.34.0 on the second).
  • Our benchmark script, scripts/bench/local_model_benchmark.py, explained step by step in Benchmark a local model in 30 minutes.
  • About 30 minutes, most of it downloads.

Step 1: what you are buying

The DGX Spark is a small desktop box built around NVIDIA’s GB10 Grace Blackwell chip: a 20-core Arm CPU (10 Cortex-X925 and 10 Cortex-A725) and a Blackwell GPU sharing 128 GB of LPDDR5x memory on a 256-bit bus at 273 GB/s. It takes up to 4 TB of NVMe, has a 10 GbE port and ConnectX networking that links up to four units. NVIDIA quotes “up to 1 PFLOP FP4”; we did not measure that and this post does not use it. All of this is from the NVIDIA specification page, read on 2026-11-02.

Two numbers on that sheet matter for language models, and neither is the petaflop. The first is 128 GB: that is how big a model can be. The second is 273 GB/s: that is how fast one user gets tokens from a model that fills it. Keep both in mind for the table below.

Price: in Spain, the NVIDIA Marketplace listed the Founders Edition at EUR 7,490 on 16 November 2026; the link is in our hardware catalogue. Prices move; check the retailer the day you buy.

Step 2: how we run two of them

J4SGON S.L. runs on two units since 2026-09-15. Box one (we call it PRIME) runs the desktop, the build of this website, about forty Docker containers, three Ollama instances and our agents. Box two is headless and serves the bigger models: vLLM, the 17 GB and 23 GB Ollama models, and a vision model.

They are joined by one QSFP cable on the ConnectX ports. On 2026-09-15 we measured the link with iperf3 -c <second-box> -P 8 -f g: 107 Gbit/s with eight streams, 28 Gbit/s with one, 0.26 ms latency. A 40 GB model file crosses in about three seconds. For our purposes, a model on the second box is a local model.

Step 3: reproduce the table

From the first box:

bash
python3 scripts/bench/local_model_benchmark.py \
  --models llama3.1:8b,qwen2.5-coder:7b,deepseek-r1:14b,qwen2.5vl:7b,gemma4:26b \
  --runs 3 --max-tokens 200 --think false \
  --out docs/measurements/dgx-spark-review-prime-2026-11-02.json

Against the second box, from the first:

bash
python3 scripts/bench/local_model_benchmark.py \
  --ollama http://<second-box>:11434 \
  --models qwen3.6:35b,gemma4:26b \
  --runs 3 --max-tokens 200 --think false --max-model-gb 25 \
  --out docs/measurements/dgx-spark-review-spark2-2026-11-02.json

The script refuses to load a file over 20 GB by default. That limit is deliberate, and point 1 below explains why we raise it only on the box without a desktop.

What we measured

Measured 2026-11-02 on our two NVIDIA DGX Sparks (GB10, 128 GB unified memory). Same 278-word prompt (a meeting note to summarise) for every model, temperature 0, fixed seed, 200 output tokens, thinking off. Run 1 is a cold load and is discarded; run 2 is reported. Raw results, cold runs included, are in dgx-spark-review-prime-2026-11-02.json and dgx-spark-review-spark2-2026-11-02.json.

Model (Ollama tag, all Q4_K_M)Box, contextLoaded sizeGeneration tok/sFirst tokenCold load from disk
llama3.1:8b (dense)PRIME, 32k9.2 GB43.70.39 s3.8 s
qwen2.5-coder:7b (dense)PRIME, 32k6.6 GB46.60.31 s3.8 s
qwen2.5vl:7b (vision, dense)PRIME, 32k6.6 GB43.30.31 s15.8 s
deepseek-r1:14b (dense)PRIME, 32k15.3 GB22.40.43 s20.1 s
gemma4:26b (MoE, 128 experts, 8 per token)PRIME, 32k17.5 GB61.10.69 s61.9 s
gemma4:26b (same file)spark2, 16k22.6 GB68.20.21 s5.9 s
qwen3.6:35b (MoE, 256 experts, 8 per token)spark2, 16k23.1 GB75.80.24 s4.6 s

Expert counts come from Ollama’s /api/show (expert_count, expert_used_count), read the same day.

Three things in that table are worth a second look.

The fastest models are the biggest ones. The 35B and the 26B both beat the 8B by a wide margin. Both are mixture-of-experts models: each token touches 8 experts out of 256 or 128, so the GPU reads a fraction of the weights per token. The dense 14B, which reads all of its weights every time, is the slowest row. Parameter count tells you how much memory you need. It does not tell you how fast you will go.

Same file, two sizes. gemma4:26b occupies 17.5 GB on the first box at a 32k context and 22.6 GB on the second at 16k. The second box runs Ollama 0.34, which sizes the cache and the parallel slots differently. Read ollama ps after every upgrade, not the file size on the download page.

Cold loads depend on the disk cache, not the model. gemma4 took 62 seconds to load on PRIME, where the file had been evicted from memory, and 6 seconds on spark2, where it was still cached. Do not benchmark a first run.

The 70B class

We did not load a dense 70B today; the box that runs our desktop keeps a 20 GB limit (point 1 below). Two dated figures stand in. Our own run of qwen2.5vl:72b (Q4_K_M, 49 GB resident, 8k context) gave 3.2 tok/s cold and 2.7 tok/s on the second run on PRIME, measured 2026-09-09 with other services sharing the box, before we kept a result file for every run, so this one has no receipt; the detail is in our Qwen2.5-72B review. The Ollama blog (23 October 2025, not our run) reports llama3.1:70b at 4.4 tok/s decode and 1,911 tok/s prefill on a DGX Spark.

The arithmetic agrees with both. Every generated token reads every weight once. At 273 GB/s, a model whose weights occupy 43 GB cannot exceed about 6 tokens per second (273 divided by 43) for a single user. Memory bandwidth, not FLOPS, is the ceiling for one person talking to one dense model. Our Llama 3.3 70B review explains which tasks justify that speed anyway.

What the spec sheet won’t tell you

  1. Unified memory has no safety net for the GPU. On a PC with a graphics card, a model that does not fit fails to load with an error. Here the GPU, the desktop and every container share the same 128 GB. When it runs out, the symptom is a frozen desktop and NV_ERR_NO_MEMORY in the driver log. After one freeze too many we set two rules: keep 16 GB free on the box that runs the desktop, and never load a file over 20 GB on it. Those go to the second unit.
  2. Bandwidth caps single-user speed. The machine is at its best with models between 17 and 60 GB and with several people asking at once, not with one person waiting on a dense 70B.
  3. Ollama’s defaults move between versions. Version 0.34 sized a model’s cache for its full context length (262,144 tokens for qwen3.6) times the number of parallel slots. On the second box that made loads fail until we pinned OLLAMA_CONTEXT_LENGTH. Check ollama ps after each upgrade.
  4. It is Arm (aarch64). Almost everything runs. Some Python wheels and CUDA 13 builds are not published for it and have to be compiled. Budget an afternoon the first time.
  5. Mixture-of-experts models are the sweet spot. Two of them generate faster than an 8B dense model while using 17 to 23 GB. That is the shape of model this memory is for.
  6. Power draw and noise. We have not put a wall meter on it yet, so this post carries no watts and no decibels. It sits on a desk and we hear the room, not the box, which is an impression, not a measurement. We will publish the reading when we have it.

Verdict

  • Buy. You need CUDA and more than 32 GB for models, you serve several simultaneous users, you fine-tune LoRA adapters, or you want two boxes pooled over 200 GbE. Nothing else at this size does all four.
  • Wait. Your model list fits in 24 to 32 GB. A used 24 GB graphics card in a PC you already own, or a Mac with 64 GB, covers it for less. The hardware catalogue names them with prices.
  • Skip. You are one person chatting with 7B to 14B models. The laptop you have does that today; in the catalogue, a 16 GB Mac mini M6 was EUR 1,069 (Apple, 16 November 2026) and a 14B is a tight fit.

When not to buy this

If your big-model work is one long document a month, rent the cloud for that hour and keep a small model local for everything else. If your models fit in the memory you already own, buying more memory makes nothing faster: an 8B model fits on a far cheaper box. Buy the Spark for the models and the people that only it can hold.

Next steps

Work with us

We size the model and the machine by measuring, not by guessing. If you want to see your own task running on real hardware, book a 15-minute call or see how we work in consulting.

Diagram
Share: LinkedIn X
Veredicto semanal

Get new guides before anyone else

Subscribe and we tell you when new guides, templates and workflows go up. One email a week, no spam.

Already published: 102 guides and 25 templates. All free, no signup.

Bonus: the local-AI starter pack PDF when you subscribe
Once a week No spam Unsubscribe anytime

See what you get