AI Models

Open models to run on your own machine

53 open-source models selected for local deployment. Each with its use case, the memory it needs and how to install it.

Source: Hugging Face and Ollama. Sizes checked on 6 October 2026.

53 models 8 categories 20 machines compared

Which model fits your machine?

Pick a model and a quantisation to see which machines it fits.

Work out the memory a model needs

Pick a model, a quantisation and a context. The bars show which machines it fits.

Which machines it fits

From least to most memory. Open a machine for its spec sheet and where to buy it.

Jetson Orin Nano Super 8 GB · €230
RPi 5 + AI HAT+ 2 16 GB · €200
RTX 4060 Ti 16GB Build 16 GB · €1,000
Mac Mini M4 (24GB) 24 GB · €920
RTX 3090 24GB (Used) Build 24 GB · €1,200
MacBook Pro M4 Pro (24GB) 24 GB · €2,199
RTX 4090 24GB Build 24 GB · €2,500
Beelink SER8 32 GB · €550
Lenovo ThinkPad P16s Gen 3 (RTX A500) 32 GB · €1,650
Framework 16 (Radeon RX 7700S) 32 GB · €1,900
Mac Mini M4 Pro (48GB) 48 GB · €1,840
Mac Studio M4 Max (48GB) 48 GB · €2,300
AMD W7900 48GB Build 48 GB · €4,000
Supermicro 1U Server (L40S 48GB) 48 GB · €12,000
ASUS NUC 14 Pro 64 GB · €650
Jetson AGX Orin 64GB 64 GB · €1,840
Jetson T4000 (64GB) 64 GB · €1,999
Minisforum UM890 Pro 96 GB · €750
Dell PowerEdge R760xa (2x RTX A6000) 96 GB · €18,500
Jetson AGX Thor (128GB) 128 GB · €3,499

Cloud GPU rental (hourly)

Cloud A100 80GB (RunPod) 80 GB · hourly rental
Cloud H100 80GB (RunPod) 80 GB · hourly rental

Fits with room Tight (over 85% of usable) Does not fit

Tip:

Shall we set it up on your machine?

Book a 15-minute call Choose hardware Work out the cost

Get this analysis by email

Next step: Hardware selector ROI calculator Grants

My hardware:

More filters: provider

To start with

The largest chat and code models that fit in 8 GB (Q4_K_M).

Qwen 3 8B Alibaba · 8B · 5.03 GB ollama pull qwen3:8b Gemma 4 E4B Google · E4B (8B raw, edge) · 4.98 GB ollama pull gemma4:e4b Qwen2.5-Coder 7B Alibaba · 7B · 4.68 GB ollama pull qwen2.5-coder:7b

53 models

Click a model for details

Llama 4 Scout

General / Chat

Meta · 109B (17B active MoE) · 10M ctx · Llama 4 Community

Largest open context window ever (10M tokens). MoE = only 17B params active, so inference is fast despite 109B total.

Massive context (10M tokens)Document analysisMulti-turn reasoning ~8 tok/s

Q4_K_M

65.4 GB > 24 GB

114.5 GB > 24 GB

FP16

215.6 GB > 24 GB

Model details

Recommended hardware

Mac Studio 128GB or DGX Spark 128GB (Q4); FP16 needs a multi-GPU server

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 8 tok/s MoE: 17B active

Install with Ollama

ollama pull llama4:scout

meta-llama/Llama-4-Scout-17B-16E-Instruct

View on Hugging Face ↗ Machines

Llama 4 Maverick

General / Chat

Meta · 400B (17B active, 128 experts) · 1M ctx · Llama 4 Community

128-expert MoE with only 17B active. Competes with GPT-4 class. Requires multi-GPU or cloud.

Top-tier reasoningComplex analysisResearch ~3 tok/s

Q4_K_M

242.8 GB > 24 GB

425.8 GB > 24 GB

FP16

801.5 GB > 24 GB

Model details

Recommended hardware

Multi-GPU cluster (not practical for edge)

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 3 tok/s Requires 200GB+

Install with Ollama

ollama pull llama4:maverick

meta-llama/Llama-4-Maverick-17B-128E-Instruct

View on Hugging Face ↗ Machines

Gemma 3 4B

Vision / Multimodal

Google · 4B · 128K ctx · Gemma License

Smallest multimodal model available. Only 3GB at Q4. Fits Jetson Orin Nano. Perfect for vision tasks on minimal hardware.

Lightweight multimodalImage understandingEdge deployment128K context ~65 tok/s

Q4_K_M

2.49 GB ≤ 8 GB

4.13 GB ≤ 8 GB

FP16

7.77 GB ≤ 8 GB

Model details

Recommended hardware

Jetson Orin Nano 8GB | Mac Mini M4 16GB

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 65 tok/s FP16: 40 tok/s

Install with Ollama

ollama pull gemma3:4b

google/gemma-3-4b-it

View on Hugging Face ↗ Machines

Gemma 3 27B

Vision / Multimodal

Google · 27B (also 1B/4B/12B) · 128K ctx · Gemma License

First multimodal open model that fits on a Mac Mini. 4B variant needs only 3GB. SigLIP vision encoder. 140+ languages.

Multimodal (text + images)Document scanningInvoice processing128K context window ~14 tok/s

Q4_K_M

16.6 GB ≤ 24 GB

28.7 GB > 24 GB

FP16

54.8 GB > 24 GB

Model details

Recommended hardware

4B: any 8GB+ device | 27B: Mac Mini M4 Pro 32GB+

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 14 tok/s FP16: 7 tok/s

Install with Ollama

ollama pull gemma3:27b

google/gemma-3-27b-it

View on Hugging Face ↗ Machines

Gemma 4 26B

General / Chat

Google · 26B (4B active MoE) · 128K ctx · Apache 2.0

Apache 2.0 license (fully open). Best quality-per-param in the 27B class. Runs on Mac Mini M4 24GB at Q4.

High-quality chatInstruction followingCreative writing ~14 tok/s

Q4_K_M

17 GB ≤ 24 GB

26.9 GB > 24 GB

FP16

50.5 GB > 24 GB

Model details

Recommended hardware

Mac Mini M4 24GB (Q4, short context) or a 24GB GPU (Q4)

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 14 tok/s FP16: 7 tok/s

Install with Ollama

ollama pull gemma4:26b

google/gemma-4-26B-A4B-it

View on Hugging Face ↗ Machines

Gemma 4 E4B

General / Chat

Google · E4B (8B raw, edge) · 128K ctx · Apache 2.0

Tiny but punches above its weight. Runs on Jetson Orin Nano. Apache 2.0.

Fast inferenceEdge deploymentMobile/IoT ~58 tok/s

Q4_K_M

4.98 GB ≤ 8 GB

8.19 GB ≤ 16 GB

FP16

15.1 GB ≤ 16 GB

Model details

Recommended hardware

Jetson Orin Nano 8GB (FP16) or any device

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 58 tok/s FP16: 35 tok/s

Install with Ollama

ollama pull gemma4:e4b

google/gemma-4-E4B-it

View on Hugging Face ↗ Machines

Qwen 3 32B

General / Chat

Alibaba · 32B · 131.072K ctx · Apache 2.0

Thinking mode (deep reasoning) + non-thinking mode. Best multilingual open model. Apache 2.0.

Multilingual (29 languages)ReasoningTool use ~12 tok/s

Q4_K_M

19.8 GB ≤ 24 GB

34.8 GB > 24 GB

FP16

65.6 GB > 24 GB

Model details

Recommended hardware

Mac Mini M4 Pro 48GB (Q8) or RTX 4090 24GB (Q4)

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 12 tok/s FP16: 6 tok/s

Install with Ollama

ollama pull qwen3:32b

Qwen/Qwen3-32B

View on Hugging Face ↗ Machines

Qwen 3 8B

General / Chat

Alibaba · 8B · 131.072K ctx · Apache 2.0

Sweet spot for edge: great quality at 8B, dual thinking modes, strong Spanish. Fits on 8GB devices.

General purposeSpanish languageFast reasoning ~45 tok/s

Q4_K_M

5.03 GB ≤ 8 GB

8.71 GB ≤ 16 GB

FP16

16.4 GB ≤ 24 GB

Model details

Recommended hardware

Jetson Orin Nano 8GB (Q4) or Mac Mini M4 16GB (Q8)

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 45 tok/s FP16: 22 tok/s

Install with Ollama

ollama pull qwen3:8b

Qwen/Qwen3-8B

View on Hugging Face ↗ Machines

Phi-4

General / Chat

Microsoft · 14B · 16.384K ctx · MIT

Best-in-class for STEM at 14B. MIT license. Fits 16GB devices at Q4.

STEM reasoningMathCode generation ~28 tok/s

Q4_K_M

9.05 GB ≤ 16 GB

15.6 GB ≤ 16 GB

FP16

29.3 GB > 24 GB

Model details

Recommended hardware

Mac Mini M4 16GB (Q4) or Mac Mini M4 24GB (Q8)

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 28 tok/s FP16: 14 tok/s

Install with Ollama

ollama pull phi4

microsoft/phi-4

View on Hugging Face ↗ Machines

DeepSeek V3

General / Chat

DeepSeek · 671B (37B active MoE) · 128K ctx · DeepSeek License

Only 37B active params via MoE. Cloud-only for most users but API is very cheap ($0.27/1M input).

GPT-4 class reasoningComplex tasksResearch ~5 tok/s

Q4_K_M

404.4 GB > 24 GB

713.3 GB > 24 GB

FP16

1342 GB > 24 GB

Model details

Recommended hardware

Cloud API recommended ($0.27/$1.10 per 1M tokens)

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 5 tok/s 671B MoE, needs multi-GPU

Install with Ollama

ollama pull deepseek-v3

deepseek-ai/DeepSeek-V3

View on Hugging Face ↗ Machines

DeepSeek R1

General / Chat

DeepSeek · 14B distill (full R1: 671B MoE) · 128K ctx · MIT

Best open-source reasoning model. 14B distill fits Mac Mini M4 16GB. Shows explicit thinking process. MIT license.

Chain-of-thought reasoningMath (97.3% MATH-500)Code debuggingLegal/financial analysis ~4 tok/s

Q4_K_M

8.99 GB ≤ 16 GB

15.7 GB ≤ 16 GB

FP16

29.6 GB > 24 GB

Model details

Recommended hardware

14B: Mac Mini M4 16GB | 32B: 32GB+ unified memory

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 4 tok/s 685B MoE, needs multi-GPU

Install with Ollama

ollama pull deepseek-r1:14b

deepseek-ai/DeepSeek-R1

View on Hugging Face ↗ Machines

Qwen3-Coder-Next

Coding

Alibaba · 80B (3B active MoE) · 262.144K ctx · Apache 2.0

Only 3B active params: fast for its size. 256K context for huge codebases. >70% SWE-Bench. Apache 2.0.

Code generation (370 languages)SWE-Bench >70%Large repos ~5 tok/s

Q4_K_M

48.5 GB > 24 GB

84.8 GB > 24 GB

FP16

159.5 GB > 24 GB

Model details

Recommended hardware

Mac Studio 64GB+ or DGX Spark 128GB (Q4)

Indicative speed (M3 Pro 32 GB, not measured by us)

Q4: 5 tok/s Needs 48GB+

Install with Ollama

ollama pull qwen3-coder-next

Qwen/Qwen3-Coder-Next

View on Hugging Face ↗ Machines

Load more models

Tool

AI Model Comparer

Select 2-3 models for a side-by-side comparison.

Reference: the cloud

When you need the cloud

For jobs that need a bigger model than your machine holds, this is what the API costs. We only publish prices we have checked on each provider's official page.

US dollars per million tokens. Verified on 9 September 2026.
Model	Provider	Input	Output	Context
GPT-5	OpenAI	$1.25	$10.00	400K
GPT-5 Mini	OpenAI	$0.25	$2.00	400K
Claude Sonnet 5	Anthropic	$2.00	$10.00	200K
Claude Haiku 4.5	Anthropic	$1.00	$5.00	200K

Sources: OpenAI · Anthropic. To compare with what it costs locally, use the cost calculator.

Cloud or local?

Cloud API

Access to the largest models
No hardware to maintain
Instant scaling
Data leaves your network
Cost grows with use
Provider dependency

Local AI

Data never leaves your network
Fixed cost (hardware once)
GDPR easier to meet
No network latency
Smaller models
Initial investment

Consult local deployment →

Veredicto semanal

Get new guides before anyone else

Subscribe and we tell you when new guides, templates and workflows go up. One email a week, no spam.

Already published: 79 guides and 25 templates. All free, no signup.

Bonus: the EU AI Act checklist, ready to complete

Once a week No spam Unsubscribe anytime

See what you get

The EU AI Act now applies: a checklist you can complete

Tell us what you want to run

Tell us what you want to run and on what budget. We will tell you which hardware you need, which model fits, and what to expect from it, before you spend anything.

Let's talk for 15 minutes WhatsApp

First call free, 15 min Local-first: your data stays on your network Open tools and guides

79 free guides · 17 compliance templates

Open models to run on your own machine

Which model fits your machine?

Work out the memory a model needs

Which machines it fits

Cloud GPU rental (hourly)

Shall we set it up on your machine?

To start with

Llama 4 Scout

Llama 4 Maverick

Gemma 3 4B

Gemma 3 27B

Gemma 4 26B

Gemma 4 E4B

Qwen 3 32B

Qwen 3 8B

Phi-4

DeepSeek V3

DeepSeek R1

Qwen3-Coder-Next

AI Model Comparer

Comparison

Verdict

When you need the cloud

Cloud or local?

Cloud API

Local AI

Get new guides before anyone else

Tell us what you want to run

VORLUX AI

Open models to run on your own machine

Which model fits your machine?

Work out the memory a model needs

Which machines it fits

Cloud GPU rental (hourly)

Shall we set it up on your machine?

Local model catalogue

To start with

Llama 4 Scout

Llama 4 Maverick

Gemma 3 4B

Gemma 3 27B

Gemma 4 26B

Gemma 4 E4B

Qwen 3 32B

Qwen 3 8B

Phi-4

DeepSeek V3

DeepSeek R1

Qwen3-Coder-Next

AI Model Comparer

Comparison

Verdict

When you need the cloud

Cloud or local?

Cloud API

Local AI

Get new guides before anyone else

Tell us what you want to run

VORLUX AI