By the end of this post you will get JSON out of a local model that your code can parse every time, and you will know how to check whether the values inside it are right. The 7B files we tested loaded in under 5 GB at Q4, so a 16 GB laptop is enough; our numbers come from a DGX Spark. We tested six models and files on 40 synthetic Spanish invoices to see where each one breaks.
The test
We wrote a generator, not a model, to produce 40 fictitious Spanish invoices from a fixed seed. Ten are formal invoices, ten are e-mails that announce one, ten are till receipts and ten are messy layouts with stray spaces and “N.I.F.”, the way a bad scan reads. Every document names the issuer and a client, each with its own tax ID.
Amounts come in Spanish formats (“1.234,56 €”, “1 234,56 EUR”) and dates in three styles. The model must return nine fields: invoice number, date, issuer name, issuer tax ID, client tax ID, base, VAT rate, VAT amount and total. The right answers come from the same generator, so no model wrote any of them. The companies are invented.
These are synthetic invoices, not real ones. That makes the right answers certain, but it also means we did not measure accuracy on real invoices. Expect your own documents to score differently, which is why Step 2 has you score ten of them.
We asked three ways, with the same Spanish instruction and temperature 0: a plain prompt, format: "json", and format set to a JSON schema.
What you need
- Ollama 0.5 or newer. Schemas arrived in December 2024 (Ollama’s announcement). Our install guide covers a laptop.
- A 7B or 8B model;
qwen2.5:7bloaded in 4.8 GB with a 4k context in our run. - Ten real documents of your own and the right answers for them, written by a person. Without answers you only measure how confident the model sounds.
- Python 3 with
requests.
Step 1: write the schema
Name every field, give it a type and mark it required. Use number for money: a number field cannot hold a euro sign or a Spanish decimal comma.
{
"type": "object",
"properties": {
"invoice_number": {"type": "string"},
"issue_date": {"type": "string", "description": "YYYY-MM-DD"},
"issuer_name": {"type": "string"},
"issuer_tax_id": {"type": "string"},
"client_tax_id": {"type": "string"},
"base_amount": {"type": "number"},
"vat_rate": {"type": "number"},
"vat_amount": {"type": "number"},
"total_amount": {"type": "number"}
},
"required": ["invoice_number", "issue_date", "issuer_name", "issuer_tax_id",
"client_tax_id", "base_amount", "vat_rate", "vat_amount", "total_amount"]
}Save it as schema.json. Keep the field list in your prompt as well: the schema fixes the shape, the prompt says what each field means.
Step 2: send it and score it
import json, requests
SCHEMA = json.load(open("schema.json"))
PROMPT = ("Extract the invoice fields as JSON. issue_date as YYYY-MM-DD, amounts as numbers "
"with a decimal point, tax IDs without spaces or dashes.\n\nDocument:\n")
def ok(field, got, gold):
if field.endswith("_amount") or field == "vat_rate":
return isinstance(got, (int, float)) and abs(got - gold) <= 0.01
if field.endswith("tax_id"):
return str(got).replace("-", "").replace(" ", "").upper() == gold
return str(got).strip().casefold() == str(gold).strip().casefold()
docs = json.load(open("my-invoices.json")) # [{"text": "...", "gold": {...}}, ...]
right = total = 0
for d in docs:
r = requests.post("http://localhost:11434/api/chat", json={
"model": "qwen2.5:7b", "stream": False, "format": SCHEMA,
"options": {"temperature": 0, "num_ctx": 4096},
"messages": [{"role": "user", "content": PROMPT + d["text"]}]},
timeout=300).json()
got = json.loads(r["message"]["content"])
right += sum(ok(k, got.get(k), v) for k, v in d["gold"].items())
total += len(d["gold"])
print(f"{right}/{total} fields correct")Expected output: one line such as 329/360 fields correct, which is what Q4_K_M scored on our set with the bench’s Spanish prompt; this shorter prompt can score a few fields either way. If json.loads fails, run ollama --version: before 0.5 the schema in format is ignored. Our 40 invoices with their answers are public at structured-extraction-es-invoices-2026-11-05.json; load them with json.load(open(...))["docs"] to run your model on the same set first.
What we measured
Measured 2026-11-05 on our first DGX Spark (spark-43d5, NVIDIA GB10, 128 GB unified memory), Ollama 0.30.10, one model loaded at a time with num_ctx 4096 and seed 42. Raw file: structured-extraction-2026-11-05.json.
python3 scripts/bench/run_for_topic.py --host prime --slug structured-extraction \
--bench "structured_extraction.py --models llama3.1:8b,qwen2.5-coder:7b,qwen2.5:7b-instruct-q4_K_M,qwen2.5:7b-instruct-q8_0,qwen2.5:7b-instruct-fp16,gemma4:26b"Schema mode, 40 invoices, 360 fields:
| Model | File | Loaded | Fields right | Invoices fully right | E-mails fully right | Seconds per invoice | tok/s |
|---|---|---|---|---|---|---|---|
| gemma4:26b (MoE) | 18.0 GB | 17.2 GB | 360 | 40 | 10 of 10 | 3.07 | 62.3 |
| qwen2.5:7b Q8_0 | 8.1 GB | 7.9 GB | 344 | 34 | 5 of 10 | 4.96 | 27.9 |
| qwen2.5:7b FP16 | 15.2 GB | 14.5 GB | 344 | 34 | 5 of 10 | 8.36 | 16.3 |
| qwen2.5-coder:7b | 4.7 GB | 4.8 GB | 339 | 33 | 4 of 10 | 3.46 | 41.0 |
| qwen2.5:7b Q4_K_M | 4.7 GB | 4.8 GB | 329 | 26 | 2 of 10 | 3.37 | 42.1 |
| llama3.1:8b | 4.9 GB | 5.3 GB | 321 | 22 | 1 of 10 | 2.88 | 40.6 |
The three ways of asking (fields right of 360, and replies that were bare JSON your code can load directly):
| Model | Plain prompt | format: "json" | format: schema | Bare JSON: prompt / json / schema |
|---|---|---|---|---|
| gemma4:26b | 360 | 360 | 360 | 0 / 40 / 40 |
| qwen2.5:7b Q8_0 | 341 | 344 | 344 | 5 / 40 / 40 |
| qwen2.5:7b FP16 | 341 | 344 | 344 | 3 / 40 / 40 |
| qwen2.5-coder:7b | 344 | 339 | 339 | 0 / 40 / 40 |
| qwen2.5:7b Q4_K_M | 332 | 329 | 329 | 6 / 40 / 40 |
| llama3.1:8b | 328 | 321 | 321 | 0 / 40 / 40 |
Without format, 226 of 240 replies came wrapped in a code fence or a sentence (“Here is the JSON…”). Every one parsed once we cut the fence out, but that is code you have to write and maintain. With format: "json", llama3.1:8b once returned all four numbers as strings; the schema run on the same invoice returned numbers. Changing the mode moved accuracy by 7 fields or fewer of 360.
What the spec sheet won’t tell you
- Who issued it is the hard field. In the e-mails, the client’s details sit in the body and the issuer signs at the bottom. The 7B and 8B models got the issuer wrong in 5 to 9 of 10 e-mails, every time by naming the client instead. gemma4:26b never did.
- Amounts were the easy part. All three Qwen2.5 7B Instruct files got base, VAT and total right on all 40 invoices in schema mode. The errors were names, tax IDs and one date.
- Q4 drops letters from tax IDs. The Q4_K_M file returned a tax ID without its letter on six invoices where Q8_0 kept it; five of those were written in spaced groups, like “A 09 014 170” coming back as “09014170”. Q8_0 got 8 invoices fully right that Q4_K_M missed, and none the other way round.
- “12/06/2026” became December. All three Qwen2.5 7B Instruct files read one till receipt’s date as month first. Ask for ISO dates and check the ones that land in the future.
- The schema did not slow it down here. Schema replies were shorter than prompt replies (no fence, no preamble), so the seconds per invoice were lower for all six. llama3.1:8b was already loaded for another job on the box when our run started, so its timings may have been disturbed and are the least reliable in the table.
Verdict
- Schema: buy (it is free). Send
formatwith a schema on every extraction call. It is one field. - Model: measure first. On these synthetic invoices, gemma4:26b (17.2 GB loaded, so plan on 24 GB or more) was right on every field and took 3.07 s per invoice, faster than any Qwen2.5 7B file. On a 16 GB machine, Qwen2.5 7B at Q8_0 scored best of what we tested that fits, with a person reviewing issuer and tax IDs.
- Skip extraction straight into your books from a 7B model with no review step. Even the best 7B file in our test got the issuer wrong on half the e-mailed invoices.
When not to do this
If your invoices arrive as scans or photos, you need text first: OCR or a vision model, which this test does not cover. We measured that separately with these same 40 invoices turned into images: read scanned invoices with a local vision model. If you receive structured e-invoices (Facturae XML, or the formats your invoicing software exports), parse the file; a language model adds errors to data that is already exact. And if you process a dozen invoices a month, typing them is faster than building and checking any of this.
Next steps
- See what Q4_K_M loses against Q8_0 and FP16 on these same invoices: Q4_K_M vs Q8_0 vs FP16, measured.
- Choose the encoding with your memory in mind: GGUF quantization: pick the right file.
- Set
num_ctxbefore long documents fill it: Ollama num_ctx memory, measured. - Feed the extracted fields into search over your own documents: local RAG with Qdrant and bge-m3.
- Before real invoices go through it, read the DPIA we filled in for this same extraction.
Work with us
We size the model and the machine by measuring, not by guessing. If you want to see your own documents going through a local model on real hardware, book a free 15-minute call or see how we work in consulting.