# Ollama Structured Output with a JSON Schema: 40 Invoices Measured

> We pulled 9 fields from 40 synthetic Spanish invoices with 6 local models. A JSON schema made every reply clean JSON; the model decided accuracy.

- Published: 2026-11-12
- Author: Jacobo González Jaspe
- Language: en
- Tags: Ollama, Structured output, Benchmarks, Invoices, Local AI
- Canonical URL: https://vorluxai.com/en/blog/ollama-structured-output-invoices-measured-en/
- Source: VORLUX AI (J4SGON S.L.), https://vorluxai.com/

---

By the end of this post you will get JSON out of a local model that your code can parse every time, and you will know how to check whether the values inside it are right. The 7B files we tested loaded in under 5 GB at Q4, so a 16 GB laptop is enough; our numbers come from a DGX Spark. We tested six models and files on 40 synthetic Spanish invoices to see where each one breaks.

> **TL;DR**
> - You get: field-by-field accuracy for six local models on 40 invoices, three ways of asking, and the scoring code.
> - It costs nothing: one `format` field in the request.
> - Verdict: always send a schema. It turned every reply into clean JSON. Accuracy came from the model: gemma4:26b got 360 of 360 fields, the 7B and 8B models 321 to 344, and every one of those smaller models stumbled on who issued an invoice sent by e-mail.

## The test

We wrote a generator, not a model, to produce 40 fictitious Spanish invoices from a fixed seed. Ten are formal invoices, ten are e-mails that announce one, ten are till receipts and ten are messy layouts with stray spaces and "N.I.F.", the way a bad scan reads. Every document names the issuer and a client, each with its own tax ID.

Amounts come in Spanish formats ("1.234,56 €", "1 234,56 EUR") and dates in three styles. The model must return nine fields: invoice number, date, issuer name, issuer tax ID, client tax ID, base, VAT rate, VAT amount and total. The right answers come from the same generator, so no model wrote any of them. The companies are invented.

These are synthetic invoices, not real ones. That makes the right answers certain, but it also means we did not measure accuracy on real invoices. Expect your own documents to score differently, which is why Step 2 has you score ten of them.

We asked three ways, with the same Spanish instruction and temperature 0: a plain prompt, `format: "json"`, and `format` set to a JSON schema.

## What you need

- Ollama 0.5 or newer. Schemas arrived in December 2024 ([Ollama's announcement](https://ollama.com/blog/structured-outputs)). Our [install guide](https://vorluxai.com/en/blog/install-ollama-first-local-llm-en/) covers a laptop.
- A 7B or 8B model; `qwen2.5:7b` loaded in 4.8 GB with a 4k context in our run.
- Ten real documents of your own and the right answers for them, written by a person. Without answers you only measure how confident the model sounds.
- Python 3 with `requests`.

## Step 1: write the schema

Name every field, give it a type and mark it required. Use `number` for money: a number field cannot hold a euro sign or a Spanish decimal comma.

```json
{
  "type": "object",
  "properties": {
    "invoice_number": {"type": "string"},
    "issue_date":     {"type": "string", "description": "YYYY-MM-DD"},
    "issuer_name":    {"type": "string"},
    "issuer_tax_id":  {"type": "string"},
    "client_tax_id":  {"type": "string"},
    "base_amount":    {"type": "number"},
    "vat_rate":       {"type": "number"},
    "vat_amount":     {"type": "number"},
    "total_amount":   {"type": "number"}
  },
  "required": ["invoice_number", "issue_date", "issuer_name", "issuer_tax_id",
               "client_tax_id", "base_amount", "vat_rate", "vat_amount", "total_amount"]
}
```

Save it as `schema.json`. Keep the field list in your prompt as well: the schema fixes the shape, the prompt says what each field means.

## Step 2: send it and score it

```python
import json, requests

SCHEMA = json.load(open("schema.json"))
PROMPT = ("Extract the invoice fields as JSON. issue_date as YYYY-MM-DD, amounts as numbers "
          "with a decimal point, tax IDs without spaces or dashes.\n\nDocument:\n")

def ok(field, got, gold):
    if field.endswith("_amount") or field == "vat_rate":
        return isinstance(got, (int, float)) and abs(got - gold) <= 0.01
    if field.endswith("tax_id"):
        return str(got).replace("-", "").replace(" ", "").upper() == gold
    return str(got).strip().casefold() == str(gold).strip().casefold()

docs = json.load(open("my-invoices.json"))   # [{"text": "...", "gold": {...}}, ...]
right = total = 0
for d in docs:
    r = requests.post("http://localhost:11434/api/chat", json={
        "model": "qwen2.5:7b", "stream": False, "format": SCHEMA,
        "options": {"temperature": 0, "num_ctx": 4096},
        "messages": [{"role": "user", "content": PROMPT + d["text"]}]},
        timeout=300).json()
    got = json.loads(r["message"]["content"])
    right += sum(ok(k, got.get(k), v) for k, v in d["gold"].items())
    total += len(d["gold"])
print(f"{right}/{total} fields correct")
```

Expected output: one line such as `329/360 fields correct`, which is what Q4_K_M scored on our set with the bench's Spanish prompt; this shorter prompt can score a few fields either way. If `json.loads` fails, run `ollama --version`: before 0.5 the schema in `format` is ignored. Our 40 invoices with their answers are public at [structured-extraction-es-invoices-2026-11-05.json](https://vorluxai.com/data/measurements/structured-extraction-es-invoices-2026-11-05.json); load them with `json.load(open(...))["docs"]` to run your model on the same set first.

## What we measured

Measured 2026-11-05 on our first DGX Spark (`spark-43d5`, NVIDIA GB10, 128 GB unified memory), Ollama 0.30.10, one model loaded at a time with `num_ctx` 4096 and seed 42. Raw file: [structured-extraction-2026-11-05.json](https://vorluxai.com/data/measurements/structured-extraction-2026-11-05.json).

```bash
python3 scripts/bench/run_for_topic.py --host prime --slug structured-extraction \
  --bench "structured_extraction.py --models llama3.1:8b,qwen2.5-coder:7b,qwen2.5:7b-instruct-q4_K_M,qwen2.5:7b-instruct-q8_0,qwen2.5:7b-instruct-fp16,gemma4:26b"
```

**Schema mode, 40 invoices, 360 fields:**

| Model | File | Loaded | Fields right | Invoices fully right | E-mails fully right | Seconds per invoice | tok/s |
|---|---|---|---|---|---|---|---|
| gemma4:26b (MoE) | 18.0 GB | 17.2 GB | 360 | 40 | 10 of 10 | 3.07 | 62.3 |
| qwen2.5:7b Q8_0 | 8.1 GB | 7.9 GB | 344 | 34 | 5 of 10 | 4.96 | 27.9 |
| qwen2.5:7b FP16 | 15.2 GB | 14.5 GB | 344 | 34 | 5 of 10 | 8.36 | 16.3 |
| qwen2.5-coder:7b | 4.7 GB | 4.8 GB | 339 | 33 | 4 of 10 | 3.46 | 41.0 |
| qwen2.5:7b Q4_K_M | 4.7 GB | 4.8 GB | 329 | 26 | 2 of 10 | 3.37 | 42.1 |
| llama3.1:8b | 4.9 GB | 5.3 GB | 321 | 22 | 1 of 10 | 2.88 | 40.6 |

**The three ways of asking** (fields right of 360, and replies that were bare JSON your code can load directly):

| Model | Plain prompt | `format: "json"` | `format: schema` | Bare JSON: prompt / json / schema |
|---|---|---|---|---|
| gemma4:26b | 360 | 360 | 360 | 0 / 40 / 40 |
| qwen2.5:7b Q8_0 | 341 | 344 | 344 | 5 / 40 / 40 |
| qwen2.5:7b FP16 | 341 | 344 | 344 | 3 / 40 / 40 |
| qwen2.5-coder:7b | 344 | 339 | 339 | 0 / 40 / 40 |
| qwen2.5:7b Q4_K_M | 332 | 329 | 329 | 6 / 40 / 40 |
| llama3.1:8b | 328 | 321 | 321 | 0 / 40 / 40 |

Without `format`, 226 of 240 replies came wrapped in a code fence or a sentence ("Here is the JSON..."). Every one parsed once we cut the fence out, but that is code you have to write and maintain. With `format: "json"`, llama3.1:8b once returned all four numbers as strings; the schema run on the same invoice returned numbers. Changing the mode moved accuracy by 7 fields or fewer of 360.

## What the spec sheet won't tell you

- **Who issued it is the hard field.** In the e-mails, the client's details sit in the body and the issuer signs at the bottom. The 7B and 8B models got the issuer wrong in 5 to 9 of 10 e-mails, every time by naming the client instead. gemma4:26b never did.
- **Amounts were the easy part.** All three Qwen2.5 7B Instruct files got base, VAT and total right on all 40 invoices in schema mode. The errors were names, tax IDs and one date.
- **Q4 drops letters from tax IDs.** The Q4_K_M file returned a tax ID without its letter on six invoices where Q8_0 kept it; five of those were written in spaced groups, like "A 09 014 170" coming back as "09014170". Q8_0 got 8 invoices fully right that Q4_K_M missed, and none the other way round.
- **"12/06/2026" became December.** All three Qwen2.5 7B Instruct files read one till receipt's date as month first. Ask for ISO dates and check the ones that land in the future.
- **The schema did not slow it down here.** Schema replies were shorter than prompt replies (no fence, no preamble), so the seconds per invoice were lower for all six. llama3.1:8b was already loaded for another job on the box when our run started, so its timings may have been disturbed and are the least reliable in the table.

## Verdict

- **Schema: buy (it is free).** Send `format` with a schema on every extraction call. It is one field.
- **Model: measure first.** On these synthetic invoices, gemma4:26b (17.2 GB loaded, so plan on 24 GB or more) was right on every field and took 3.07 s per invoice, faster than any Qwen2.5 7B file. On a 16 GB machine, Qwen2.5 7B at Q8_0 scored best of what we tested that fits, with a person reviewing issuer and tax IDs.
- **Skip** extraction straight into your books from a 7B model with no review step. Even the best 7B file in our test got the issuer wrong on half the e-mailed invoices.

## When not to do this

If your invoices arrive as scans or photos, you need text first: OCR or a vision model, which this test does not cover. We measured that separately with these same 40 invoices turned into images: read scanned invoices with a local vision model. If you receive structured e-invoices (Facturae XML, or the formats your invoicing software exports), parse the file; a language model adds errors to data that is already exact. And if you process a dozen invoices a month, typing them is faster than building and checking any of this.

## Next steps

- See what Q4_K_M loses against Q8_0 and FP16 on these same invoices: [Q4_K_M vs Q8_0 vs FP16, measured](https://vorluxai.com/en/blog/q4-vs-q8-vs-fp16-measured-en/).
- Choose the encoding with your memory in mind: [GGUF quantization: pick the right file](https://vorluxai.com/en/blog/gguf-quantization-pick-the-right-file-for-your-hardware-en/).
- Set `num_ctx` before long documents fill it: [Ollama num_ctx memory, measured](https://vorluxai.com/en/blog/ollama-context-length-memory-measured-en/).
- Feed the extracted fields into search over your own documents: [local RAG with Qdrant and bge-m3](https://vorluxai.com/en/blog/local-rag-qdrant-bge-m3-measured-en/).
- Before real invoices go through it, read the DPIA we filled in for this same extraction.

## Work with us

We size the model and the machine by measuring, not by guessing. If you want to see your own documents going through a local model on real hardware, [book a free 15-minute call](https://vorluxai.com/en/contact/) or [see how we work in consulting](https://vorluxai.com/en/consulting/).
