Run llama.cpp on an Intel Arc GPU with SYCL: 16 GB of VRAM for the Price of a Used 3060

By the end of this post you will have llama.cpp compiled against Intel’s oneAPI toolchain, an Intel Arc card listed as a SYCL device, and a quantized model answering prompts on it. The cheapest hardware this works on: a second-hand Arc A770 16 GB or a new Arc B580 12 GB, both priced around what a used RTX 3060 costs.

One honest note before we start. We do not own an Intel Arc GPU. Every speed in this post is a third-party measurement with a link, and we say where each one comes from. The commands are taken from the official llama.cpp SYCL documentation as it read on 2026-09-09.

Why an Arc card is worth a look

Memory is what decides which models you can run, and Intel sells it cheaply. The Arc A770 has 16 GB of GDDR6 at 560 GB/s and sells for roughly USD 250 to 300. The Arc B580 has 12 GB at about USD 249. For comparison, a used RTX 3060 12 GB trades at USD 250 to 278 in August 2026.

That 16 GB is enough for an 8B model at Q8_0 (about 8 GB), a 14B model at Q4_K_M (about 9 GB), or an 8B model with a long context. The catch is software: CUDA is more mature, and Ollama has no native Arc backend. llama.cpp with SYCL is the direct route, and it is not hard.

What you need

• A PC with an Intel Arc A-series or B-series card, or a 13th-gen-or-newer Intel CPU with a built-in Arc GPU. The docs warn that an iGPU with fewer than 80 execution units “will likely be too slow for practical use”.
• Resizable BAR enabled in the UEFI. Third-party guides call this mandatory for Arc performance. The A770 also needs two 8-pin PCIe power connectors.
• Linux (Ubuntu 22.04 or 24.04 are the most-tested paths) with the Intel GPU driver installed and your user in the render and video groups. Windows works too; the docs cover it.
• About 20 GB of free disk for oneAPI, the build and one model. About 45 minutes the first time.
• A GGUF model, for example Llama-3.1-8B-Instruct-Q4_K_M.gguf from Hugging Face.

Step 1: Install oneAPI and check the GPU is visible

Install Intel’s oneAPI Base Toolkit, or the smaller Intel Deep Learning Essentials package, keeping the default /opt/intel/oneapi path. The docs list 2025.3.3, 2025.2.1, 2025.1 and 2024.1 as verified releases. Then load the environment and list SYCL devices:

source /opt/intel/oneapi/setvars.sh
sycl-ls

You want a line containing [level_zero:gpu] with your Arc card. If it is missing, add your user to the render and video groups and log out and back in. If libsycl.so is reported missing later, you forgot to run setvars.sh in that shell.

Step 2: Build llama.cpp with the SYCL backend

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_SYCL=ON -DCMAKE_C_COMPILER=icx \
  -DCMAKE_CXX_COMPILER=icpx -DGGML_SYCL_F16=ON
cmake --build build --config Release -j -v

GGML_SYCL_F16=ON is the recommended FP16 build. There is also a convenience script, ./examples/sycl/build.sh, that runs the same steps. Expected result: binaries in build/bin/, including llama-ls-sycl-device, llama-cli, llama-completion and llama-server.

Step 3: Confirm llama.cpp sees the card and run a model

./build/bin/llama-ls-sycl-device
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-completion \
  -m models/Llama-3.1-8B-Instruct-Q4_K_M.gguf \
  -ngl 99 -sm none -mg 0 --load-mode auto \
  -p "Write three subject lines for a quote follow-up email."

-ngl 99 offloads every layer to the GPU. -sm none -mg 0 pins a single GPU. ZES_ENABLE_SYSMAN=1 lets llama.cpp query free memory on the device. If you have two Arc cards, replace -sm none -mg 0 with -sm layer to split layers across them. To pick a specific device, set ONEAPI_DEVICE_SELECTOR="level_zero:0". Flash attention is on by default (GGML_SYCL_ENABLE_FLASH_ATTN=1).

For a persistent local API, swap llama-completion for llama-server with the same -m and -ngl flags; it listens on port 8080 with an OpenAI-compatible endpoint.

Step 4: Fix the two errors everyone hits

Symptom	What to do
UR_RESULT_ERROR_OUT_OF_DEVICE_MEMORY	Shorten the context with -c 8192, or use a smaller quant (Q4 instead of Q5).
“can’t allocate 5GB” on Level Zero	Run with UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1.
No [level_zero:gpu] in sycl-ls	Check render/video group membership; log out and in.
libsycl.so not found	source /opt/intel/oneapi/setvars.sh in this shell.
Slow first start	Expected: the SYCL backend compiles kernels just-in-time. Runtime speed is unaffected.

All five come straight from the “Known Issues” section of the official document.

What third parties measured

None of these are our numbers. All use llama.cpp’s llama-bench, tg128 is generation speed over 128 tokens, pp512 is prompt processing over 512 tokens.

GPU	Model	Generation (tg128)	Prompt (pp512)	Source, date
Arc A770 16 GB	llama-2-7b Q4_0	55.5 tok/s (64.5 with flash attention)	894 tok/s	llama.cpp discussion #23313, 2026-05-22
Arc B580 12 GB	llama-2-7b Q4_0	73.8 tok/s	2,064 tok/s	same thread, 2026-05-27
Arc Pro B60	llama-2-7b Q4_0	84.4 tok/s	1,869 tok/s	same thread, 2026-08-27
Arc Pro B70	llama-2-7b Q4_0	109.7 tok/s	2,830 tok/s	same thread, 2026-08-26
Lunar Lake iGPU	llama-2-7b Q4_0	19.2 tok/s	652 tok/s	same thread, 2026-06-02
Arc B580 12 GB (Vulkan backend)	Llama 3.1 8B	12.9 tok/s	not given	InsiderLLM, updated 2026-07-27

Read the spread honestly. The same B580 shows 73.8 tok/s in one report and 12.9 tok/s in another. The differences are the backend (SYCL versus Vulkan), the driver version, the model and the build date. The official doc records that the 2025.2 oneAPI release alone raised the A770 from 42 to 55 tok/s on Q4_0. The lesson: build from a recent commit, use a verified oneAPI release, and run llama-bench yourself before you judge the card.

If SYCL gives you trouble, Vulkan is the fallback with fewer moving parts: cmake -B build -DGGML_VULKAN=1 needs only the Mesa Vulkan driver, at the cost of speed on some models.

Where this fits and what it cannot do

An Arc A770 running an 8B model at 50-plus tokens per second is a capable back end for an internal assistant, document summaries, or a coding helper, at a card price under EUR 300. The 16 GB also leaves room for a 14B model at Q4_K_M, which a 12 GB card does not.

Limits to plan for:

• Ollama does not support Arc natively. Intel’s IPEX-LLM project provides a bridge, but the clean path is llama.cpp’s own llama-server. Tools that speak the OpenAI API (Open WebUI, n8n, most SDKs) work with it unchanged.
• Driver sensitivity. Kernel and driver versions matter more than on NVIDIA. Stick to a supported LTS distribution.
• No --split-mode row and no ahead-of-time compilation in the SYCL backend as of the 2026.02 update; the first launch is slow while kernels compile.
• SYCL does not target NVIDIA or AMD cards at the moment; the plugins are unavailable. For those, use the CUDA, HIP or Vulkan backends.

Next steps

• Pick the right quant for 12 or 16 GB: Quantization explained.
• See how Arc compares with a Jetson, a Mac mini or a NUC: Edge AI hardware guide 2026.
• Work out whether the card pays for itself: Cloud vs local break-even.

Work with us

We help small teams choose a GPU by memory first and marketing second, then set up llama.cpp or Ollama so the data stays in the building. If you want a second opinion before you buy, get in touch or see how our consulting works.