View all articles
Intel Arcllama.cppHardwareGuideLocal AI

llama.cpp on Intel Arc with SYCL or Vulkan: the documented path

JG
Jacobo González Jaspe
|

Reviewed:

Abstract illustration: ribbons of amber and white light weaving in a gentle curve. Models
Illustration generated with AI on our own machine.
This article is also available in Spanish:llama.cpp en Intel Arc con SYCL o Vulkan: la ruta documentada

By the end of this guide you will know the documented route to run a GGUF model on an Intel Arc graphics card with llama.cpp, through SYCL or through Vulkan, and how to check that the card is really doing the work. The cheapest card it applies to is one you already have.

Read this before you copy a command. We do not own an Intel Arc card and we have never run any step on this page. No command here was executed by us and no speed here is ours. Every step, flag, package name and figure comes from the primary source linked next to it, read on 16 November 2026. Where the sources are silent or disagree, we say so.

Who this is for

You have an Arc A-series, B-series or Pro B-series card, or a recent Intel laptop with built-in Arc graphics, and you want models to run on it. This is not a buying recommendation: our hardware catalogue is, and it makes no speed claim for Arc.

Versions this was checked against, all read on 16 November 2026:

The cards and their memory

Memory decides which models fit. These figures are from Intel’s own specification pages, and the Linux columns from Intel’s tables of Xe and i915 driver GPUs.

CardMemoryBandwidthBoard powerKernel driverFull Linux support from
Arc A3806 GB186 GB/s75 Wi915Ubuntu 24.04 (6.2)
Arc A5808 GB512 GB/s185 Wi915Ubuntu 24.04 (6.2)
Arc A7508 GB512 GB/s225 Wi915Ubuntu 24.04 (6.2)
Arc A770 16GB16 GB560 GB/s225 Wi915Ubuntu 24.04 (6.2)
Arc B57010 GB380 GB/s150 WxeUbuntu 24.04 (6.12)
Arc B58012 GB456 GB/s190 WxeUbuntu 24.04 (6.12)
Arc Pro B5016 GB224 GB/s70 WxeUbuntu 24.04 (6.14)
Arc Pro B6024 GB456 GB/s200 WxeUbuntu 24.04 (6.15)
Arc Pro B7032 GB608 GB/s230 WxeUbuntu 25.10 (6.17)

An A770 with 8 GB also exists: check which one you have. The number in brackets is the kernel Intel gives for full support. The SYCL document adds built-in Arc graphics (Meteor Lake, Arrow Lake, Lunar Lake) and integrated GPUs from 11th-generation Core onwards, and warns that below 80 execution units an integrated GPU is too slow to be useful.

Intel’s desktop quick start guide says Resizable BAR (Smart Access Memory on AMD boards) “must be enabled for optimal performance in all applications”. Turn it on in the UEFI before anything else.

SYCL or Vulkan: what the documentation says

llama.cpp’s README lists both backends in one table: SYCL with target “Intel GPU”, Vulkan with target “GPU”. Neither document recommends one over the other for Arc, and neither publishes a comparison.

SYCLVulkan
NeedsIntel GPU driver with Level Zero, plus the oneAPI Toolkit to buildA Vulkan driver, plus Vulkan headers and glslc to build
Works onIntel GPUs only: support for other vendors was removed in February 2026Any Vulkan GPU
Maintained byIts own contributor group, with a detailed documentThe general build guide, one section
Ready-made binariesUbuntu (FP16 and FP32), WindowsUbuntu, Windows

Our reading, not the project’s: start with Vulkan if you want the fewest moving parts, and with SYCL if you want the backend written for this hardware. Then measure both on your own model.

What you need (Linux)

  • A kernel at or above the one in the table. Intel’s installation guide puts it simply: use Ubuntu 24.04 if your card needs 6.8 or later, and 26.04 if it needs 6.17.
  • The Intel compute packages (Step 1).
  • Your user in the render group.
  • A GGUF model. To choose the file, see how to pick a GGUF quantisation.

The SYCL document lists Ubuntu 22.04, Fedora Silverblue 39 and Arch Linux as verified on Linux, and Windows 11.

Step 1: install the GPU compute driver

Check that the system sees the card and that a render node exists:

bash
lspci -nn | grep -Ei 'VGA|DISPLAY'
uname -r
ls /dev/dri

On Ubuntu 24.04 or 26.04, Intel’s PPA page gives:

bash
sudo apt-get update
sudo apt-get install -y software-properties-common
sudo add-apt-repository -y ppa:kobuk-team/intel-graphics
sudo apt-get install -y libze-intel-gpu1 libze1 intel-metrics-discovery intel-opencl-icd clinfo intel-gsc
sudo gpasswd -a ${USER} render
newgrp render
clinfo | grep "Device Name"

You should see your card’s name. On Ubuntu 24.04, B-series cards need the hardware enablement kernel, which Ubuntu Desktop tracks by default.

Other distributions. Intel’s guide covers Ubuntu only, and recommends the kernel and packages your distribution ships. Fedora packages the same runtime as intel-compute-runtime, with intel-level-zero and intel-opencl beside it. We found no Intel page for Fedora or Arch, so treat those names as a starting point.

Step 2: install the oneAPI Toolkit (SYCL only)

Since release 2026.0, Intel’s Base and HPC toolkits are one product, the Intel oneAPI Toolkit. The SYCL document recommends it over the smaller Deep Learning Essentials package, because that package no longer includes oneDNN. Its verified releases are 2026.2, 2026.1, 2025.3.3, 2025.2.1, 2025.1 and 2024.1.

Ubuntu and Debian, from Intel’s APT page:

bash
wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \
  | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" \
  | sudo tee /etc/apt/sources.list.d/oneAPI.list
sudo apt update
sudo apt install intel-oneapi-toolkit

Fedora and RHEL, from the YUM/DNF page: create the repository file, then install.

bash
tee > /tmp/oneAPI.repo << EOF
[oneAPI]
name=Intel® oneAPI repository
baseurl=https://yum.repos.intel.com/oneapi
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://yum.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
EOF
sudo mv /tmp/oneAPI.repo /etc/yum.repos.d
sudo dnf install intel-oneapi-toolkit

openSUSE, from the Zypper page: sudo zypper addrepo https://yum.repos.intel.com/oneapi oneAPI, then sudo zypper install intel-oneapi-toolkit.

All three pages say the same thing about graphics: the toolkit does not include GPU drivers. That is Step 1.

Step 3: load the environment and find the card

bash
source /opt/intel/oneapi/setvars.sh
sycl-ls

Run setvars.sh in every new shell. The document’s sample output has a line beginning [level_zero:gpu][level_zero:0] followed by the card’s name. No such line means no GPU for SYCL: go to the failure table.

Step 4: build llama.cpp with SYCL

bash
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
source /opt/intel/oneapi/setvars.sh
cmake -B build -DGGML_SYCL=ON -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_SYCL_F16=ON
cmake --build build --config Release -j -v

GGML_SYCL_F16=ON is the FP16 build, which the document recommends in most cases; drop the flag for FP32. It also says to test both, because they differ in accuracy and speed by model. The script ./examples/sycl/build.sh fp16 does the same.

A naming note. The README’s quick start now uses one llama command (llama cli, llama serve). The backend documents still name the tools a source build produces, such as llama-cli and llama-server. This guide uses those.

Step 5: check the card is used, then run a model

bash
./build/bin/llama-ls-sycl-device
./build/bin/llama-server --list-devices

The first prints a table with one row per device, marked [level_zero:gpu:0], and its memory in bytes. The second, in the document’s example, prints:

text
Available devices:
  SYCL0: Intel(R) Arc(TM) A770 Graphics (15473 MiB, 15473 MiB free)

Now run with every layer on the card:

bash
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-completion -no-cnv -m models/MODEL.gguf \
  -p "Building a website can be done in 10 simple steps:" -n 400 -e \
  -ngl 99 -sm none -mg 0 --load-mode auto

-ngl 99 asks for all layers on the GPU. -sm none -mg 0 pins device 0. ZES_ENABLE_SYSMAN=1 lets llama.cpp read free memory on the card. In the log, the document tells you to look for one of these:

text
detect 1 SYCL GPUs: [0] with top Max compute units:512
use 1 SYCL GPUs: [0] with Max compute units:512

It also says the loaded size appears as llm_load_tensors: buffer_size. In our own CUDA logs the proof of full offload is a line like offloaded 33/33 layers to GPU. The SYCL document does not show that line, so we cannot promise its wording there.

With an integrated GPU and a card together, the card is listed first. Pick with --device SYCL0, or export ONEAPI_DEVICE_SELECTOR="level_zero:0".

Step 6: serve it

bash
ZES_ENABLE_SYSMAN=1 ./build/bin/llama-server -m models/MODEL.gguf -ngl 99 -c 8192

The server README gives the default address as 127.0.0.1:9931. Older guides, ours included, said 8080: pass --port if you depend on a number. Set -c yourself.

Windows

  • No build. The SYCL document says the Windows package on the releases page (llama-<tag>-bin-win-sycl-x64.zip) bundles the SYCL runtime, so oneAPI is not needed. It is FP32 only; an FP16 build is an open request. Install the driver from Intel’s Arc drivers page first.
  • Build. Install Visual Studio and oneAPI, open the “Intel oneAPI command prompt”, and run examples\sycl\win-build-sycl.bat. The Windows section still names the old Base Toolkit, while the project’s own release build installs Deep Learning Essentials 2026.2 plus a separate oneDNN package. We could not settle which is right.
  • Vulkan. llama-<tag>-bin-win-vulkan-x64.zip needs only the graphics driver.

Under WSL, the Docker document says of its SYCL images that support “has not been verified yet”.

The Vulkan route

No oneAPI. On Ubuntu or Debian the build guide gives:

bash
sudo apt-get install libvulkan-dev glslc spirv-headers
vulkaninfo
cmake -B build -DGGML_VULKAN=1
cmake --build build --config Release
./build/bin/llama-cli -m "PATH_TO_MODEL" -p "Hi you how are you" -ngl 99

On Fedora and openSUSE the headers package is spirv-headers-devel. If vulkaninfo reports errors, stop and fix the driver. At start-up the guide says to look for a line of this shape, with your GPU’s name:

text
ggml_vulkan: Using Intel(R) Graphics (ADL GT2) | uma: 1 | fp16: 1 | warp size: 32

To skip the build, use llama-<tag>-bin-ubuntu-vulkan-x64.tar.gz. The Ubuntu SYCL packages are less self-contained: the document says they need drivers and a oneAPI of the same version as the build (2026.2 this month) and recommends Intel’s Docker image. Images ghcr.io/ggml-org/llama.cpp:server-intel and :server-vulkan exist; the Docker document notes they are built by CI but not tested.

Failures the sources document

What you seeCause and documented fixSource
error while loading shared libraries: libsycl.so: cannot open shared object fileoneAPI missing or not loaded. source /opt/intel/oneapi/setvars.shSYCL document
No level_zero:gpu line in sycl-lsTry sudo sycl-ls. If it appears there, add your user to render and video, log out and in. Otherwise recheck the driverSYCL document
Native API returns: 39 (UR_RESULT_ERROR_OUT_OF_DEVICE_MEMORY) or failed to allocate SYCL0 bufferOut of device memory. Set -c 8192 or less, or use a smaller quantisation or modelSYCL document
can't allocate 5000000000 Bytes of memory on deviceOnly when the Level Zero path is disabled: export UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1SYCL document
B-series: load refused when the model does not fitDeliberate. The driver would accept the allocation and spill into system RAM (compute-runtime issue 1047, open). Shrink the model or the contextSYCL document
Crash after setting SYCL_CACHE_PERSISTENT=1The document advises against it. Delete ~/.cache/libsycl_cache/ and unset itSYCL document
Compiler errors mid-buildRemove the build folder and build cleanSYCL document
Slow first startExpected: kernels compile at first run. Later speed is unaffectedSYCL document

The last row is a user report filed this week, not a documented fix.

What fits on each card

An Arc card is dedicated VRAM, so the sum from our VRAM guide applies: weights, plus context cache times parallel slots, plus buffers. The catalogue works it at Q4_K_M for one person: 6.49 GB for an 8B at 8k of context (9.71 GB at 32k), 11.10 GB for a 14B (15.93), 16.67 GB for a 24B (20.70) and 23.00 GB for a 32B (29.44). A card keeps 0.5 GB back. “Tight” means over 85% of the rest. Each cell reads 8k / 32k.

Card memoryUsable8B14B24B32B
6 GB (A380)5.5 GBnononono
8 GB (A580, A750)7.5 GBtight / nononono
10 GB (B570)9.5 GByes / nononono
12 GB (B580)11.5 GByes / yestight / nonono
16 GB (A770, Pro B50)15.5 GByes / yesyes / nonono
24 GB (Pro B60)23.5 GByes / yesyes / yesyes / tighttight / no
32 GB (Pro B70)31.5 GByes / yesyes / yesyes / yesyes / tight

For other models, contexts or several users, use the VRAM explorer, corrected today to this same sum. One caveat: we checked the sum against CUDA loads on our machines, not SYCL or Vulkan ones. The weights and cache are arithmetic; the buffer allowance may differ on Arc.

Speed: what others reported

None of this is ours. llama.cpp’s SYCL contributors keep a results thread where users post llama-bench runs. Its own header says the data is not double-checked. Every row below is the SYCL backend, one card, flash attention off unless noted; pp512 is prompt processing and tg128 generation, in tokens per second.

CardModel, quantisationBuildpp512tg128System, date
Arc A380Llama 2 7B Q4_0FP16495.1623.84Ubuntu 24.04.4, 5 Jun 2026
Arc A770Llama 2 7B Q4_0FP32937.2459.03Ubuntu 24.04.4, 19 May 2026
Arc A770Llama 2 7B Q4_0FP161,876.7968.72Debian 13, 5 Nov 2026
Arc B570Llama 2 7B Q4_0FP161,355.9572.52Ubuntu 24.04.4, 1 Jun 2026
Arc B580Llama 2 7B Q4_0FP162,063.5273.76Ubuntu 25.10, 27 May 2026
Arc Pro B60Llama 2 7B Q4_0FP161,869.1680.03Fedora 43, 27 Aug 2026
Arc Pro B70Llama 2 7B Q4_0FP162,830.14105.64Ubuntu 26.04, 26 Aug 2026
Arc Pro B70Llama 3.1 8B Q4_K_MFP32, flash attention on1,068.9488.49Ubuntu 26.04, 23 Jun 2026

Most rows use an old model, because it is the thread’s reference test. The two A770 rows differ by build, system and five months of commits. We found no Vulkan figure for an Arc card with hardware, model, quantisation and backend all named, so we give none; the Vulkan number in our September version is removed. To get yours: benchmark a local model in 30 minutes.

Ollama, OpenVINO and IPEX-LLM

Ollama. Its hardware page names NVIDIA and AMD cards in tables and no Intel card. It has a Vulkan section: extra GPU support on Windows and Linux, enabled by default when that backend is installed, with a link to Intel’s Linux driver instructions. For memory-aware scheduling it gives sudo setcap cap_perfmon+ep /usr/local/bin/ollama; OLLAMA_VULKAN=0 turns Vulkan off. It does not mention SYCL. The latest release on GitHub was v0.40.16 (9 November 2026). The SYCL document adds that its maintainers cannot handle Ollama problems: reproduce them on llama.cpp first. If Ollama is what you want, start from installing Ollama.

OpenVINO. llama.cpp has an OpenVINO backend, marked “In Progress” in the README. Its document says it was validated on Core Ultra laptops, supports a subset of operations, and handles text models only. One to watch, not the first choice for an Arc card.

IPEX-LLM. Intel’s bridge for llama.cpp and Ollama, which our September version suggested, is archived. Its README says Intel will no longer maintain it and that it has known security issues.

Prices

Only two Arc cards are in our hardware catalogue, read on 16 November 2026: the Arc B580 at EUR 470.13 new and the Arc Pro B70 at EUR 2,058.49, both at PcComponentes. The RTX 3060 12 GB was EUR 509.00 there the same day. We have no sourced price for the others, so we give none.

When not to do this

  • You already own an NVIDIA card. Use it. Ollama’s page lists it by name; our CUDA on Fedora guide covers the driver.
  • You are buying a 12 GB card and want the least friction. The B580 was EUR 38.87 cheaper than the RTX 3060 that day. That is the whole saving, against a route you have just read in full.
  • You need 24 GB. The catalogue’s second-hand RTX 3090 was asked at EUR 900 to 1,400, below the 32 GB Pro B70, which has more memory. We cannot tell you which is faster.
  • You want SYCL only to avoid a build. Take the Vulkan binary.

Frequent questions

Do I need oneAPI to use an Arc card? No. Vulkan needs none, and the Windows SYCL package carries its own runtime.

Which is faster, SYCL or Vulkan? The documentation does not say, and the SYCL thread forbids cross-backend tables. Run llama-bench on both.

FP16 or FP32? The document recommends FP16 in most cases. The thread’s notes say it helps prompt processing most, costs a little generation speed, and that FP32 is more accurate.

Can I use two Arc cards? Yes. --split-mode layer is the default; tensor is tuned for two GPUs; row is not supported on SYCL.

Does an integrated Arc GPU count? It is supported, and shares system memory. The document asks for at least 8 GB of shared memory for a 7B at Q4_0.

Can SYCL drive my NVIDIA or AMD card too? Not since February 2026. Use CUDA, HIP or Vulkan.

Limits

We ran nothing. No command on this page has been executed by us, on any Arc hardware, on any system. The fit table is arithmetic checked on other hardware. The speeds are user reports that their own host does not verify. We found no Intel driver page for Fedora or Arch, no primary Vulkan benchmark, and one Intel link in the SYCL document (oneAPI system requirements) no longer resolved. These projects move weekly: compare our dates with what you find.

Next steps

Work with us

We size the model and the machine by measuring, not by guessing, and we say when we have not measured. If you want a second opinion before you buy a card, get in touch or see how our consulting works.

Diagram
Share: LinkedIn X
Veredicto semanal

Get new guides before anyone else

Subscribe and we tell you when new guides, templates and workflows go up. One email a week, no spam.

Already published: 102 guides and 25 templates. All free, no signup.

Bonus: the local-AI starter pack PDF when you subscribe
Once a week No spam Unsubscribe anytime

See what you get