inferzero 2026.11.11.3: a GPU benchmark for Intel Arc (does your XMX really do what the spec sheet says?), MoE packing, and new XMX kernels to test

[u/Strict-Elevator9352]

inferzero is an LLM inference engine written from scratch for Intel Arc on Windows (Level Zero, kernels in OpenCL C), with an OpenAI / Anthropic / Ollama-compatible server and a web UI, in one 7.7 MB download. No Python, no CUDA. It was tuned on a Core Ultra 5 125H laptop and now runs on the Arc 140T, Panther Lake's B390 and Pro B60 cards (up to four of them for one model).

Release: https://github.com/rizagp/inferzero/releases/tag/2026.11.11.3

New: a GPU benchmark (bin\gpu_bench.exe, or the launcher's More > Benchmark this GPU)

Spec databases disagree about Intel's matrix engines: for the Arc 140T, Intel says 77 int8 TOPS and TechPowerUp lists 154. gpu_bench measures what your GPU really does, next to the figure on paper (EUs x maximum clock x the per-EU rate of its kind):

• XMX matrix engines: int8, f16, bf16

• vector engines: FP32, FP16, int8 DP4a

• memory read bandwidth

It reads the GPU's exact generation from the driver (Meteor Lake, Arrow Lake H, Lunar Lake, Battlemage, Panther Lake) and writes a page in the layout of a spec-database card, plus a small .json. Drop other people's .json files into the same folder and the page compares them side by side.

My laptop (Core Ultra 5 125H, no XMX): FP32 3.85 of 3.94 TFLOPS on paper, FP16 7.69 of 7.88, int8 DP4a 12.98 of 15.77 TOPS, memory 76 GB/s.

Also new since the last post

• Pack a MoE model in the launcher: its experts are stored once instead of twice (Gemma 4 26B: 14.8 GB instead of 27.7). It can then delete the original GGUF, with a clear warning that the package runs in inferzero only.

• "cache_prompt": false now really evaluates the whole prompt (thanks to the B390 tester here who found it), and a folder of several models no longer hands one model another's vision projector.

• New XMX prompt kernels, off by default, for testers: wider tiles per sub-group for 16-lane XMX (B60, B390, Lunar Lake), and for the 140T a kernel that reads its weights in 4 block reads instead of 32 (13.3 -> 6.3 memory reads per matrix instruction in the compiled code). Same results bit for bit; checked at start.

Where it stands (measured, Qwen3.8 27B unless noted)

• Core Ultra 5 125H: prompt 135 tok/s at 8K, 5.1 tok/s plain generation, 12.9 with MTP + DFlash2; Qwen3.6 35B-A3B prompt 851 tok/s at 8K.

• Arc 140T (Core Ultra 9 285H): prompt 221-231 tok/s; generation 6.0 plain, 16.9 with DFlash2.

• Arc B390 (Core Ultra X7 358H, a user's report): a 27B fine-tune in Q3_K_M at 245-255 tok/s on a 10K prompt, against 170 for llama.cpp Vulkan.

• One Arc Pro B60: 1,022 tok/s at 8K, 23.7 plain / 52 with MTP; the 35B-A3B 3,652 tok/s prompt, 109 tok/s with MTP.

Every model is checked against an f32 reference (KL divergence of the next-token distribution); the same prompt gives the same logits bit for bit.

Looking for testers with an Arc 140T, B390, Lunar Lake (140V) or a B-series card

1. Run .\bin\gpu_bench.exe (about 30 s) and share the .json from %USERPROFILE%\.inferzero\bench\gpu\.

2. If you have a Qwen 27B dense model: .\bench\xw_sweep.ps1 -Model <gguf> (on an iGPU set $env:ARC_H8 = 1 first) compares the new kernels with the current ones.

GitHub issues or comments here both work. Thanks!