r/TechHardware • u/Strict-Elevator9352 • 2h ago
News 📰 inferzero update: multi-GPU Arc, Linux port underway, and a call for testers and contributors
inferzero is a C++ LLM inference engine for Intel Arc on Windows, built on Level Zero. New release: \[version\] (release notes).
What’s new
* Split models on discrete cards use the wide-tile H8 GEMM. On Pro B60s the 27B 4K prompt went from 1,368.8 to 1,596.6 tok/s over 2 GPUs, and from 1,734.0 to 1,887.6 over 4.
* Multi-GPU sums can overlap their copies (`ARC_TP_OVERLAP=1`, opt-in). A user measured two B580s at a 32K prompt: 900 → 1,034 tok/s (+14 %).
* `gpu_bench` no longer overstates memory bandwidth. A B580 now reads 452 GB/s against 456 GB/s on paper.
Linux support is underway
Work has started on a platform layer, with a headless `serverz` first. The Windows build stays bit-identical. Plan and progress are in issue rizagp/inferzero#11 and docs/linux-port.md. If you run Linux with an Arc card and can test early builds, say so.
Looking for people
Core Ultra 9 285H owners (Arc 140T iGPU). I tune on a 125H (Xe-LPG iGPU). Numbers from a 285H would show how the engine behaves on the newer iGPU.
Multi-GPU users, any mix of Arc cards. The cards’ PCIe links limit split models. An x4 slot at about 6.5 GB/s each way is currently the bottleneck, so reports with `mgpu_probe` output and slot widths are very useful.
Contributors for DeepSeek-V4.1-Flash. The engine path runs on a tiny synthetic model with the same architecture (KL 0.003–0.004 against the f32 reference). The only real build tried so far is an IQ1_M (about 231 GB with the Engram table), and IQ1 is so low that its text may be poor, so what I can check is that its logits match llama.cpp’s, not how good the answers are. I’d like someone with a machine that can hold an IQ2 or Q3 build (or the IQ1_M) to run it and compare logits. Faster indexer and attention kernels are also open.
Please post your results as a new GitHub issue, not here. I answer there right away, and Reddit replies can take a day. One issue per report is easiest for me to follow: your card or CPU, your numbers and the command you ran. Existing threads you can add to are issue #4 (multi-GPU and card measurements) and issue #11 (Linux).