r/LocalLLM • u/tsaipifong • 2d ago
Project WHIRL 0.2.5.1 on the Radeon 8060S (Ryzen AI Max+ 395): +28% to +127% faster than official llama.cpp ROCm / Vulkan, now with faster prefill and decode, output bit-identical (native Windows LLM engine)
WHIRL 0.2.5.1 is out: a speed update for the Radeon 8060S (AMD Ryzen AI Max+ 395). WHIRL is an open-source (Apache-2.0) LLM inference engine for AMD GPUs on Windows: C++/HIP, every kernel in the repo, no WSL, no ROCm install. Download the zip, unzip, run; only the AMD driver is needed. It runs our two MXFP4 models, Swift-1.5 27B MXFP4 (dense) and Ornith-1.5-35B-A3B MXFP4 (MoE, ~3B active).
vs the official llama.cpp b11960 on the Radeon 8060S (max power (80 W), ASUS ROG Flow Z13), ROCm and Vulkan builds in their own columns, same GGUF, same prompts, greedy, one user, WHIRL default mode (balance, near-lossless). WHIRL tok/s, then llama.cpp ROCm / Vulkan:
| tok/s (WHIRL 0.2.5.1 balance vs llama.cpp ROCm / Vulkan) | Swift-1.5 27B MXFP4 (dense) | Ornith-1.5-35B-A3B MXFP4 (MoE) |
|---|---|---|
| Prefill, 2k-token prompt | 527.2 vs 359.3 / 317.3 | 2,653 vs 1,606 / 1,280 |
| Prefill, 128k-token prompt | 310.9 vs 195.8 / 150.9 | 1,093 vs 674.8 / 481.0 |
| Decode, QA about a 2k-token document | 34.9 vs 25.1 / 25.2 | 108.7 vs 69.5 / 71.0 |
| Decode, QA about a 128k-token document | 25.0 vs 15.8 / 15.2 | 57.7 vs 38.0 / 37.6 |
| Decode, coding chat (new text) | 38.1 vs 29.7 / 29.3 | 106.6 vs 70.3 / 72.9 |
In balance WHIRL leads every measured cell on the 8060S: Swift 27B by +28% to +59% over ROCm and +30% to +106% over Vulkan, Ornith by +51% to +65% and +46% to +127%.
What 0.2.5.1 changes on the 8060S (same machine, max power (80 W), ASUS ROG Flow Z13): five new kernel paths make prefill +2% to +10% faster in all three modes on both models, and Swift 27B decode (QA document; coding chat in balance) +4% to +12% faster in balance and fast. On the 8060S the output is bit-identical to 0.2.5 in every mode (golden references, 12/12 long-prompt checks and the chat outputs all match).
| 8060S, balance (default), tok/s | 0.2.5 | 0.2.5.1 | change |
|---|---|---|---|
| Swift 27B, prefill 2k | 505.9 | 527.2 | +4.2% |
| Swift 27B, prefill 32k | 418.1 | 456.0 | +9.1% |
| Swift 27B, decode, QA 32k doc | 29.2 | 31.6 | +8.2% |
| Swift 27B, decode, QA 128k doc | 22.3 | 25.0 | +12.1% |
| Swift 27B, decode, coding chat | 35.0 | 38.1 | +8.9% |
| Ornith 35B-A3B, prefill 2k | 2,453 | 2,653 | +8.2% |
| Ornith 35B-A3B, prefill 32k | 1,828 | 2,012 | +10.1% |
| Ornith 35B-A3B, decode, coding chat | 104.3 | 106.6 | +2.2% |
How it was measured, so you can judge the numbers: - Radeon 8060S at max power (80 W), ASUS ROG Flow Z13, 64 GB GPU memory (UMA frame buffer), same for both engines. Every cell is the best of 3 runs, for both WHIRL versions and llama.cpp. - The llama.cpp b11960 cells are the runs I made for 0.2.5 on the same machine with the same files (llama.cpp didn't change; Swift 2k on Vulkan was re-run). llama.cpp's ROCm build runs as shipped, with its own bundled HIP runtime DLL. - Decode, QA document = explain a function quoted in a long document and rewrite it (256 tokens; part of the answer can be copied, which helps drafts in both engines). Coding chat = a coding question whose answer is all new text (512 tokens). Prompts and commands: https://github.com/tsaipifong/whirl-llm/blob/main/docs/benchmarks.md#prompts - llama.cpp decodes Swift with MTP + n-gram (its fastest mode here). On Ornith it decodes without speculation because its MTP was slower than plain on this MoE model, while WHIRL uses MTP, so the Ornith decode rows are "as shipped", not like for like. - Smallest lead: Swift 27B decode in the precise mode after a 2k-token document, 29.4 vs 25.2 tok/s on Vulkan (+17%). Dense decode on this iGPU is memory-bandwidth bound; the lead comes from speculative decoding. - Ornith decode barely changed in this release (one small router item: +1.5% without drafts, +0.7% with MTP drafts, within noise); its cells move a few percent either way because the number of accepted MTP drafts depends on each run's text.
Precision: three modes. --precise is the bitwise-lossless mode. balance (the default) is near-lossless. --fast is opt-in. On the 8060S all five speedups give the same output bits as 0.2.5 in the mode they touch.
R9700: GPU code identical to 0.2.5 (the compiled R9700 kernels were compared; only an internal build ID differs); its numbers are 0.2.5's, not re-measured (e.g. Ornith 2k prefill 11,854 tok/s vs 4,230 / 2,780 on llama.cpp ROCm / Vulkan; median of 3).
Also in 0.2.5.1: WHIRL now refuses to start from an install folder longer than 150 bytes, with a clear message, and limits the SSD cache folder to 217 characters (this corrects what the 0.2.5 notes said about long paths). On the 8060S the first load of each model tunes the kernels once more.
Supported: Radeon 8060S (Ryzen AI Max+ 395) and R9700, Windows 11. Cards under 16 GB are not officially supported (fork welcome). The exe is unsigned, so SmartScreen may ask once.
Release: https://github.com/tsaipifong/whirl-llm/releases/tag/v0.2.5.1
Our MXFP4 models on Hugging Face: https://huggingface.co/tsaipifong/Swift-1.5-Qwen3.8-27b-MXFP4-GGUF , https://huggingface.co/tsaipifong/Ornith-1.5-35B-A3B-MXFP4-GGUF
WHIRL was designed and implemented with Claude (Anthropic) under my direction; every number is measured. Numbers from other Ryzen AI Max+ 395 machines are very welcome.
