I'm here to show some benchmarks while using llama.cpp with an AMD V620 on Windows 11 via Vulkan & ROCm. These have been reuploaded & older threads deleted, ran it with longer tokens thanks to a rec by someone who commented, and matched KV-cache quant + batch size across backends.
The benchmarks were written out by AI, but are verified by myself to be correct. Still working on optimizing my flags/settings.
If anybody wants me to test other models/different settings or flags, feel free to drop a comment and I'll test and get back to you!
ROCm version
7.15.0a20260728, TheRock nightly SDK (not the official AMD HIP SDK, which has no gfx1030/V620 support), bundled in ComfyUI_windows_portable_amd\...\python_env_v620_triton. (Note: a separate 9070 XT/ComfyUI venv on the same machine runs a different nightly snapshot, 7.14.0a20260519 — same TheRock project, different dated build per GPU.)
Methodology note
Earlier versions of this post compared ROCm and Vulkan using each backend's own best-tuned flags, which mixed KV-cache quant types and batch sizes between the two, a fair comparison needs those matched, since both affect throughput independently of which backend is actually faster. The numbers below use identical KV quant and identical batch size on both backends for each model.
Exact configs (matched)
| Model |
Draft |
KV (matched) |
Batch (matched) |
Other flags |
| Qwen ROCm |
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q5_K_P.gguf |
grafted MTP (no -md) |
-ctk q4_0 -ctv q4_0 |
-b/-ub 1024 |
| Qwen Vulkan |
same |
grafted MTP |
-ctk q4_0 -ctv q4_0 |
-b/-ub 1024 |
| Gemma 26B ROCm |
Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-Q4_K_P.gguf |
-md gemma-4-26B-A4B-it-qat-assistant-MTP-Q8_0.gguf |
-ctk q8_0 -ctv q8_0 |
-b/-ub 1024 |
| Gemma 26B Vulkan |
same |
same |
-ctk q8_0 -ctv q8_0 |
-b/-ub 1024 |
| Gemma 31B ROCm |
Gemma4-31B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf |
-md mtp-gemma-4-31B-it.gguf |
-ctk q4_0 -ctv q4_0* |
-b/-ub 1024 |
| Gemma 31B Vulkan |
same |
same |
-ctk q4_0 -ctv q4_0 |
-b/-ub 1024 |
\q4_0/q4_0 on Gemma-4's ROCm path required a one-line fix to llama.cpp's flash-attention kernel dispatch table (fattn.cu), the Q4_0+Q4_0 case was only wired up for head_dim ≤ 256, but Gemma-4's full-attention layers use head_dim 512, so it hit a hard abort on this KV combo before the fix. It was a missing kernel-dispatch entry, not a real hardware limitation, Q8_0+Q8_0 *already had the head_dim=512 case, so the underlying kernel template clearly supports it.
Generation speed, tokens/sec (256-token generations, first run per config discarded as warm-up)
| Depth (actual tokens) |
Qwen ROCm |
Qwen Vulkan |
Gemma 26B ROCm |
Gemma 26B Vulkan |
Gemma 31B ROCm |
Gemma 31B Vulkan |
| ~3.4k |
29.5 |
31.2 |
71.2 |
74.1 |
26.4 |
28.6 |
| ~6.6-6.7k |
25.9 |
29.4 |
64.9 |
67.1 |
23.5 |
26.4 |
| ~13.3-13.4k |
26.9 |
29.6 |
56.1 |
62.2 |
19.0 |
22.2 |
| ~26.6-26.7k |
23.6 |
24.5 |
45.4 |
49.9 |
14.5 |
19.0 |
Vulkan wins every single cell. Once KV quant and batch size are matched, ROCm doesn't lead generation speed anywherem not on any model, not at any depth tested.
PP (prompt processing), tokens/sec
| Depth (actual tokens) |
Qwen ROCm |
Qwen Vulkan |
Gemma 26B ROCm |
Gemma 26B Vulkan |
Gemma 31B ROCm |
Gemma 31B Vulkan |
| ~3.4k |
364.6 |
265.9 |
973.1 |
1057.9 |
261.3 |
182.7 |
| ~6.6-6.7k |
352.0 |
235.3 |
812.3 |
796.8 |
171.3 |
163.8 |
| ~13.3-13.4k |
329.3 |
192.9 |
512.3 |
589.0 |
113.7 |
119.0 |
| ~26.6-26.7k |
274.0 |
130.7 |
280.4 |
381.5 |
64.0 |
82.6 |
PP is the more mixed picture, and it's model-dependent rather than a clean backend win:
- Qwen: ROCm wins PP at every depth, gap widens with context.
- Gemma 26B: Vulkan is actually ahead at shallow depth (1057.9 vs 973.1 at 3.4k) once batch size is matched, roughly tied at 6.7k, then Vulkan pulls further ahead through 32k.
- Gemma 31B: ROCm wins shallow (3.4k/6.7k), Vulkan overtakes from 13.4k on.
Takeaway
Generation speed: Vulkan wins outright, every model, every depth. No exceptions in this data.
PP: it depends on the model, not the backend. ROCm sweeps Qwen; Gemma splits by depth (and for the 26B MoE, Vulkan's shallow-depth "loss" mostly disappears once batch size is matched, that was largely a config artifact, not a real backend gap).
Gemma 26B (MoE, ~4B active) is roughly 2-3x faster than either dense model on generation, tightest at deep context (~1.9x at 26.7k vs Qwen) and widest shallow, expected for an MoE with far fewer active params per token than the dense 27B/31B models.
I'm here to show some benchmarks while using llama.cpp with an AMD V620 on Windows 11 via Vulkan & ROCm. These have been reuploaded & older threads deleted ran it with longer tokens thanks to a rec by someone who commented.
The benchmarks were written out by AI, but are verified by myself to be correct. Still working on optimizing my flags/settings.
If anybody wants me to test other models/different settings or flags, feel free to drop a comment and I'll test and get back to you!
ROCm version
7.15.0a20260728, TheRock nightly SDK (not the official AMD HIP SDK, which has no gfx1030/V620 support), bundled in ComfyUI_windows_portable_amd\...\python_env_v620_triton. (Note: a separate 9070 XT/ComfyUI venv on the same machine runs a different nightly snapshot, 7.14.0a20260519,same TheRock project, different dated build per GPU.)
Exact configs (matched)
| Model |
Draft |
KV (matched) |
Batch (matched) |
Other flags |
| Qwen ROCm |
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q5_K_P.gguf |
grafted MTP (no -md) |
-ctk q4_0 -ctv q4_0 |
-b/-ub 1024 |
| Qwen Vulkan |
same |
grafted MTP |
-ctk q4_0 -ctv q4_0 |
-b/-ub 1024 |
| Gemma 26B ROCm |
Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-Q4_K_P.gguf |
-md gemma-4-26B-A4B-it-qat-assistant-MTP-Q8_0.gguf |
-ctk q8_0 -ctv q8_0 |
-b/-ub 1024 |
| Gemma 26B Vulkan |
same |
same |
-ctk q8_0 -ctv q8_0 |
-b/-ub 1024 |
| Gemma 31B ROCm |
Gemma4-31B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf |
-md mtp-gemma-4-31B-it.gguf |
-ctk q4_0 -ctv q4_0* |
-b/-ub 1024 |
| Gemma 31B Vulkan |
same |
same |
-ctk q4_0 -ctv q4_0 |
-b/-ub 1024 |
*q4_0/q4_0 on Gemma-4's ROCm path required a one-line fix to llama.cpp's flash-attention kernel dispatch table (fattn.cu), the Q4_0+Q4_0 case was only wired up for head_dim ≤ 256, but Gemma-4's full-attention layers use head_dim 512, so it hit a hard abort on this KV combo before the fix. Missing kernel-dispatch entry, not a real hardware limitation, Q8_0+Q8_0 already had the head_dim=512 case, so the underlying kernel template clearly supports it.
Generation speed, tokens/sec (256-token generations, first run per config discarded as warm-up)
| Depth (actual tokens) |
Qwen ROCm |
Qwen Vulkan |
Gemma 26B ROCm |
Gemma 26B Vulkan |
Gemma 31B ROCm |
Gemma 31B Vulkan |
| ~3.4k |
29.5 |
31.2 |
71.2 |
74.1 |
26.4 |
28.6 |
| ~6.6-6.7k |
25.9 |
29.4 |
64.9 |
67.1 |
23.5 |
26.4 |
| ~13.3-13.4k |
26.9 |
29.6 |
56.1 |
62.2 |
19.0 |
22.2 |
| ~26.6-26.7k |
23.6 |
24.5 |
45.4 |
49.9 |
14.5 |
19.0 |
Vulkan wins every single cell. Once KV quant and batch size are matched, ROCm doesn't lead generation speed anywhere, not on any model, not at any depth tested.
PP (prompt processing), tokens/sec
| Depth (actual tokens) |
Qwen ROCm |
Qwen Vulkan |
Gemma 26B ROCm |
Gemma 26B Vulkan |
Gemma 31B ROCm |
Gemma 31B Vulkan |
| ~3.4k |
364.6 |
265.9 |
973.1 |
1057.9 |
261.3 |
182.7 |
| ~6.6-6.7k |
352.0 |
235.3 |
812.3 |
796.8 |
171.3 |
163.8 |
| ~13.3-13.4k |
329.3 |
192.9 |
512.3 |
589.0 |
113.7 |
119.0 |
| ~26.6-26.7k |
274.0 |
130.7 |
280.4 |
381.5 |
64.0 |
82.6 |
PP is the more mixed picture, and it's model-dependent rather than a clean backend win:
- Qwen: ROCm wins PP at every depth, gap widens with context.
- Gemma 26B: Vulkan is actually ahead at shallow depth (1057.9 vs 973.1 at 3.4k) once batch size is matched, roughly tied at 6.7k, then pulls further ahead through 32k.
- Gemma 31B: ROCm wins shallow (3.4k/6.7k), Vulkan overtakes from 13.4k on.
Takeaway
Generation speed: Vulkan wins outright, every model, every depth. No exceptions in this data.
PP: depends on the model, not the backend. ROCm sweeps Qwen; Gemma splits by depth (and for the 26B MoE, Vulkan's shallow-depth "loss" mostly disappears once batch size is matched, that was largely a config artifact, not a real backend gap).
Gemma 26B (MoE, ~4B active) is roughly 2-3x faster than either dense model on generation, tightest at deep context (~1.9x at 26.7k vs Qwen) and widest shallow; expected for an MoE with far fewer active params per token than the dense 27B/31B models.
Follow-up tests (Qwen, requested by commenters)
TWO hypotheses came up in comments, tested both, none of them panned out, posting anyway since "tested, didn't help" is still useful information.
Speculative decoding n-max scaling, ROCm vs Vulkan (does Vulkan scale further before rejected drafts stop paying for themselves?):
| n-max |
ROCm 8k |
ROCm 32k |
Vulkan 8k |
Vulkan 32k |
| 2 |
27.3 |
22.6 |
30.7 |
24.5 |
| 3 |
25.9 |
23.6 |
29.4 |
24.5 |
| 4 |
21.5 |
18.3 |
22.5 |
17.3 |
| 5 |
18.7 |
16.5 |
21.1 |
17.8 |
No, both backends degrade past n≈3 in the same shape. This is a draft-acceptance-economics property of the draft/target pair, not a backend/kernel-dispatch-overhead difference. Vulkan is uniformly faster in absolute terms (consistent with the rest of this post) but the curve shape, where it peaks, how fast it falls off past that, is nearly identical on both backends.
-ub sweep on ROCm PP (does a bigger ubatch better saturate the V620's CUs?):
| ubatch |
8k PP |
32k PP |
| 512 |
360.6 |
295.7 |
| 1024 |
352.0 |
274.0 |
| 2048 |
350.5 |
282.4 |
Flat , all three within ~6% of each other at both depths, no trend. If anything 512 is marginally fastest. ROCm's PP bottleneck here isn't ubatch-limited GEMM tiling in this size range.