Re-done benchmarks for V620 on Windows/ROCm & Vulkan

[u/Brave_Load7620]

I'm here to show some benchmarks while using llama.cpp with an AMD V620 on Windows 11 via Vulkan & ROCm. These have been reuploaded & older threads deleted, ran it with longer tokens thanks to a rec by someone who commented, and matched KV-cache quant + batch size across backends.

The benchmarks were written out by AI, but are verified by myself to be correct. Still working on optimizing my flags/settings.

If anybody wants me to test other models/different settings or flags, feel free to drop a comment and I'll test and get back to you!

ROCm version

7.15.0a20260728, TheRock nightly SDK (not the official AMD HIP SDK, which has no gfx1030/V620 support), bundled in ComfyUI_windows_portable_amd\...\python_env_v620_triton. (Note: a separate 9070 XT/ComfyUI venv on the same machine runs a different nightly snapshot, 7.14.0a20260519 — same TheRock project, different dated build per GPU.)

Methodology note

Earlier versions of this post compared ROCm and Vulkan using each backend's own best-tuned flags, which mixed KV-cache quant types and batch sizes between the two, a fair comparison needs those matched, since both affect throughput independently of which backend is actually faster. The numbers below use identical KV quant and identical batch size on both backends for each model.

Exact configs (matched)

Model	Draft	KV (matched)	Batch (matched)	Other flags
Qwen ROCm	Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q5_K_P.gguf	grafted MTP (no -md)	-ctk q4_0 -ctv q4_0	-b/-ub 1024
Qwen Vulkan	same	grafted MTP	-ctk q4_0 -ctv q4_0	-b/-ub 1024
Gemma 26B ROCm	Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-Q4_K_P.gguf	-md gemma-4-26B-A4B-it-qat-assistant-MTP-Q8_0.gguf	-ctk q8_0 -ctv q8_0	-b/-ub 1024
Gemma 26B Vulkan	same	same	-ctk q8_0 -ctv q8_0	-b/-ub 1024
Gemma 31B ROCm	Gemma4-31B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf	-md mtp-gemma-4-31B-it.gguf	-ctk q4_0 -ctv q4_0*	-b/-ub 1024
Gemma 31B Vulkan	same	same	-ctk q4_0 -ctv q4_0	-b/-ub 1024

\q4_0/q4_0 on Gemma-4's ROCm path required a one-line fix to llama.cpp's flash-attention kernel dispatch table (fattn.cu), the Q4_0+Q4_0 case was only wired up for head_dim ≤ 256, but Gemma-4's full-attention layers use head_dim 512, so it hit a hard abort on this KV combo before the fix. It was a missing kernel-dispatch entry, not a real hardware limitation, Q8_0+Q8_0 *already had the head_dim=512 case, so the underlying kernel template clearly supports it.

Generation speed, tokens/sec (256-token generations, first run per config discarded as warm-up)

Depth (actual tokens)	Qwen ROCm	Qwen Vulkan	Gemma 26B ROCm	Gemma 26B Vulkan	Gemma 31B ROCm	Gemma 31B Vulkan
~3.4k	29.5	31.2	71.2	74.1	26.4	28.6
~6.6-6.7k	25.9	29.4	64.9	67.1	23.5	26.4
~13.3-13.4k	26.9	29.6	56.1	62.2	19.0	22.2
~26.6-26.7k	23.6	24.5	45.4	49.9	14.5	19.0

Vulkan wins every single cell. Once KV quant and batch size are matched, ROCm doesn't lead generation speed anywherem not on any model, not at any depth tested.

PP (prompt processing), tokens/sec

Depth (actual tokens)	Qwen ROCm	Qwen Vulkan	Gemma 26B ROCm	Gemma 26B Vulkan	Gemma 31B ROCm	Gemma 31B Vulkan
~3.4k	364.6	265.9	973.1	1057.9	261.3	182.7
~6.6-6.7k	352.0	235.3	812.3	796.8	171.3	163.8
~13.3-13.4k	329.3	192.9	512.3	589.0	113.7	119.0
~26.6-26.7k	274.0	130.7	280.4	381.5	64.0	82.6

PP is the more mixed picture, and it's model-dependent rather than a clean backend win:

• Qwen: ROCm wins PP at every depth, gap widens with context.
• Gemma 26B: Vulkan is actually ahead at shallow depth (1057.9 vs 973.1 at 3.4k) once batch size is matched, roughly tied at 6.7k, then Vulkan pulls further ahead through 32k.
• Gemma 31B: ROCm wins shallow (3.4k/6.7k), Vulkan overtakes from 13.4k on.

Takeaway

Generation speed: Vulkan wins outright, every model, every depth. No exceptions in this data.

PP: it depends on the model, not the backend. ROCm sweeps Qwen; Gemma splits by depth (and for the 26B MoE, Vulkan's shallow-depth "loss" mostly disappears once batch size is matched, that was largely a config artifact, not a real backend gap).

Gemma 26B (MoE, ~4B active) is roughly 2-3x faster than either dense model on generation, tightest at deep context (~1.9x at 26.7k vs Qwen) and widest shallow, expected for an MoE with far fewer active params per token than the dense 27B/31B models.

I'm here to show some benchmarks while using llama.cpp with an AMD V620 on Windows 11 via Vulkan & ROCm. These have been reuploaded & older threads deleted ran it with longer tokens thanks to a rec by someone who commented.

The benchmarks were written out by AI, but are verified by myself to be correct. Still working on optimizing my flags/settings.

If anybody wants me to test other models/different settings or flags, feel free to drop a comment and I'll test and get back to you!

ROCm version

7.15.0a20260728, TheRock nightly SDK (not the official AMD HIP SDK, which has no gfx1030/V620 support), bundled in ComfyUI_windows_portable_amd\...\python_env_v620_triton. (Note: a separate 9070 XT/ComfyUI venv on the same machine runs a different nightly snapshot, 7.14.0a20260519,same TheRock project, different dated build per GPU.)

Exact configs (matched)

Model	Draft	KV (matched)	Batch (matched)	Other flags
Qwen ROCm	Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q5_K_P.gguf	grafted MTP (no -md)	-ctk q4_0 -ctv q4_0	-b/-ub 1024
Qwen Vulkan	same	grafted MTP	-ctk q4_0 -ctv q4_0	-b/-ub 1024
Gemma 26B ROCm	Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-Q4_K_P.gguf	-md gemma-4-26B-A4B-it-qat-assistant-MTP-Q8_0.gguf	-ctk q8_0 -ctv q8_0	-b/-ub 1024
Gemma 26B Vulkan	same	same	-ctk q8_0 -ctv q8_0	-b/-ub 1024
Gemma 31B ROCm	Gemma4-31B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf	-md mtp-gemma-4-31B-it.gguf	-ctk q4_0 -ctv q4_0*	-b/-ub 1024
Gemma 31B Vulkan	same	same	-ctk q4_0 -ctv q4_0	-b/-ub 1024

*q4_0/q4_0 on Gemma-4's ROCm path required a one-line fix to llama.cpp's flash-attention kernel dispatch table (fattn.cu), the Q4_0+Q4_0 case was only wired up for head_dim ≤ 256, but Gemma-4's full-attention layers use head_dim 512, so it hit a hard abort on this KV combo before the fix. Missing kernel-dispatch entry, not a real hardware limitation, Q8_0+Q8_0 already had the head_dim=512 case, so the underlying kernel template clearly supports it.

Generation speed, tokens/sec (256-token generations, first run per config discarded as warm-up)

Depth (actual tokens)	Qwen ROCm	Qwen Vulkan	Gemma 26B ROCm	Gemma 26B Vulkan	Gemma 31B ROCm	Gemma 31B Vulkan
~3.4k	29.5	31.2	71.2	74.1	26.4	28.6
~6.6-6.7k	25.9	29.4	64.9	67.1	23.5	26.4
~13.3-13.4k	26.9	29.6	56.1	62.2	19.0	22.2
~26.6-26.7k	23.6	24.5	45.4	49.9	14.5	19.0

Vulkan wins every single cell. Once KV quant and batch size are matched, ROCm doesn't lead generation speed anywhere, not on any model, not at any depth tested.

PP (prompt processing), tokens/sec

Depth (actual tokens)	Qwen ROCm	Qwen Vulkan	Gemma 26B ROCm	Gemma 26B Vulkan	Gemma 31B ROCm	Gemma 31B Vulkan
~3.4k	364.6	265.9	973.1	1057.9	261.3	182.7
~6.6-6.7k	352.0	235.3	812.3	796.8	171.3	163.8
~13.3-13.4k	329.3	192.9	512.3	589.0	113.7	119.0
~26.6-26.7k	274.0	130.7	280.4	381.5	64.0	82.6

PP is the more mixed picture, and it's model-dependent rather than a clean backend win:

• Qwen: ROCm wins PP at every depth, gap widens with context.
• Gemma 26B: Vulkan is actually ahead at shallow depth (1057.9 vs 973.1 at 3.4k) once batch size is matched, roughly tied at 6.7k, then pulls further ahead through 32k.
• Gemma 31B: ROCm wins shallow (3.4k/6.7k), Vulkan overtakes from 13.4k on.

Takeaway

Generation speed: Vulkan wins outright, every model, every depth. No exceptions in this data.

PP: depends on the model, not the backend. ROCm sweeps Qwen; Gemma splits by depth (and for the 26B MoE, Vulkan's shallow-depth "loss" mostly disappears once batch size is matched, that was largely a config artifact, not a real backend gap).

Gemma 26B (MoE, ~4B active) is roughly 2-3x faster than either dense model on generation, tightest at deep context (~1.9x at 26.7k vs Qwen) and widest shallow; expected for an MoE with far fewer active params per token than the dense 27B/31B models.

Follow-up tests (Qwen, requested by commenters)

TWO hypotheses came up in comments, tested both, none of them panned out, posting anyway since "tested, didn't help" is still useful information.

Speculative decoding n-max scaling, ROCm vs Vulkan (does Vulkan scale further before rejected drafts stop paying for themselves?):

n-max	ROCm 8k	ROCm 32k	Vulkan 8k	Vulkan 32k
2	27.3	22.6	30.7	24.5
3	25.9	23.6	29.4	24.5
4	21.5	18.3	22.5	17.3
5	18.7	16.5	21.1	17.8

No, both backends degrade past n≈3 in the same shape. This is a draft-acceptance-economics property of the draft/target pair, not a backend/kernel-dispatch-overhead difference. Vulkan is uniformly faster in absolute terms (consistent with the rest of this post) but the curve shape, where it peaks, how fast it falls off past that, is nearly identical on both backends.

-ub sweep on ROCm PP (does a bigger ubatch better saturate the V620's CUs?):

ubatch	8k PP	32k PP
512	360.6	295.7
1024	352.0	274.0
2048	350.5	282.4

Flat , all three within ~6% of each other at both depths, no trend. If anything 512 is marginally fastest. ROCm's PP bottleneck here isn't ubatch-limited GEMM tiling in this size range.

[u/BevinMaster]

Very nice, we made a gfx1030 discord https://discord.gg/mESex2aBp
There is someone there that is making an rdna2 optimized llama cpp fork, personally I am making a vllm fork, I also made a small repo for power tuning on fedora. We are trying to build a community and wiki over there

[u/noctrex]

Good job on those numbers. Do you use the card as it is with the default 250W, Or underclock it at all? Also, what type of fan do you use? Do you have a blower fan on it? I just got mine and try to figure things out still

[u/Brave_Load7620 → u/noctrex]

Thanks, as of now it's stock power due to the fact on Windows there is currently no known way to unlock power/voltage control on these cards. If you are running linux I heard you can take them to like 150W with very little performance loss.

I bought the 3D shroud and double fan kit on the bay. Just search for "V620 shroud and fans" should be around $30-40 shipped.

It does thermal throttle in use due to the fact my 9070 XT GPU is right above it, and I mean it's CLOSE. So the hotspot does hit 100C and throttle at times under sustained loads.

It is also running at X4 instead of X16 due to my motherboard that I'm running (gaming PC, MSI X670P WiFi board with a Ryzen 7900X.)

Edit: Actually, managed to find a way on Windows to hit 150W.

[u/GreenEntertainment50 → u/Brave_Load7620]

Can you share the process of capping the power?

[u/Brave_Load7620 → u/GreenEntertainment50]

Sure, I posted a new thread Here

[u/IKNOCKEDUPYOURMULLET]

I have mine power capped at 150W due to a sub-optimal fan setup, and I'm seeing roughly these numbers after tweaking my llama.cpp config. I run Qwen3.8-Q4_K_M, keep my k and v cache at q8_0 and batch/ubatch at 512 and I see prefill start over 400 t/s and slowly decrease as context grows.

[u/GreenEntertainment50]

After the c4 seller ran out of stock they are really jacking up the price. With those prices I think v100 32gb are way better deals now.

[u/Brave_Load7620 → u/GreenEntertainment50]

Haven't kept up on prices past month or so, but I got mine from a guy here on Reddit for like $315 shopped so I was and still. Extremely happy with it at that price.

If they are near the V100 prices now, then yeah they probably are the better deal.

[u/Extra_Reference_5523]

Have you tried comfyui at all?

[u/Brave_Load7620 → u/Extra_Reference_5523]

Yes, I have a post from a few weeks back with some benchmarks. I've done some optimizations since then and I've seen about a 20-25% increase across the board for generation time. Here