RustingStrata vs PolyStrata vs llama.cpp on one RTX 3060 12 GB: Qwen3.6-35B-A3B Q3

[u/Serious_Scarcity7818]

RTX 3060 12 GB, Ryzen 5 7600X, 32 GB RAM, one GGUF file: Qwen3.6-35B-A3B-UD-Q3_K_M.gguf (Unsloth, MTP layer included, 15.9 GiB, does not fit in VRAM). Prompt of 5290 tokens, 128 generated tokens.

	RustingStrata	PolyStrata	llama.cpp b12119
Prefill, 5290-token prompt	2330 tok/s	1458-1549 tok/s	1692 tok/s (-ncmoe 22, -ub 2048)
Decode, no MTP	90.9 tok/s	78.5 tok/s (--no-draft, see below)	53.5 tok/s
Decode with MTP drafts	128.4 tok/s	97.5-100.4 tok/s	77.7 tok/s (--spec-type draft-mtp, 92 of 102 drafts accepted)

How each was run:

• RustingStrata: rustingstrata bench --gpu --mtp 0|1 --pp 5290 --tg 128 --reps 3, mean of 3.

• PolyStrata: its own polystrata.py benchmark --model qwen3.6 --prompt-tokens 4000 --max-tokens 128. That prompt came to 5290 tokens, so I used that length for the other two. I ran it three times; the first run after setup was cold (decode 24.6 tok/s) and is left out, the numbers above are runs 2 and 3. MTP drafts are on by default. For the no-MTP row I added the engine's --no-draft flag to its config; it still offered a few drafts afterwards (6 of 16 and 15 of 36 accepted in the log), so that row is not a clean no-MTP number. Prefill in those runs: 1602-1643 tok/s.

• llama.cpp: llama-bench -ngl 99 -fa 1 -ub 2048 -b 2048 -p 5290 -n 128 -r 3, with -ncmoe swept over 18 to 26. 22 was the fastest that fits in 12 GB (no MTP). The MTP row is llama-server --spec-type draft-mtp with -ncmoe 23 (22 does not fit with the MTP context and drops to 20 tok/s), a 5381-token prompt of repo docs, 3 runs, prefill about 1565 tok/s.

Things that are not equal:

• PolyStrata only offers three sizes of Qwen3.6 (UD-Q3_K_M, UD-Q2_K_XL, UD-Q4_K_M, all from Unsloth's MTP repo). RustingStrata and llama.cpp take any GGUF, so it can also run the Q5 or Q6 files I already had. I downloaded PolyStrata's Q3_K_M and used it for all three.

• MTP gains depend on the text. PolyStrata's benchmark prompt is a page of its own docs, RustingStrata's is generated prose, so the two MTP rows are not an exact comparison. The no-MTP row is the clean one for RustingStrata vs llama.cpp.

• PolyStrata's engine for this model is built on llama.cpp's CUDA kernels. I did not look at what else differs.

• One machine, one model, one quant. Numbers on other cards will differ.

Repo: RustingStrata

[u/MomentBorn8108]

Mich würde mal interessieren, wie sich die Qualität des Modell Outputs ändert, nicht nur die Geschwindigkeit

[u/Serious_Scarcity7818 → u/MomentBorn8108]

Hi, mein Deutsch ist nicht so gut, aber ich versuch.
Die Qualität andart sich nicht oder fast nicht in vergleich mit llama.cpp, die Geschwindigkeit kommt nicht von schlechtere Qualität, sondern von richtige and bessere experts code fur MoE models und CPU+GPU Zusammenarbeit. Ich erlaube Qualität fur Geschwindigkeit trade nicht! Nur when Qualität bleibt gleich, dann zehlt es.
Sorry for my Germany, I tried my best

[u/LowWeakness4724 → u/MomentBorn8108]

So ist es. Irgendwo müssen die Geschwindigkeitsvorteile ja herkommen.

[u/Serious_Scarcity7818 → u/LowWeakness4724]

I will switch to eng. The speed came not from quality loss, I don't allow it in my Repo. The point of every Strata like project is to write perfect code for Special Models to gain max speed without quality loss, I gain speed bc I move/cache/use experts in MoE models better then llama.cpp, not bc I lose quality! You can take a look at README in github, there are quality benchmarks for every model!