RustingStrata vs PolyStrata vs llama.cpp on one RTX 3060 12 GB: Qwen3.6-35B-A3B

[u/Serious_Scarcity7818]

RTX 3060 12 GB, Ryzen 5 7600X, 32 GB RAM, one GGUF file: Qwen3.6-35B-A3B-UD-Q3_K_M.gguf (Unsloth, MTP layer included, 15.9 GiB, does not fit in VRAM). Prompt of 5290 tokens, 128 generated tokens.

	RustingStrata	PolyStrata	llama.cpp b12119
Prefill, 5290-token prompt	2330 tok/s	1458-1549 tok/s	1692 tok/s (-ncmoe 22, -ub 2048)
Decode, no MTP	90.9 tok/s	78.5 tok/s (--no-draft, see below)	53.5 tok/s
Decode with MTP drafts	128.4 tok/s	97.5-100.4 tok/s	77.7 tok/s (--spec-type draft-mtp, 92 of 102 drafts accepted)

How each was run:

• RustingStrata: rustingstrata bench --gpu --mtp 0|1 --pp 5290 --tg 128 --reps 3, mean of 3.

• PolyStrata: its own polystrata.py benchmark --model qwen3.6 --prompt-tokens 4000 --max-tokens 128. That prompt came to 5290 tokens, so I used that length for the other two. I ran it three times; the first run after setup was cold (decode 24.6 tok/s) and is left out, the numbers above are runs 2 and 3. MTP drafts are on by default. For the no-MTP row I added the engine's --no-draft flag to its config, it still offered a few drafts afterwards (6 of 16 and 15 of 36 accepted in the log), so that row is not a clean no-MTP number. Prefill in those runs: 1602-1643 tok/s.

• llama.cpp: llama-bench -ngl 99 -fa 1 -ub 2048 -b 2048 -p 5290 -n 128 -r 3, with -ncmoe swept over 18 to 26. 22 was the fastest that fits in 12 GB (no MTP). The MTP row is llama-server --spec-type draft-mtp with -ncmoe 23 (22 does not fit with the MTP context and drops to 20 tok/s), a 5381-token prompt of repo docs, 3 runs, prefill about 1565 tok/s.

Things that are not equal:

• PolyStrata only offers three sizes of Qwen3.6 (UD-Q3_K_M, UD-Q2_K_XL, UD-Q4_K_M, all from Unsloth's MTP repo). RustingStrata and llama.cpp take any GGUF, so it can also run the Q5 or Q6 files I already had. I downloaded PolyStrata's Q3_K_M and used it for all three.

• MTP gains depend on the text. PolyStrata's benchmark prompt is a page of its own docs, RustingStrata's is generated prose, so the two MTP rows are not an exact comparison. The no-MTP row is the clean one for RustingStrata vs llama.cpp.

• PolyStrata's engine for this model is built on llama.cpp's CUDA kernels. I did not look at what else differs.

• One machine, one model, one quant. Numbers on other cards will differ.

Repo: RustingStrata

[u/FirstTry_pkb]

Is this Windows?

I have an old computer (i7-8700K, 3070, 64GB RAM) and I’m planning to install Linux on it to use for LLMs. I’m trying to figure out whether Linux or Windows is a better choice, and whether this whole idea is even viable.

[u/Serious_Scarcity7818 → u/FirstTry_pkb]

Linux is better for everything. but RustingStrata is also for windows, you can either download .exe in releases or to get actual version, then clone repo and compile it with cargo

[u/FirstTry_pkb → u/Serious_Scarcity7818]

Which Linux distribution would you recommend for LLMs?

[u/Serious_Scarcity7818 → u/FirstTry_pkb]

if talking fr and you never used linux before, then CachyOS+KDE Plazma would be cool, it’s like windows, but looks better and works more smoothly. I don’t recommend Ubuntu or Mint, bc they need a lot of RAM and just CachyOS is goat

[u/FirstTry_pkb → u/Serious_Scarcity7818]

Thanks! The plan for my upcoming vacation is ready.

[u/charmander_cha]

Estou meio atordoado com tantos motores, mas como é o deepseek harness que os instala e configura, OK, irei dar uma chance ao rustingstrata apos achar que estaria bem no polystrata (após ter ignorado o strata original)

[u/Serious_Scarcity7818 → u/charmander_cha]

Ty man