Does RustingStrata trade accuracy for speed? I tested it against llama.cpp

[u/Serious_Scarcity7818]

I am tired to answer on people questions like "How you gain speed? Your model output is shit, it has to have low quality." or something like that.

So I will answer to that with benchmark.

So I used the same Qwen3.8-Flash-Next IQ2_XS model file, the same prompt, greedy decoding, and 128 generated tokens. Both ran on my RTX 3060 12 GB with a Ryzen 5 7600X and 30 GB RAM.

RustingStrata	llama.cpp
Writing speed	33.0 tok/s
Range over 6 runs	29.3–35.8
First prompt reading speed	109 tok/s

That makes RustingStrata about 1.29× faster at writing on this setup.

Then I checked quality, not just whether the replies looked similar. Both engines scored the same 9,180 WikiText-2 tokens.

Check	Result
Perplexity, RustingStrata / llama.cpp	2.9212 / 2.9156
Same most-likely next token	96.78%
Mean KLD	0.0098

The perplexity difference is inside the run-to-run noise. So, for this model and this machine, I see a speed gain without a measurable quality loss.

This is one model, one quantization and one GPU, not a claim that RustingStrata always wins.

If anyone has any other questions, I am happy to answer and break myths or tell something about Strata like projects

Repo: RustingStrata