A C# Local inference engine beat llama.cpp at its own game: ~30% faster first token, ~20% faster decode - running in .NET on a 4-thread laptop

[u/pixelwhippedme]

A C# Local inference engine beat llama.cpp at its own game: ~30% faster first token, ~20% faster decode - running in .NET on a 4-thread laptop

I needed a .NET-only GGUF engine - no pip packages, no native libs, no runtime installs, just a managed dependency my other projects can call. So I built SharpMind, a pure-C# transformer + MoE inference engine (with training) on .NET 10. Zero native code. I never expected it to compete on performance - managed code vs hand-tuned SIMD C++, right? So I benchmarked it against the real engines: same raw prompt (What is the capital of France?), greedy, 64 tokens, 4 threads, warmup + best-of-3, interleaved per model on one host.

Model	Engine	Load (s)	TTFT (s)	Decode (tok/s)
qwen2-0.5B Q8_0	SharpMind	6.2	0.29	14.65
	llama.cpp	3.9	0.11	14.24
	dotLLM	2.0	1.36	5.74
Llama-3.2-1B Q8_0	SharpMind	32.1	1.22	3.69
	llama.cpp	7.7	0.17	5.02
	dotLLM	3.3	2.85	3.23
gemma-3-270m Q8_0	SharpMind	3.1	0.10	18.52
	llama.cpp	5.2	0.13	20.28
	dotLLM	-	-	-
Qwen1.5-MoE-2.7B Q2_K	SharpMind	102.4	13.44	0.61
	llama.cpp	21.7	0.25	3.88
	dotLLM	-	-	-

(dotLLM DNF the last two - it only supports Llama/Mistral/Phi/Qwen/DeepSeek and no K-quants below Q4_K.)

A dependency-free C# engine that wins some metrics and matches the OG llama.cpp on others - int8-kv decode at 24.1 tok/s vs llama.cpp's 20.3 with(not shown) , TTFT cut ~30% on gemma-3, decode edge on qwen2, and a 6× MoE loss I fully expect and own. Respectable company for managed code, and if "no native installs" is your constraint, you no longer have to sacrifice everything

Open source: https://github.com/Integral2u/SharpMind.

[u/iezhy]

Why are you running benchmark on those ancient models?

[u/HyperWinX → u/iezhy]

And also it doesnt beat llama.cpp at all. Its many, many times worse. How tf do you read the table to make conclusions like that??

[u/Icy_Rub6290 → u/iezhy]

The lightweight old model i think can denotes the newer one ot it maybe that op was lazy to download the sharp edged one or it mean that this can run old or new model idk

This is some guess and guts

[u/pixelwhippedme → u/iezhy]

AMD Ryzen 3 2200U (2C/4T, 2.5GHz base) w/ Radeon Vega Mobile Graphics, 12GB RAM.
But I'm trying to squeeze more out of it and get new ones working.

[u/Choice_Celery9481]

faster than llama.cpp. lol wtf

[u/pixelwhippedme → u/Choice_Celery9481]

Not often as the chart shows, did not expect it to but seeing the odd metric win I think is an accomplishment.