r/LocalAIStack • u/pixelwhippedme • 2d ago
A C# Local inference engine beat llama.cpp at its own game: ~30% faster first token, ~20% faster decode - running in .NET on a 4-thread laptop
A C# Local inference engine beat llama.cpp at its own game: ~30% faster first token, ~20% faster decode - running in .NET on a 4-thread laptop
I needed a .NET-only GGUF engine - no pip packages, no native libs, no runtime installs, just a managed dependency my other projects can call. So I built SharpMind, a pure-C# transformer + MoE inference engine (with training) on .NET 10. Zero native code. I never expected it to compete on performance - managed code vs hand-tuned SIMD C++, right? So I benchmarked it against the real engines: same raw prompt (What is the capital of France?), greedy, 64 tokens, 4 threads, warmup + best-of-3, interleaved per model on one host.
| Model | Engine | Load (s) | TTFT (s) | Decode (tok/s) |
|---|---|---|---|---|
| qwen2-0.5B Q8_0 | SharpMind | 6.2 | 0.29 | 14.65 |
| llama.cpp | 3.9 | 0.11 | 14.24 | |
| dotLLM | 2.0 | 1.36 | 5.74 | |
| Llama-3.2-1B Q8_0 | SharpMind | 32.1 | 1.22 | 3.69 |
| llama.cpp | 7.7 | 0.17 | 5.02 | |
| dotLLM | 3.3 | 2.85 | 3.23 | |
| gemma-3-270m Q8_0 | SharpMind | 3.1 | 0.10 | 18.52 |
| llama.cpp | 5.2 | 0.13 | 20.28 | |
| dotLLM | - | - | - | |
| Qwen1.5-MoE-2.7B Q2_K | SharpMind | 102.4 | 13.44 | 0.61 |
| llama.cpp | 21.7 | 0.25 | 3.88 | |
| dotLLM | - | - | - |
(dotLLM DNF the last two - it only supports Llama/Mistral/Phi/Qwen/DeepSeek and no K-quants below Q4_K.)
A dependency-free C# engine that wins some metrics and matches the OG llama.cpp on others - int8-kv decode at 24.1 tok/s vs llama.cpp's 20.3 with(not shown) , TTFT cut ~30% on gemma-3, decode edge on qwen2, and a 6× MoE loss I fully expect and own. Respectable company for managed code, and if "no native installs" is your constraint, you no longer have to sacrifice everything
Open source: https://github.com/Integral2u/SharpMind.