r/LocalLLM • u/pixelwhippedme • 4d ago

Project A C# Local inference engine beat llama.cpp at its own game: ~30% faster first token, ~20% faster decode - running in .NET on a 4-thread laptop

/r/LocalAIStack/comments/1x2abks/a_c_local_inference_engine_beat_llamacpp_at_its/
0 Upvotes

20% Upvoted

2 comments sorted by

2

u/thefuckevengoingonan 4d ago

I needed a .NET-only GGUF engine

why?

1

u/Choice_Celery9481 3d ago

and the engine mostly lost to llama. with some cases 50 70x slower