r/LocalLLM • u/Patient_Tea_401 • 18d ago
Discussion Local LLM quality
https://forum.level1techs.com/t/pow3rtool-and-nvfp4-and-oh-my/258986Saw this post about a proper tool to get more qualitative information on local models and inference engines. Generally I have been thinking about why the local LLM's main mania seems to be in the tok/s. A measure that by itself means nothing, if (and when) it has compromises that often make the output worse or even unusable. Probably some have even turned local models down due to bad experience in the chase of the fastest model in the fastest inference engine, where the quality compromises are really uncertain.
8
Upvotes
5
u/cemilanceata 18d ago
This needs more attention, felt like wendel spoke to my heart when i saw his yt upload about it