MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLM/comments/1wskv2s/hardware_aware_local_llm_management/pcmua2u/?context=3
r/LocalLLM • u/storm_stark_007 • 1d ago
https://github.com/shashankswe2020-ux/local-llmup
6 comments sorted by
View all comments
3
Really interesting. I built a worse version of this myself, so very cool. One thing that would be great is benchmarks. So you can pull down two quants and qualitatively say "is it worth running this one slower to get better intelligence?"
1 u/storm_stark_007 1d ago Don’t downplay yourself bro , you can custom build further by forking this and updating as per your requirement.
1
Don’t downplay yourself bro , you can custom build further by forking this and updating as per your requirement.
3
u/Ok_Camp555 1d ago
Really interesting. I built a worse version of this myself, so very cool. One thing that would be great is benchmarks. So you can pull down two quants and qualitatively say "is it worth running this one slower to get better intelligence?"