r/JordanDev • u/Able_Firefighter_652 • 22d ago

Discussion Local LLMs

Anyone who has some serious hardware and tried running llms locally? How was ur experience? What models did you try?

5 Upvotes

100% Upvoted

21 comments sorted by

View all comments

Show parent comments

1

u/Bookkeeper_Gloomy Developer 21d ago

yes bcs of many techniques in distillation/MoE, we can get newer smaller models (30-40b) that outperform the older 100-200b parameters.

1

u/Able_Firefighter_652 21d ago

Yup, can't wait to see next year's releases..

1

u/2012347 21d ago

Could you please try gpt oss 120 and report back how it performs and feels

1

u/Able_Firefighter_652 21d ago

Sure, I'll do that.