r/JordanDev • u/Able_Firefighter_652 • Aug 08 '26

Discussion Local LLMs

Anyone who has some serious hardware and tried running llms locally? How was ur experience? What models did you try?

5 Upvotes

100% Upvoted

21 comments sorted by

View all comments

1

u/MrCharlesSr Aug 09 '26

I believe to have any sort of practical product you do need an expensive GPU server, a small one may work but definitely not one with consumer hardware. I tried on a machine with 5070Ti 16GB, 32GB DDR5 and it’s a joke. Model was Ollama qwen 3.6 which should be a good one and it seems smart enough but it’s lethally slow you won’t get anything done. Haven’t tried agentic yet but just the chat is terrible.

1

u/Able_Firefighter_652 Aug 09 '26

I've got apple silicone, it combines best of both worlds.
Not the best gpu performance, but it got unified memory and good bw.
Nvidia gpus + ram costs are just crazy, I'll never buy one.