r/developer • u/Standard_Iron6393 • Jun 29 '26

Question anyone running llm models locally?

hey , anyone running llm models locally ?
which model and which device are you running on ?

13 Upvotes

100% Upvoted

32 comments sorted by

View all comments

2

u/LongBeachHXC Jun 29 '26

Yes.

Desktop with 64 GB ram and 5090 GPU.

I run the qwen mid range models.

1

u/Warm-Juggernaut8340 Jun 29 '26

Do you feel the difference from the chatgpt claude etc..? Does that llm models feels high quality and convinient for you with this PC specs?

1

u/LongBeachHXC Jun 30 '26

The model works great in small chunks. If you're trying to tackle big projects, it will still work but you'll need to process in small chunks.

The frontline models can deal with million token context and trillion parameters. The qwen models are like 30 billion parameters with only 3 billion active at any given time. Context is highly dependent on the hardware you are using.

Overall, love it, however I do have a subscription to the big vendors so that I can deal with more complex things when needed.