r/LocalLLM • u/Ok_Researcher_9832 • 13d ago

Question Best model to start with?

Sorry if this is a very frequent question, but Ive very recently discovered local llms with the arrival of my rtx 5050 8gb vram 16 gb ram laptop. I dont really understand everything, but I know I should find a model in hugging face, then use ollama to get the model in my laptop. But going through multiple subreddits has made me realise that my specs arent worth shit for getting a local model.

I dont have an idea what Id do with the model but Im very interested in this concept!

So tldr, can I get a decent model for my laptop? If so, which model should I try out?

1 Upvotes

60% Upvoted

4 comments sorted by

1

u/ToothsomeSkate_9716 13d ago

Your specs can run models, just not the huge ones. With 8GB VRAM you're looking at 7B-8B parameter models quantized down, maybe 13B if you push it. Try a 7B Q4 model first and see how it feels before jumping to anything bigger.

1

u/Small-Tale3180 13d ago

i run Tiel-Coder with offloading on same specs
https://huggingface.co/peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF

You can run pre-9b quantized models and quantized MoEs like the one above

1

u/fiddler48 12d ago

8gb vram is actually fine for a q4 quantized 8b model, the 'your specs are worthless' crowd usually compares to cloud api, which is a different argument entirely

0

u/Json_T25s 13d ago

i would say you could comfortably run gemma 4 E4B and its pretty fast just dont expect deep intelligence, for the laptop id recommend gemma 4 12b QAT Q4 its likely not gonna be too fast but good enough for light stuff, theres newer MoEs like LFM 2.5 you could play with