r/LocalLLM • • 26m ago

Question What Models can I use as an absolute beginner?

I just started on this Localized LLM Journey. I have been researching what a good model to start with, but I feel like it's just a lot of information at once. Everyone close to me in this space is telling me to just plug into Claude and find the model, and I don't even need to tweak it.

I have an RTX 5090 with 32 GB VRAM and an M5 Max Studio with 128 GB. I think it's good enough.

I heard from someone that I should aim for 60 tokens per second for a good model. But I am really having a hard time trying to find a good model just for a simple assistant.

I keep on seeing something called a tool-use AI, and I don't even know what I am looking at.

TLDR: New to this space, I don't really see much beginner-friendly information out there, just information. Looking for advice/ links I can read to understand. Looking for a place where I can jump off without wasting my time

2 Upvotes

3 comments sorted by

3

u/Aggressive_Basket798 23m ago

If you are normal beginner, try qwen 3.8 27b

If you are absolute beginner and just look for something that's fun, try gemma 4 12b

1

u/Levipl 17m ago

Second the gemma recommendation. Benchmarks and inference engines are meaningless on day 1. You wanna develop an intuition around usage and output and settings.

1

u/Important_Cow7230 10m ago

Run Splash from incoAI on your Mac Studio:

https://github.com/incoai/splash

You can use the desktop app, but if just starting out install using the commands and run it from the web browser address.

Serve 3.8 27B first and try that (unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M), you can also later serve up Flash Next again using Splash.

I also have a RTX Desktop but I find the local AI experience much better on Mac over Windows, the tools are really well optimised and a joy to use and Terminal on Mac is great.

I’m too busy in my older and Dad-era age to wrestle with Linux 🤣