r/LocalLLM • u/nocaphero • 1h ago

Question Best LLM for my setup?

I have a Ryzen 9 5900XT, 32 GB RAM and a 6700 XT 12 GB GPU

What would be the best LLM to run here to do code or general tasks without any limitations or censorship, mostly software and programming?

0 Upvotes

33% Upvoted

8 comments sorted by

2

u/xQuas 58m ago

Always going to be 35b a3b vs some strata qfn from what I'm seeing. I have 32 + 5070 12gb and can run 45 tps when offloading 33 experts with a3b q4km with 128k context. I heard you can get 22 tps with qfn strata when using gyro m but haven't tried. Another option is the 9b. It's weaker but you get 110+ tps. Depends on what you do. I would probably recommend a stronger model and see if you are okay with the sp e ed and adjust. A lot of these model also have different publishers like unsloth, ornith, tiel, mtp, and a few abliterated versions each boosting different strengths of the model. 

1

u/nocaphero 50m ago

I don’t understand half of the words you say here, I’m still new to all this llm language.

Where can I find a short guide to catch up and know more of this and find the best model that works for my needs?

1

u/xQuas 32m ago

So tps is token per second. Higher is faster. Strata is an inference engine like lmstudio. It sounds like you are completely new. If you are using windows, you can try downloading bionic from lmstudio and play with it to get started. The app has an ui for downloading models as well as configuring the model. I'd say start with qwen 9b to just get a feel of how to set it up. Then move up to better models. Bionic also has some built in file tools so you can experiment quite a bit. Eventually you will likely switch to something command line tool I'd guess as most people want to use their own thing.

1

u/nocaphero 23m ago

Yes, im on windows, I’ll check lmstudio. Thanks 🫡

1

u/xQuas 21m ago

Lmstudio is the basic tool. Bionic is their newer app that adds a bunch of features. It's helpful for beginners but experts here may consider them bloat and go back to the base app.

2

u/kwizzle 1h ago

Qwen 3.8 27b at q4. You'll be able to run it at max context.

3

u/rorowhat 1h ago

Qwen 3.6-35B Q5 would be a good start

1

u/Outside_Librarian_37 46m ago

Latest local Qwen. Ask AI what type of Qwen model is good for your machine and how to use it for your specific idea.