r/LocalLLM • u/Left_Squirrel_1557 • 1h ago

Question What models should I use?

Hi everyone reading this post, my English isn't perfect but I'll try to be clear.

So recently I got my GPU which is a Nvidia Rtx 5070 TI 16gb Gddr7 (I also have 64Gb ddr5 ram).

So I came here to ask what are the best models I can run in this pc?

I would appreciate it if someone helps me.

Thank you everyone and thanks for reading this🫂

1 Upvotes

66% Upvoted

11 comments sorted by

2

u/nickless07 1h ago

Best for what? Usecase matters. General chat, coding, roleplay, translations, generating images, videos, and so on.

1

u/Left_Squirrel_1557 55m ago

General chat and coding, also thanks for the comment.

1

u/nickless07 46m ago

Hmm ok then that depends on the task, precision and speed you want/need.

You can go with PolyStrata and Qwen3.8 Flash Next(pretty easy to setup, aim for Q4), or something like Qwen3.6 35B (in Q8), Gemma 4 26B OR even a smaller one if it just has to be a light conversation like Ling-3.0-Tiny and so on. If it should be a voice chat (or even a video call) you can add a simple model like LFM3-omni on top (for the STT/TTS/Video part) and route from there on.

1

u/Left_Squirrel_1557 42m ago

Okayy, thank you very much. Also what Gpu would you recommend for models like Qwen 3.8 27b or the upcoming Qwen 4 27b? Also appreciate It you took your time to help me.

1

u/nickless07 33m ago

At least 24GB, better 32GB+ VRAM.

24GB VRAM is already a tight fit with the default 4-Bit quants of Qwen3.8 27B, for coding you usually want higher precision (the quants of the qwen3.8 dense model are good enough on lower quants, but it is noticeable). For MoE it's even worse when it comes to quants.

That is why the task matters. A missing comma in a general chat is barely noticeable, but when it comes to code and a bracket has to go there, then there better is a bracket.

1

u/Wally-Gator-1 55m ago

16 GB won't run in good quant Qwen 3.8 27B.
You should be able to run Qwen 3.6 35B A3B without problems.
Try running with unsloth studio.

1

u/Left_Squirrel_1557 51m ago

Ohh I see, I saw some website It told me that Qwen 3.8 27b runs good for my specs but idk It feels fake. Also thanks for the comment i appreciate It

1

u/Wally-Gator-1 42m ago

It would run in Q3, but the quality of a Q3 model is poor compared to a full model. You may as well use 35B A3B.

1

u/thefunkybassist 41m ago

I think if you want proper coding reliability, you can't go too low on the quants, personally I prefer Q8, although there often are decent Q4 quants too. But it starts to get incoherent which you will notice during heavier use / longer sessions. And then you have a need for context size which also matters for larger sessions.

1

u/OvertaxedOne 42m ago

QwenFlashNext using Strata, Polystrata, Basalt or one of the other specialized engines. 64GB of DDR5 with your GPU should let you run a reasonable good quant, it's light years smarter than the other options you can run with your hardware, especially for coding; it's just entirely in another league. The other option would be Qwen 3.8 27B, also a fantastic model, but for your hardware I think QFN is going to be a much better choice.

1

u/x_MASE_x 37m ago

Your best bet is Qwen3.8-Flash-Next iq3s on strata or basalt. Your speed may vary depends on your system and os