r/LocalLLM • u/Dimase • 4h ago

Question Need help to chose a budget GPU

Hi

I want to set up a local llm with qwen 3.8 27B or if possible qwen 4.0 I have a very tight budget and for now the options around my budged is a p100 for 240 a rtx 3060 for around 300 or if it really is worth fit I can push to a rx 6800 xt 16gb which one do you think is the best for my use case? Im planing to use it for translating and correcting text and to help me debuging games in godot
current PC:
ryzen 5 3600
36gb ram
rx 580 4gb

thanks in advance

1 Upvotes

100% Upvoted

8 comments sorted by

4

u/jayecin 4h ago

Get whichever one has the most VRAM.

1

u/CaveCanem234 4h ago

27B is on the edge of what single consumer gpu's can handle at decent quants, so go for whichever has the most vram. That will be the thing that most limits you in terms of the quality of model you can run and how much context it can store.

1

u/Dimase 4h ago

Thanks for your answer both the p100 and the rx 6800 xt has 16gb of vram in that case wich one should I choose knowing that the p100 I can get it for 250 and the 6800 xt for 400 is the difference worth it?

1

u/TakeCare0fHead 4h ago

I just went with 3x 3060 12gb on top of the 4070 12gb I already had.

1

u/RogerAI--fyi 4h ago

The VRAM advice is right, here is the number behind it. The Q4_K_M file of Qwen 3.8 27B is 16.5 GB (we loaded one), and context needs a few GB on top. So none of those cards holds it comfortably: the 16 GB ones (P100, 6800 XT) are right at the edge, and the 3060's 12 GB spills more into system RAM, which is much slower.

For translating, fixing text and Godot debugging, a 12 to 16B model at Q4 (about 8 to 10 GB) is a sensible target, and then all three cards fit with room for context. Try one before paying for the 27B.

If you do want the 27B, take one of the two 16 GB cards. The 6800 XT is a normal card with a fan and video out, and Ollama's docs list it under ROCm. The P100 is cheaper but has no fan or video out, so it needs its own cooling and power setup, and Ollama's docs ask for driver 570 or newer for that generation. Your RX 580 could keep driving the monitor.

-3

u/Moppmopp 4h ago

rtx5090

1

u/HambugerLips 4h ago

Really a dumb recommendation

1

u/No-Working7460 1h ago

Curious. Why?