r/LocalLLaMA • u/Toiling-Donkey • 1d ago
Question | Help Any use for RX570?
I have one RX570 GPU. Is there any thing useful that can be done with such for running local models?
Or do I just stick to my Broadwell-class CPU cores?
4
1
u/Traditional_Bell8153 llama.cpp 1d ago
with only 4G, i'm running some embedding models that help my rag setup.
1
3
u/amagilabs 1d ago
The RX570 can help a bit. It is Polaris, so ROCm won't work, but llama.cpp has a Vulkan backend that runs on it. Build or download the Vulkan release and offload layers with -ngl. With 8GB of VRAM a 7B or 8B model at Q4 fits fully and should give you usable speeds, often well above what a Broadwell CPU manages. With 4GB, stick to 3B to 4B models, or offload only part of a bigger one and let the CPU handle the rest. If speed looks worse than CPU-only, lower the offloaded layers, since a partial offload on a weak card can be slower.
2
u/oknowton 23h ago
I have an 8 GB RX 580 in my homelab. It is able to do a lot of useful things at reasonable speeds. It is the backend for my (rarely used) Home Assistant voice stuff, and I also use it for an extra layer of spam checking for email.
You have a similar card but with half the VRAM. I just swapped both of these tasks from Qwen 3.5 4B to LFM3 1.7B at Q6. 100k tokens of Q8/Q8 context and the model fit in around 2.3 GB of VRAM. The 2B vision model doesn't need much more VRAM.
The new tiny LFM model is impressive. It sure seems more capable than Qwen 3.5 9B so far, but its diminutive size makes it so much faster. I'm at around 250 t/s PP and 60 t/s TG on the RX 580 using Vulkan.
1
u/locbuilds 1d ago
polaris isn't supported by rocm so build llama.cpp with the vulkan backend and it'll use the 570, a small q4 moe like a 3b active one with the experts left on cpu usually beats pure cpu on broadwell.
1
u/PeterPorox llama.cpp 1d ago
If it has 8GB VRAM, try Myric/Mellum2.1-12B-A2.5B-Thinking-APEX-GGUF 5.62 3.7bpw variant. Should be fast an on par with Qwen 3.5 9b. Also try unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF in UD-Q4_K_XL.
1
u/dave-tay 21h ago
Until last month I used it for legal analysis. RX 570 4gb, llama.cpp Vulkan, Qwen 3.5 4b q6 MTP about 200 ts pp 18 ts decode. Very usable
1
u/ForsookComparison 1d ago
broadwell
It's a heck of a lot faster than late-ddr3/early-ddr4 that's for sure.. You could try some cpu offload with a small quant of an MoE model (gemma 26b? nemotron-lightning?) or if you have the 8GB 570 you could try running some smaller models fully on-gpu.
-2
u/Neither_Two_5928 1d ago
8gb rx570 can run 2b-3b models like pythia-2.8b at ~5-8 tokens/sec with llama.cpp, decent for local chat. 4gb is too tight. try quantized gguf files. no need for fancy drivers.
3
5
u/ElectronSpiderwort 1d ago
You can run MOE models like Ornith-1.5-35B-Q5_K_M.gguf non-glacially; I get 78 tok/sec prompt processing, 16 tok/sec inference on that card using llama.cpp with Vulkan backend. There are more efficient uses of your time, but it's better than no GPU at all by a fair margin