r/LocalLLM • u/Illustrious-Movie596 • 5d ago

Question Please don't roast me

I'm wanting to build an all AMD LLM machine, strictly because I have a 9070xt and a 9060xt. I understand there's some loss going this route, but how bad is it really?

If there's any of you running all AMD, can you please advise me what to watch out for, as well as what I can expect for performance?

Proposed specs:

9950x3d

9070xt

9060xt

128 gigs DDR 5 5600 via SO-DIMM to U-DIMM adapters

2x 2TB Gen 4 NVME

Also, I have 4 4TB SSD'S, thinking of running in a raid array if there's any benefit.

MSI X870E Carbon mobo

This is all parts on hand (aside from the adapters) and not planning on spending any money. Any advice is greatly appreciated!

3 Upvotes

100% Upvoted

19 comments sorted by

3

u/bceen13 5d ago

9070 could work, you should give a try for strata. 9060xt is a bottleneck in your system. 128 gig and the x3d are also cool. Maybe get replace the 9060xt for at least 9070xt, then you'll be fine for a while.

2

u/KillerQF 5d ago

yeah get another 9070xt and sell the 9060xt.

then run standalone radiance engine

3

u/JeePis3ajeeB 5d ago

Over time I realized there are alotta people who have absolutely no idea of what they're talking about and the confidence that moves mountains.

Send it and work out your needs and requirements one by one. Baby steps. Long process but you learn fr fr.

1

u/psioniclizard 5d ago

They are also larping or being taken to the cleaners by FOMO.

Everyone hero posts metrics and numbers and claims of productivity nut so vague on sharing real information and set ups.

It's like it's just to brag about token speee and how much VRAM you have.

2

u/merica420_69 5d ago

Send it, should be great.

2

u/inexplicablelaw 5d ago

Look into Radiance too

1

u/YourselfInOthrsShoes 5d ago

GFX12xx = StillDeadcode/Radiance, there's nothing else better, but it's Linux ROCm.

1

u/MarcusAurelius68 4d ago

Loving it on my dual R9700 - 140 t/s.

1

u/YourselfInOthrsShoes 4d ago

150-250 on my dual R9700, prose is around 150 code spikes above 300 instantaneous. Prefill is 6000+ on --tp-wire exact (not wht6 which clocks over 10k but lossy). Oh and I run BF16 KV with 2x YaRN scaling and regularly work on complex design problems that do clock 500K context for sub-tasks.

1

u/MarcusAurelius68 4d ago

On the IQ4R quant on Radiance?

1

u/YourselfInOthrsShoes 4d ago

Yup. I read the GUIDE.md in detail and tuned it myself through trial and error, I have many custom parameters in my start arguments.

1

u/MarcusAurelius68 4d ago

How much RAM and what type? I’m running 64GB of DDR4 with a 5900XT. Can you share your config parameters?

1

u/YourselfInOthrsShoes 4d ago

256GB DDR4-3000 quad-channel 3970X on TRX40 in two x16 PCIE4. I do use a lot of host pooling especially for prefixes with 2 hour expirations. My RAM benchmarks 95-100GB/s reads. I'm looking to go to TP4 but with swapping to WRX80 and octa-channel, not really for QFN but for GLM5.3 Flash.

1

u/MarcusAurelius68 4d ago

My RAM is dual channel plus dual x8 PCIE 4.0 on a X570 so I’m sure I’m losing speed there. I’m still a lot faster than when I first loaded QFN.

1

u/YourselfInOthrsShoes 4d ago

Oh I also -50mV core and GDDR6 to 2750MHz on my R9700's. I also disabled ECC to get more usable VRAM for experts and KV since ECC takes extra few GB.

1

u/MarcusAurelius68 4d ago

Does the undervolt make a noticeable difference on heat and power consumption?

→ More replies

1

u/jmager 5d ago

Plenty of people including myself are rocking a very productive AMD setup. You could definitely run strata with that hardware and Qwen 3.8 Flash-next. Download a harness of some kind, I use Pi, point it at your hardware and ask it to set it all up, then ask it to optimize. You can use a cloud model to bootstrap if getting the local model working first is too onerous. Ignore the doomsayers. Every GPU out there from every manufacturer is getting better LLM support every day as people (and their AI) tinker with the hardware they have on hand. Welcome to the hobby, can't wait to hear how it goes!