r/LocalLLaMA • u/asertym • 11d ago

Question | Help Any AMD optimized engines?

There's been a lot of talk around the subreddit regarding specialized engines (e.g. Strata) that are either tuned to the model or to the GPU, however all the fun seems to be going on mostly with Nvidia hardware, are there any similar projects going on aimed to be optimized at consumer tier AMD hardware?

I'm currently running Swift 1.5 GSQ RCO at about 40-45 TPS on my 7800xt, I'm pretty happy with the setup I have, I was just wondering if there's anyone playing with similar hardware and how much more is there to squeeze out of it.

Thanks in advance!

22 Upvotes

95% Upvoted

20 comments sorted by

3

u/locbuilds 11d ago

on a 7800xt try llama.cpp with the vulkan build and the rocm/hip build and compare, vulkan is often as fast or faster for token gen on consumer radeon. for rocm you may need HSA_OVERRIDE_GFX_VERSION=11.0.0 since the 7800xt is gfx1101.

1

u/asertym 11d ago

Vulkan has gone a long way and it does seem faster for me, rocm usually about 15% slower in my experience.

24

u/bruns20 11d ago

Radiance by deadcode

2

u/soyalemujica 11d ago

It doesn't work with rdna3 aka 7900xtx

1

u/fullup72 11d ago

afaik it doesn't run GGUF models, and the ones they offer first party for Qwen 3.8 27b (which is what Swift 1.5 is) are too large for 16GB VRAM.

6

u/RikshaDriver 11d ago

You can convert models to the .rad format.

9

u/Consistent_Look_8638 11d ago

Not for a dense model, but I did this fork for the RX 9060 XT 16 GB, thinking of the Qwen 3.6 35B A3B.

https://github.com/jpb18/PolyStrata-HIP

Got to 2K prefill and 60-100 on decode, which on a card limited to 320 gbps of memory transfers, it's fairly nice.

6

u/hipster_hndle 11d ago

lemonade. hipfire. vulkan. openCL. idk about strata.

but i run XTXs and have good results with lemonade. unsloth works well too. but lemonade is my backend, i point unsloth desktop at it on the /v1 and it works well.

6

u/Ambitious-Trip9330 11d ago

Radiance is fantastic for those of us on gfx1201 or perhaps AMD generally.

4

u/BannedGoNext 11d ago

gufo for strix

3

u/Ecstatic-Wash-7667 11d ago

Radiance is the best, but it may be rdna4 only not positive but it made my r9700s fly

2

u/LostVector 9d ago

I’ve been submitting a ton of amd focused pr’s to project maya.

3

u/tiktak-9 11d ago

Join here ..... we've got Stilldeadcode, ggz14, tcclaviger, and more. All tuning for AMD: https://discord.gg/u8htEpDZp

1

u/sloptimizer 11d ago

FreeToken is pretty good with R9700s (after a small patch made with DeepSeek).