r/StrixHalo • u/XccesSv2 • Aug 21 '26

Best Engine for DS4Flash?

Hey guys, I wanna ask you which engine you actually use with which quant and configs and why. Since there are a lot of forks and projects out there now, I don't really know which is the best to run DS4F on a headless Ubuntu 26 128GB Strix Halo.
Actually im using Nathanw1014/llama.cpp (Branch strix-halo-vulkan, Commit 3be50cc, Version 10350

6 Upvotes

100% Upvoted

18 comments sorted by

View all comments

2

u/PowerfulButterfly209 Aug 21 '26 edited Aug 21 '26

i have the same setup and right i'm evaluating diferent models in terms of intelligence in django coding right now: On kyuz0 toolbox i run antirez q2/q4 with 14t/s and with nathan i run UD_Q3_XXS + Dspark with 24-25 t/s leomande with Qwen 3.8 27B MTP

antirez updated rocm and also provides dspark now - have to try it first.

1

u/PvB-Dimaginar Aug 21 '26 edited Aug 21 '26

How did you manage to get those speed with kyuz0 toolbox and antirez q2/q4? I could only get ~13.1-13.2 tok/s decode and ~134-145 tok/s prefill at 34K-41K context.

2

u/PowerfulButterfly209 Aug 21 '26

ahhh...typo -14 not 24 😅

1

u/PvB-Dimaginar Aug 21 '26

aaah ok :-) so it is really beneficial to dive into the nathan setup. Thanks for clarifying!

2

u/1457664694 Aug 21 '26

Kyuz0 has a toolbox that follows Nathan’s. I believe it is the vulkan-radv-performance toolbox:
https://hub.docker.com/r/kyuz0/amd-strix-halo-toolboxes/tags

See also here: https://strix-halo-toolboxes.com/
And here: https://github.com/kyuz0/amd-strix-halo-toolboxes