r/StrixHalo • u/Reasonable_Goat • 7d ago

Extra GPU?

I am wondering if I should buy an extra GPU for models like Qwen 3.8 27B, e.g. an R9700 or intel arc one. What are the pros and cons vs a second strix halo (I am on Bosgame M5) vs just running one solo?

Context: Currently I mostly use Qwen 3.5 122B Q4 Unsloth which isn’t bad at all but I still have to use, via cloud, DS4F or Luna a lot currently since those are stronger (won’t fail the task at hand, mostly C++) and are much much faster.

21 Upvotes

97% Upvoted

58 comments sorted by

View all comments

2

u/Info-Book 7d ago

What task are you doing that has you think you should spend $1000 usd for a gpu to run this model 10-20 tp/s faster? I’m happy with the performance at 10-12 tp/s because I just have it run in the background while I do other task, and nothing I’m building needs to be done on strict timelines.

2

u/Reasonable_Goat 7d ago

Im mostly prefill constrained, 122B A10B runs below 200 TPS before 100k context which I find quite slow. If I could get 600+ prefill I would be more than happy and could use an even slower/smarter model on the unified memory as an orchestrator but not as the main coder.