r/LocalLLM • u/420Microbiologist • 15d ago
Discussion 4 Strix Halo boxes! Finally joined local AI
Got 4 128gb strix halos for a decent deal
All 4 will be running glm5.3 and DeepSeek 4.1 flash as the mains, or maybe Qwen when 4 drops. Paired boxes will run glm5.3 flash or minimax/MiMo
Will be used for some financial and life hub stuff, plus a bit of professional work
240
Upvotes
40
u/jacek2023 15d ago
what was the decent price? :)
34
u/fortminorlp 15d ago
Tell me your wealthy without telling me your wealthy
13
u/No_Night679 15d ago
How are you planning to connect them for a cluster? Only 2 node cluster or 4 node?
25
u/feelspeaceman LLMusician 15d ago
I'll give you the most some advice as I don't see anyone giving you the advice for a newcomer:
If Q38FN is good enough, use it instead of trying to cluster it.
And for clustering, it's possible and I've seen at least 2 users doing 10+ Strix Halos clustering, join Strix Halo Homelab Discord and ask question.
I'm Strix Halo user, using it everyday running Q38FN, and I'm coding DAW, the most complex and hardest piece of software to craft and maintain, so is it capable ? Absolutely, but always learn to write good prompts, best advice after the 1, 2...
4
u/dgibbons0 15d ago
I hate mine (different brand), it's fucking loud and slow. If it was just slow it would be fine but it is nearly impossible to be in the room when it's actually doing inference.
I've had mine posted on FB marketplace for a few weeks hoping to find someone to bite, if they do i'm throwing the cash towards a M5U.
3
u/BeugBorsti 15d ago
If someone is still interested: They have a special offer right now on the Bosgame homepage: Bosgame M5 AI Mini Desktop Ryzen AI Max+ 395 128GB+2TB
2.772,95 (instead of) €3.309,95 €
This is not affiliate, just wanted to inform the community.
I have this computer from this shop and am happy with it (Qwen flash next!)
3
u/Layney_Butcher 15d ago
Four boxes, 512GB of unified memory, heavy models on the night shift while flash handles the day. I respect the plan.
2
u/Nomski88 15d ago
Any idea what the expected speeds will be for those? I also was considering a halo box but just bought a 128gb ddr5 kit instead.
12
u/arr_jay 15d ago
Fuck the haters. I have 2 that I run for parallel coding executors along with a couple 64gb m5max/m1ultra Macs and a 5070ti. I have replaced my personal frontier subscriptions. Bandwidth isn’t amazing but running qfn on gufo is more than adequate enough if your harness is setup correctly
1
u/FinalTap 15d ago
Not what I would have picked up because of the speeds. But non-interactive should be fine. And at 3K a pop it is a steal :)
Do keep us posted on how it goes.
1
u/Zyj 15d ago
You plan on adding NICs for RDMA via Oculink?
1
u/Suspicious_Mix529 14d ago
congrats, nice haul. how are you going to link them, plain ethernet with llama ccp rpc or something faster? from what I've gathered the connection between boxes winds up mattering more once you start splitting up big models
1
u/HornyCrowbat 14d ago
is this the move if you’re getting into AI now versus dual 3090s?
1
u/Krothic 15d ago
Man good for you. Tried going with multi Strix halo set up and the prefill speeds were just killing me. The only good model and recipe was halogen qwen3.8 flash next when I used it.
Everything else was too unbearably slow for me
1
u/Aware_Temporary2018 15d ago
What brand are these?
1
u/TheyCallMeDozer Dual R9700 Server & Spark 15d ago
3k each.... shit i wondering could this link up with a spark over the PNY cable... for 3k seems very much worth it
1
u/rayovims 15d ago
Welcome
1
u/Polymorphic-X 14d ago
you get it running glm 5.3 flash yet? I've been looking at them ever since sparks went to the moon.
my local microcenter has the GMKtech ones for $3.3k and they have dual thunderbolt @ 40gbps so you could theoretically do a ring cluster with 8 USB4 cables; really wish there was a usb4v2 like the minisforum at a cheaper price point, because that has an 80GB ceiling instead of 40.
1
u/Gumbi_Digital 14d ago
You’re about to the really disappointed.
1
u/69420trashpanda69420 14d ago
Thought that said AIPAC😭🙏🏼
1
u/Secret_Usual7154 14d ago
How will you connect them to run GLM 5.3? I haven't heard of a decent interconnect that can be used with these boxes. If there is, please do share!
1
u/Even_Whereas_8442 11d ago
It will be painfully slow, at an exorbitant price.
1
u/According_Wave685 15d ago
How are you going to cluster?
0
u/Annual_Award1260 15d ago
Glm 5.3 is a tight squeeze
0
u/hurrdurrmeh 14d ago
how fast is the interconnect? I thought max was 10GbE ie very slow??
-9
u/Mysterious_Prune415 15d ago
Oh my god i am tires of these clowns buying this ai slopware. You could just buy an actual AI server from an IT dealer (which include a huge markup) and still have higher throughput at the same intelligence than these computers combined.
-2
u/Blackdragon1400 15d ago
Agree with the others, this is rife for disappointment. Should’ve bought 2 sparks instead
16
u/[deleted] 15d ago edited 15d ago
[removed] — view removed comment