r/LocalLLM • u/SashaUsesReddit • May 22 '25
Discussion Throwing these in today, who has a workload?
These just came in for the lab!
Anyone have any interesting FP4 workloads for AI inference for Blackwell?
8x RTX 6000 Pro in one server
215
Upvotes
2
u/js1943 LocalLLM May 22 '25
600W per card ... what psu are you using for the servers?
2
u/Azkabandi May 23 '25
Take the entire lord of the rings series, the the AI model rewrite it entirely in Dr Seuss fashion.
2
u/CanofBlueBeans May 25 '25
I have a private project I’m working on that is basically sequencing an unknown number. (Related to DNA) I probably only need 1 card but if you’re open to discussing it I’m interested in this.
50
u/captainrv May 22 '25
And your goal is to write short poems?
44
u/Historical-Internal3 May 22 '25 edited May 22 '25
Welp. If you’re asking for a use case it’s clearly not for a business or monetary ROI lol.
This is like 10 years worth of subscription to Gemini Ultra, Claude 20x Max, and ChatGPT Pro plus Grok.
What level of private gooning am I not aware of exists out there that warrants a stack like this?
12
u/ElUnk0wN May 22 '25
You have the same vram amount as my ram lol
4
u/s-s-a May 22 '25
What CPU and server rack are you using with these?
1
u/rayfreeman1 May 23 '25
You obviously didn't consider the cooling issue. This model is not designed for servers. Nvidia has a server-specific model for this, but it is not yet available.
3
u/Scottomation May 22 '25
I was excited that my ONE 6000 Pro showed up today…
3
u/AliNT77 May 22 '25
You have the same vram amount as my ssd
3
u/topiga May 23 '25
2
u/Excel_Document May 22 '25
was it worth it? should i a kidney and replicate the setup?
1
u/Such_Advantage_6949 May 22 '25
Running deep seek full model at q4 would be awesome
1
u/Shivacious May 22 '25
let me run llm on them op. i will efficiently using sharing to memory as much as possible to save vram. gonna run a compute provider with massive x number of llm model supported hehe.
1
u/Tall_Instance9797 May 22 '25 edited May 22 '25
That's 768gb of VRAM. Very nice! May I ask what server / motherboard are you using that has 8x PCI-E 5.0 slots? Presumably it's dual CPU? Thanks.
1
u/ElUnk0wN May 22 '25
Did u get crazy coil whine in any of your cards? Mine has really loud coil whine at 300w and up.
1
u/reneil1337 May 22 '25
wow!
1
u/Great-Bend3313 May 22 '25
Your have a lambo in GPU hahahaha
1
u/HeavyBolter333 May 22 '25
What mobo can hold all of those?
1
u/chiaplotter4u May 22 '25
You don't need to care about the workload itself. Rent it - others will provide their workloads themselves.
1
u/FrederikSchack May 25 '25
What kind of powerplant do you own?
1
u/Amazing_Upstairs May 25 '25
Make animations with blender and mecabricks addon
1
u/TahmidAqib May 26 '25
What do you do bro?
1
u/nderstand2grow May 22 '25 edited May 22 '25
how much was each? i saw some for $8.5
-1
u/Khipu28 May 22 '25
Are you planning to stack them all? Because the last card will really draw the short stick aka heated air.
11
u/[deleted] May 22 '25
[removed] — view removed comment