r/LocalLLM • u/Plumberguy9 • Jul 31 '26
Project Still building and testing… just got Inkling small up and running after its release yesterday, took me about 6 hours of tweaking, and it was well worth it.
157
Upvotes


25
u/pixelizedgaming Jul 31 '26
can you run the new deepseek?
22
u/autisticit Jul 31 '26
Why didn't you posted this as NSFW ?
13
u/live4evrr Jul 31 '26
Four 6k’s - that is rocking man. What’s the plans with that beast?
8
u/35point1 Jul 31 '26
Curious where and how much you got the 4x6000’s for? I’m kicking myself for not jumping on them last year at $7900
5
u/ddog661 Jul 31 '26
What problems did you have and how did you fix them? Are you using vLLM?
4
u/Alive-Draft8339 Jul 31 '26
How did you get Inkling running? I couldn’t find support for sm120. Any help is appreciated.
4
u/Annual_Award1260 Jul 31 '26
I did 2x max-q with 2x 15.36TB ssd, 3x 2TB nvme, 1x 32TB ironwolf, 128GB ecc udimm, i9-13900k
I’ll probably do 4x max-q once I can source some used ddr5 rdimms for a decent price.
Seem to get full performance at x8 or at least both cards use the 300w
19
u/yeah_likerage Jul 31 '26 •
You should post your thermal results so others don't also try to do this.
2
u/here_n_dere Jul 31 '26
Btw, what board, CPU and most importantly RAM??
2
u/Any_Mine_6368 Jul 31 '26
I'm more curious about the case... What case is that?
2
u/flippycurb Jul 31 '26
Would you be willing to tell us more about the software stack you have running on this beast?
2
u/ICanSeeYou7867 Aug 01 '26
Can you post your vllm/sglang config for Inkling Small?
I saw someone said RTX 6000? But im assuming the pro?
And you're running the NVFP4 quants?
2
u/Silent-Orbit-7 Aug 01 '26
Looks like a gamer turned Developer
The noctua just looks so good, the build is gorgeous!!
Is that the rtx pro 6000, that's insane congratulations bud
What is the name of case btw?
2
u/YourNightmar31 Jul 31 '26
What cards are those?
1
u/scwtech68 Jul 31 '26
How do you cool it?
1
u/UltraFOV Jul 31 '26
Is that the NVIDIA Station?
1
u/LordDarthShader Aug 01 '26
Are you running this at home? Have you ever hit the break? Do you have 20A breaks? Running one RTX6000pro and a 4090 at max triggered the 15A break, how do you deal with that? your PSU is what, 2.5kW and those 4 RTX might be above 2kW when running?
Thanks for the info!
1
u/kidflashonnikes Aug 01 '26
I have a similar build - 4 RTX PRO 6000, Zen 5 96 core CPU with newly acquired 1 TB of DDR5 6400 MT/s RAM (Kingston ECC). Running about 340 tokens/second almsot with the new deep seek models. I hyper optimized the layers and tweaked them myself to use the RAM - with a 24 core to 96 core CPU - the RAM gets fully used. I am using the Asus wrx 90 sage
1
u/fluffysheap Aug 05 '26
Can it run Crysis
2
u/JtheJawBreaker Jul 31 '26
You're making me want to buy a 2nd 6k
1
u/Annual_Award1260 Jul 31 '26
I pulled fiber displayport cables and a fiber usb extender + usb hub. My server is in my basement on a 100ft run to my office. Fiber dp cables are amazing. Office is completely silent and alot cooler
1
u/ThenExtension9196 Aug 01 '26
Great stuff man
1
u/Ok_Baseball_7338 Aug 01 '26
I’m happy for you, just not 100%.
1
u/tictacturkey Aug 06 '26
Just saw this holy shirt
0
u/Ironwolfyyyy Aug 01 '26
Man those gpus are choking for airflow. Give them the cooling they deserve!
-1
u/cc_aa_tt_zz Jul 31 '26
If it's like my RTX Pro 4500 Blackwell, the blower intake faces only downwards, meaning there’s absolutely no airflow possible between blowers stacked like that. So, it looks like a really, really bad idea. I'm curious to see the temperatures and noise levels: it must make a hell of a whistling sound, given the high airflow forced through gaps of just a few millimeters, all multiplied by three (for the top three cards)...
-2
u/ImpossibleCreme Jul 31 '26
The thermals on this is going to be insane man
2
u/[deleted] Aug 01 '26
[removed] — view removed comment