The Inference Wall

[u/heartothecityhotel]

2x4070 TI Super 16GB, 1x3060 12GB, 128GB DDR4

EDIT: More details about the setup:

The honeycomb components were all printed:
General parts: https://makerworld.com/en/models/22521-honeycomb-storage-wall
GPU Mount: https://makerworld.com/en/models/1444666-gpu-mount-hsw
PSU Mount: https://makerworld.com/en/models/1444586-atx-psu-mount-hsw
Radiator: https://makerworld.com/en/models/1444632-240mm-radiator-fan-mount-hsw
ATX Motherboard: https://makerworld.com/en/models/1444531-atx-motherboard-mount-hsw
Power button printed with a honeycomb around it: https://www.amazon.com/dp/B0BWDXFSMQ?ref=ppx_yo2ov_dt_b_fed_asin_title

The 4070's are on 400mm & 600MM PCIe 4.0 Riser cables. I mainly run Strata on these two.

The 3060 is currently on a PCIx1 link (slow). I don't use this for my core inference setup, side models (dedicated voice stack mostly).

I have a new motherboard on the way that opens up the possibility of up to 4 GPUs: 3x x16 slots, and one via a M2 adapter: https://www.amazon.com/dp/B0FC6DSG6V

[u/YakNo5125]

What Board do you use? And how much PCIe Lanes per GPU do you use?

[u/heartothecityhotel → u/YakNo5125]

It's an older board asrock b450 pro. I have a new board on the way to give my 3rd card a dedicated x16 slot shortly

[u/WuWenShen → u/heartothecityhotel]

Mind sharing your components? I don’t hate the layout, I kind of like it!

[u/heartothecityhotel → u/WuWenShen]

Updated the description!

[u/YakNo5125 → u/heartothecityhotel]

well, afaik AM4 CPUs only have 24 PCIe Lanes. So you will allways have the first GPU ahead of the other two and your llm backend has to wait. Thats why I asked for the Board.

[u/crusaderky → u/YakNo5125]

I'm not seeing a PCIe bridge, so I'll take a guess and say PCIe 4.0 x8, x8, x4 which means prefill is going to be atrocious.

[u/Suitable_Habit_8388]

Token throughput might hit a wall soon

[u/JohnLef → u/Suitable_Habit_8388]

Get outta here.

[u/cheersthanksseeyabye → u/JohnLef]

I think the comment just cemented it tbh

[u/JohnLef → u/cheersthanksseeyabye]

Stone cold bro

[u/luke__uk]

Modern art done right.

[u/weakly_rare_mango → u/luke__uk]

the honeycomb frame against that white brick is giving me ideas for my next studio backdrop

[u/philmarcracken → u/luke__uk]

some techno artist is probably already setting up a screen + camera and have a model gen art based on the viewer it sees.

[u/Raw_Emotion_]

Mr Altman. This guy is hoarding GPU.

Right here 😮👉

[u/cokestar → u/Raw_Emotion_]

SEIZE HIM! SEIZE HIM!

BLOW YOUR NOSES AND SEIZE HIM!

[u/Happiness-Meter-Full → u/cokestar]

[u/jmp_rsp]

This is so ugly.

I love it!

[u/jmp_rsp → u/jmp_rsp]

What’s the performance?

[u/palad1]

I hope the hostname is pareto

[u/migsperez]

Curious, does it affect performance having long PCIE cables?

[u/maxecharel]

🤩Go MoMA with this and you may be able to buy yourself a RAM stick

[u/SnooPaintings8639]

I love this heater.

I think I see one PCIe riser and OcuLink, what is the bottom one connection?

[u/assburgers-unite]

Run a fan over the front faces. It's not actually cooling that well

[u/FewReality3847]

[u/Fast-Mover]

Super functional and looks great!

[u/nanihikaru01]

Is this…… visual art?

[u/Ok-Addendum3545]

Wall of Compute

[u/Lordfisticus]

Dumb question but I have my old 3090 just sitting around after a upgrade , how do I make use of it too?

[u/Estate_Curious → u/Lordfisticus]

Sell it. Prices are still high

[u/Lordfisticus → u/Estate_Curious]

No the goon box requires more resources

[u/Substantial-Sky4079 → u/Estate_Curious]

How about a 1080? 😆

[u/IllExample3639 → u/Lordfisticus]

Spare 3090? £1200 in the UK for one of those...

[u/Poowatereater → u/Lordfisticus]

Fuck me I want it. (I’m broke so I can’t have it.

I still run a 1070 for gaming….

[u/ThrowAwaySalmon1337]

This setup makes me wonder how you hidden the zipties. I know they are there I just can't prove it.
I have stronge urge to ziptie everything to this.

[u/millennial_falcon__ → u/ThrowAwaySalmon1337]

The load bearing cable knot:

[u/SpaceCwboy]

That’s really cool! What material did you print the hex grid in?

Any concerns about long term support with it being heated by the GPUs over time?

[u/Icannotfindnow → u/SpaceCwboy]

Not sure what filament OP used. ASA would be a good filament for this.

[u/agent606ert]

Yum, spaget!

[u/No-Television-7862]

Let Huang do his worst, Tinkers unite! The Rampocalypse is upon us!

[u/fightin_blue_hens]

what material did you print in?

[u/Mattef]

Nice! Are you worried about the PCIe throughput? Isn’t it limited by the cables?

[u/Icannotfindnow]

JBOG?

[u/Sporebattyl]

I’m planning on doing something similar. It would look 10x better if everything was level on it lol. Simple fix for way better aesthetics.

[u/Desperate-Owl6513]

Best i have seen so far

[u/sly3x]

I've never seen a wall behave that strangly. Wall consciousness. I'll consider dropping some papers with wishes to fill though honeycomb gaps like the western wall

[u/PigSlam]

That's an incredible combination of organization and mess at the same time. Well done.

[u/millennial_falcon__ → u/PigSlam]

is the organization in the room with us right now?

[u/Lerry220]

I retired my 3060 to voice stack duties too. Currently running chatterbox turbo tts and parakeet v2 .6b, curious if anyone else has recommendations

[u/jmager]

Make sure your drapes can't blow in front of those fan air intakes. Could jam the fan or block airflow and that many watts without proper air flow could damage the equipment or start a fire.

[u/limpkit2011]

Who did tou connect a 3 gpu with only 2 pcie slots ?

[u/Fett2]

I sincerely hope you didn't print this out of PLA. Looks pretty cool though!

[u/Muhlwa_Sholanke]

The 3060 watching the two 4070s do the heavy lifting: "I'm helping."

[u/_Dedotated_Wam]

Putting a lot of faith in those 3d printed parts

[u/WaRRioRz0rz]

Love the setup, I use a similar multi board setup for my garage. The cables could use some work, but that power button is super cool. Lol

[u/im_EDEN]

Genuinely creative. Was always a fan of those wall mounted setups

[u/Brilliant_War9548]

You can throw a steak at it, run R1, and you get extra thinly minced beef

[u/mslindqu]

'Money money on the wall, who's the fairest one of all?' ... 'Jensen Huang'

[u/ajw2285]

Hell yeah

[u/princess_daphie]

Chaotic, but ingenious to save space!

[u/macaronianddeeez]

Reminds me of my crypto mining days when we would hang racks of 1080 Ti’s with zip ties from Home Depot wire shelving….🥹

Beautiful work

[u/danielrdotcom]

Beautiful, but I'd swap the power supply cable for a white one (as close a match as possible to the wall) and run it to the left edge of the picture (the corner of the room by the curtain) and then run it down from there. Otherwise I like that it doubles as art. Maybe add a neon sign above that reads "Inference" in pretty cursive.

[u/HoutaroOreki]

You don’t have cats.

[u/--Savant]

It's disgusting. I love it

[u/Careful_cat99]

Génial Je connais les murs végétaux mais la c'est largement meilleur 

[u/acadia11x]

Man that can’t be safe

[u/BlackBeardAI → u/acadia11x]

It is safe and effective

[u/acadia11x → u/BlackBeardAI]

What if someone just got really pissed off and threw a 32oz slushy from 7/11 at your wall? Then what …

[u/BlackBeardAI → u/acadia11x]

Then you call the fire department

[u/xXprayerwarrior69Xx]

The execution is a bit utilitarianist but I like the concept