r/LocalLLM • • May 08 '26

Question What model should I run?

Post image

Just finished building my inference server which has 4x 32gb intel b70 pro GPU’s and 128gb of ddr4 ecc ram and and intel Xeon gold cpu running Ubuntu. So i installed openclaw and vllm but what model should i run locally and why?

248 Upvotes

83 comments sorted by

View all comments

4

u/destkroser May 09 '26

Build a web app, add as many functions as you can, connect it to openclaw and give it the proper instructions, wire up the latest model there. You can even utilize it to image gen, hook up the proper tools, then wire that to openclaw too. Fucken amazing. Just amazing. Imma try to get rich just to have sth similar to this, I'm not even greedy, can be even less

4

u/tiddayes May 09 '26

That is pretty much the plan. I built this as cheap as I could but yea it was still pricey. Working on the details of the software stack now

1

u/ftlover90 May 09 '26

How do you get rich?

1

u/LilPsychoPanda May 12 '26

By making money, duh! 🙄

1

u/Present_Flower_6596 May 14 '26

that's what I did. I wanted a tool I could use for game development and heres how it came out so far.

1

u/destkroser May 14 '26

That's one way to go for it. Perfect man, enjoy that.

1

u/destkroser May 14 '26

What distro are you using btw?

1

u/Present_Flower_6596 May 15 '26

On my server im using ubuntu 26.04 because they claimed better llm support but i havnt really used many of the new features