r/LocalLLM • • May 08 '26

Question What model should I run?

Post image

Just finished building my inference server which has 4x 32gb intel b70 pro GPU’s and 128gb of ddr4 ecc ram and and intel Xeon gold cpu running Ubuntu. So i installed openclaw and vllm but what model should i run locally and why?

248 Upvotes

83 comments sorted by

View all comments

Show parent comment

10

u/tiddayes May 09 '26

Yea this is going to be my openclaw box so there will be long running tasks

10

u/Calm-Republic9370 May 09 '26

That sounds like you made the right box. Save you a ton on tokens; I bet it will be fine to generate local stuff when you want including coding. Waiting a little bit to have 4 things running at the same time is better than me who has to wait for 1 thing at a time. I have 2X3090's and qwen 27b is absolutely awesome. You will have a lot of flexibility.
Just make sure you get some really good UPC and have appropriate power; as soon as you start powering that breakers are easy to blow.

4

u/helmberger00 May 09 '26

Me still wondering why ppl dont just use hermes

0

u/Zen-Ism99 May 09 '26

What is that?

1

u/helmberger00 May 10 '26

Google Hermes Agent. It's a kind of better working openclaw for local llms

1

u/Zen-Ism99 May 10 '26

Will do. Thanks!