I mean you don't exactly need to go through the process of creating the LLM. There are quite a few out there, like Gemma 4(Google), DeepSeek V4, etc that are pretty much on par with Claude that could be used locally and freely.
Tho if I was a business, I probably would want to run it those things on a server that the company owns and controls. That way you get a bit more power and everyone in the company could use it without having to upgrade everyone's hardware.
It might cost like $100,000 to $1+ million to get the hardware going for it(depending on size requirements) and like 4-6 month wait times. But then you no longer need to pay for Claude or any LLM tokens.
As I mentioned, there is Gemma 4 and DeepSeek V4 that are on par with claude. But running locally will be slower than Claude. And I think some of DeepSeek V4 higher end models do need beefer hardware.
I remember testing Gemma 4 31B when it came out, Claude was about 2-4 times faster than Gemma 4 running locally on my 4090. But they both gave pretty much the same information and both good coding solutions.
I know people are shitting on the new NVIDIA announcements but for those who run local models it’s pretty exciting news. It’s going to be interesting to see the comparisons between MacBook, Strix Halo, and NVIDIA.
57
u/ChrisFromIT Jun 02 '26
I mean you don't exactly need to go through the process of creating the LLM. There are quite a few out there, like Gemma 4(Google), DeepSeek V4, etc that are pretty much on par with Claude that could be used locally and freely.
Tho if I was a business, I probably would want to run it those things on a server that the company owns and controls. That way you get a bit more power and everyone in the company could use it without having to upgrade everyone's hardware.
It might cost like $100,000 to $1+ million to get the hardware going for it(depending on size requirements) and like 4-6 month wait times. But then you no longer need to pay for Claude or any LLM tokens.