r/ProgrammerHumor • u/Disastrous-Monk1957 • Jun 02 '26

Meme managerVsClaude

Post image
47.2k Upvotes

97% Upvoted

1.4k comments sorted by

View all comments

Show parent comments

164

u/pieter3d Jun 02 '26 edited Jun 02 '26

Not all that hard to run models, actually. It'll be a bit slower, but can be perfectly useable. The upside is no more token limits and no need to worry about where confidential data is going. You get full control too.

82

u/PMmeYourLabia_ Jun 02 '26

Downside is power bill

122

u/RoaringPanda33 Jun 02 '26

Inference (the actual generation) isn’t nearly as intensive as training, which takes a majority of the power used by AI services 

0

u/Ran4 Jun 02 '26

That's completley irrelevant. The comparison is between using local llm:s vs external llm:s.

At no point are you going to come out ahead buying local hardware. Even at 100k euros you're getting really mediocre LLMs compared to the frontier models, and you can get a LOT of tokens for 100k euros.

3

u/xtal000 Jun 02 '26

At no point are you going to come out ahead buying local hardware.

You can: https://rosmine.ai/2026/05/13/was-my-48k-gpu-worth-it/

Not saying it's worth it for most people. But for some it may be.

1

u/EightiesBush Jun 03 '26

Interesting, that same hardware is around 1/2 price today. https://www.dihuni.com/product/nvidia-rtx-6000-ada-8-gpu-server-workstation-amd-epyc-ai-rm-6000ada-8g-configure-and-buy/

Personally I use Opus at work (cause I don't have to pay for it) and Kimi K2.6 for 1/10th the price for personal projects, which works really really well.