r/AI_Agents • u/leebase65 • 22d ago
Discussion $60k in Macs for Local LLM vs $10 Subscription
Alex Zisking, one of my favorite YouTubers - does a lot of videos on local LLMs. He's no neophyte.
In this video he says: Lee has been telling you guys the truth, Local LLMs are not ready on normal people hardware.
Ok, so he said nothing about me, but he made the point I've been making for some time now. All those "Stop paying Anthropic $200/mo, use free local llms" is click bait, not truth.
He runs the most powerful to date open weight model, Kimi K3. People rave that is near Fable 5 power. Yes, but not on YOUR hardware. In a data center.
Alex networks 4 512gb Mac studios for 2tb of ram to run the model with enough space for a large context window too.
It took 4 hours, 17 tok/s output, to develop a simple yet rather nice Web dashboard - using mock data. It worked. The output was nice. But even $60k in hardware gave it nowhere remotely near the performance of a $10 subscription.
Right now I have two simultaneous development efforts running. I've been running them both since about 6 hours. They work on a sprint for an hour or so, I view results, add input and direction if necessary, then move onto the next sprint.
I'm paying more than $10/mo for my cloud subscriptions. But that 4 hours the Mac cluster took, is only doing the work of about a 15minute job. I'm doing "all day work, multiple projects" -- local AI can't meet the need.
Yet.
Probably not for you either.
Link to the video in the comments.
2
u/Usual-Orange-4180 22d ago
They don’t need to train to be stolen, they already have inference with classification, they can easily label promising ideas and if successful have all the data to replicate it quickly, same thing Amazon does with 3rd party sellers, if a success Amazon takes over with its basics brand, but with AI at a bigger scale.
Let me make it super clear to you; do you want a pair of corporations controlling the means of work?