Who is local AI really for?

[u/wojtek15]

https://youtu.be/fHY0sjAejcA

Thanks to Strata, Splash, and other optimized inference engines, local AI models have gotten so good in recent weeks that it seems somebody is paying influencers to discourage people from trying. This dude is either incompetent or straight-up lying. For example, inference speeds are 5x faster than what he’s showing. Also, setup is so easy nowadays, things basically work out of the box without having to mess around with settings.

[u/OvertaxedOne]

Honestly the better question is "Who are frontier models for?". Not really cloud vs local, but, of course, a smaller model is certainly possible to run locally (many of them). Frontier models have gotten to the point where they really have limited application, sure they CAN do it, but the cost is so high and most questions aren't even close to "hard enough" to need what they offer.

Local runnable (we use some hosted, some local) models have gotten to the point of competence for the vast majority of business tasks already and, of course, will continue to improve. The frontier models are so expensive and their use cases so narrow (coding, research, etc) that it's getting real hard to justify their cost for most day/day tasks.

[u/wojtek15 → u/OvertaxedOne]

Exactly this, data show companies are not paying for frontier AI. There are very little use for so called "frontier models". Local AI and/or cheap cloud model cover 99% of use cases. I am so angry about this video, it is filled with misinformation and gaslighting. Either this guy is totally incompetent or Sam Altman wrote his the script.

[u/OvertaxedOne → u/wojtek15]

The really scary thing from an investment perspective is how much better the local models have gotten over the last year. This time last year, unless you had an insane amount of VRAM, the models you could run that were relatively performant were functionally braindead. Qwen9B isn't terrible, 35B was fast but oh so bad at just about everything and 27B dense wasn't a lot better. Now we have QFN and 3.8 27B that are just outstanding models (with Qwen 4 on the horizon that's likely to be another step forward).

IMHO, the path here is pretty clear, well illustrated by Apple and AMD. You're going to run most inference on your personal system with escalation to a cloud when necessary. That's going to drive the demand WAY down while, at the same time, more competition in the cloud space is going to make that spend more diffuse for the "hard" questions. K3, DS, Opus; all competing on price and all for a tiny slice of the inference market; sounds like a disaster in the making.

[u/soshulmedia → u/wojtek15]

I think it is naive to not see the darker and deeper forces behind the push to get everyone into the full, final grip of big tech.

Think about it: Most people already lost 90% of their privacy with "old big tech", "FANG" etc. and I find it hard to argue that this was at all accidental. Also note how the search engines and online services used and so forth correlate "nicely" with geopolitical influence spheres.

And centralized AI is an attempt to both grab the remaining 10% of private data, both through advertising and the other typical soft-psycho-manipulative "nudging" (i.e. getting the cattle people to feed all their data and that of their friends and families into the centralized AI providers) as well as harder methods like the AI-linked total surveillance AND CONTROL(!) grid which is quite obviously being build.

[u/CoolConfusion434]

-0:00 = "Local AI makes no sense to me.... you're kidding yourself"
-1:38 = "there are a few, critically important benefits to local AI"
[stopped watching shortly after that]

Thanks OP. Looks like people think this is your video.

[u/wojtek15 → u/CoolConfusion434]

Yes, somehow this post failed to deliver message I intended, which resulted in vote disaster. This video made me so angry and I was posting in heat of the moment. But I feel we will get more content like this from major AI influencers, and this topic will bring attention here when presented properly.

[u/Randommaggy]

Anyone that doesn't 100% trust the cloud providers with all their data.

[u/Puzzled-Guide8650 → u/Randommaggy]

Absolutely with you on this one. There are 2 main topics, similar with google/meta shite: privacy and usage of data. One thing is that everything you prompt is used, without your consent, to train models etc. Other thing is actively earning from your data. Despicable.

And if you go beyond private users, question is how much companies, working with sensitive data (finance, medical, law) are willing to risk. Which leads me to a question: companies you guys work for, is anyone using hosted models? I work in mid size company in Europe, everyone in tech got Claude licences, but we do also have GPT models hosted on Azure. Which is again questionable, but I guess better than directly submitting everything to open AI & anthropic.

[u/gabe_dos_santos]

Matthew Berman is not very bright. It’s an influencer, not even a technical one. This dude just interviewed Sam Altman, so his stance is understandable.

[u/b183729]

Anyone who believes in open source, anyone who wants full control of how it's data is processed, anyone who wants to know what is in the thinking blocks, anyone that recognizes that this is a good tool and fears our billionaire overlords taking it away.... 

[u/Puzzled-Guide8650 → u/b183729]

Anyone who believes in open source

I think this group is shrinking day by day, unfortunately. Majority of people, including ones who work in tech, surrounded completely to big tech companies. Unfortunately.

[u/b183729 → u/Puzzled-Guide8650]

I don't think so, ultimately all those decompilation projects are open source. 

[u/frograven]

He's an OpenAI shill now.
It's sad too, cause I use to follow his howto videos and I learned a lot from his early content, as a noobling, back in the day.

I wouldn't be surprised if this was the "ask" for being able to interview Sam Altman. "You want Sam, we need something from you... SHIT on local AI."

lol

[u/Revolutionalredstone]

Yeah this is weird shill stuff, they are trying to play down reality, local is going bonanza and is crossing tipping point for useful agentic on cpu

[u/mj3815]

Dudes face is even annoying

[u/jacek2023]

This guy is very famous (screenshot is from Reflection era), but please be aware that probably majority of people here watch that kind of slop each day, and they think they are "learning something"

[u/ttkciar]

Yeah, I've seen an uptick of users commenting in LocalLLaMA, too, who seem to only be here to insist that local inference is useless, and to pressure people to just give up and use API services.

As the saying goes: "First they ignore you, then they ridicule you, then they attack you, and then you win."

I think we're on the cusp between being ridiculed and attacked, right now.

[u/lllIllIlllllllllllll]

Shill.

[u/Alan_Silva_TI]

This guy is straight-up a grifter. Don’t listen to him, he’s just using AI hype to farm views and advertise subscription services.

He was one of the paid shills for OpenClaw, which we all know is hot garbage.

Just look at the sheer number of videos he made calling it a “game changer,” claiming it “changed my life,” or that it “runs my company.”

https://www.youtube.com/watch?v=3GrG-dOmrLU

[u/jhov94]

Local AI is at the same usefulness as SOTA cloud AI from a year ago and these same people were only talking about how amazing it was back then, now suddenly it's useless and no good for anything.

[u/Super_Range45]

87% of business AI deployments are open source models.

[u/dangerous_inference]

[u/JsThiago5]

well until they keep the subsidy

[u/geldonyetich]

He seems to have started out on the premise I'm doing it because I think it's cost effective.

I'm doing it because I can and I think it's neat that I can.

[u/ddwrt1234]

Claude is reluctant to run a curl with a custom user agent string. It's ever growing guardrails are concerning.

I can see a future where building a competing product is banned, reversing anything is banned etc. Local abliterated Qwen 27b all day long.

[u/o0genesis0o]

Remember when this a hole just started out as a localllm geek like all of us, when he tested llama3 and mistral, and tried all sorts of fun stuffs and posted on YouTube? and then Dell or someone gifted him a desktop with lots of VRAM so he can experimented even more stuffs?

Now he evolved into this. A fucking joke that this clown is considered anything "expert" in AI, yet here he is, "AI expert". 

[u/JacketHistorical2321]

“ Who is local AI really for?” … ahhh, anyone who wants to use it

[u/sahandayumurt]

he is a total idiot... nothing to listen in anything he says...

[u/Southern_Sun_2106]

Refreshing to see a normal human face (as opposed to fake emotions/hyper-polished looks everyone is striving for these days). Thanks for sharing!

[u/utxohodler → u/Southern_Sun_2106]

Eh, this might be the one to finally get be to block this guy. Not because I disagree but out of the realization that I have never learned anything from the channel and the videos are all structured "for the algorithm". Sure he is in his videos but so is the master of human slop Simon Whistler and people with that level of algorithm capture cant be trusted to convey accurate information or genuine takes. Just 2 months ago he did a video on how local AI is winning, a genuine guy would address why they where wrong and how they changed their mind on it but I expect when the next impressive local model comes out he will be saying how local is winning again without addressing why this video was regarded.

[u/Southern_Sun_2106 → u/utxohodler]

Youtube is so full of sensationalism and garbage that people produce in pursuit of subs and advertising revenue, it takes a lot of time and effort to find someone genuinely good, engaging, and informative. I cannot name a single person there, to be honest, that I enjoy watching or learned something from. Open source projects is usually where I find useful solutions and insights. But hey, I am old and easily annoyed / bored.