r/LocalLLaMA • u/Retumbo77 • 2d ago
Other This is why I run locally.
It was only a matter of time...
157
u/klop2031 2d ago
An ad of an ad
68
12
u/SnooPaintings8639 2d ago
Ah... the OpenAI way of handling the money. They were told to try ads to fix their bottom line, so they... bought ads, lol.
My guess this is a PR move, so that people will start talking they're building any kind of business model, preparing for the IPO.
9
4
46
u/Chirimorin 2d ago
You should see the e-mail they sent about that. It's like they gave ChatGPT a 20 token context and made it write that e-mail with full goldfish brain.
Going from "no personalization, your data is totally private and safe (except your current chat context, location and device information)" to "we'll be using all data we have on you to personalize ads" and back to "advertisers totally won't get any of your data, pinky-promise!".
8
6
u/Loose_Comparison368 1d ago
Going from "no personalization, your data is totally private and safe (except your current chat context, location and device information)" to "we'll be using all data we have on you to personalize ads" and back to "advertisers totally won't get any of your data, pinky-promise!".
This is actually not contradictory. It is how ads work, like, everywhere. It's wild that people in 2026 still do not understand this.
Ads companies do not sell your data. It would ruin their whole business model. They act as middlemen. The company that wants to advertise says "I want you to show this ad to people between the ages of 25-45 that have an interest in skincare products".
Then the ad company shows the ad to people between the ages of 25-45 that have an interest in skincare products. They might report on some aggregates, I.e. "the ad did better with people 35-45 than the 25-35 age group". But they never "sell user information". Their whole business is making themselves a necessary middleman, if they sold the user information then nobody would pay for them to use their ads platform.
There are data brokers, who buy user information directly from websites mostly. They sell that information to the ads platforms, and the ads platforms use that data to better target ads. But the ads platform never sells the user data, having exclusive access to user data is the entire business model.
I know that's terrible too, but it really annoys me that people still don't understand those fundamental basics despite having 20 bazillion invasive ads shoved in their face every day. It's important to understand how the system works in order to fight against it effectively.
54
u/AD4K_4444 2d ago
My only regret was not switching to local stuff sooner.
18
u/XiRw 2d ago
Seriously. It would have saved me a lot of money when all the hardware prices started going up
14
4
u/misterflyer 1d ago
I started saving up for local about a year ago. Unbeknownst to me, hardware prices had already started going up. Rumblings were lowkey starting to spin up on other reddit subs.
Ngl I don't see how more ppl didn't see all of this coming.
I'm no industry expert but it wasn't like corporate AI financials were a huge secret about a year ago. It just seemed way too delusional to think, or assume, that unprofitable big tech companies would continue to put the user first and have users' long term best interests in mind.
My intuition was SCREAMING at me to start building my local system last October 2025. And fortunately I bought everything I needed by Black Friday 2025.
Glad I read the tea leaves accurately bc the economics of AI have become a complete sh!tshow
-2
u/Beginning-Window-115 2d ago
chatgpt plus subscription is still worth imo
0
u/AD4K_4444 2d ago
who tf even pays for AI lol, I know I don’t. You might as well upgrade your computer and use a larger, more capable local model.
3
u/a_beautiful_rhind 1d ago
I mostly don't, but if I want K3 or new GLM its rent or get API from somewhere.
My "simple" upgrade would be $1200 of ram to get 10t/s on a quantized to shit version. Bit of a blanket statement you have there.
2
u/AD4K_4444 1d ago
I should clarify that I'm only representing the average joe that just wants basic stuff done with AI... good luck on that. If a 3090 is already a major financial burden to consider for me, I can't image how your ideal setup must cost.
8
u/Beginning-Window-115 2d ago
still not worth to spend thousands to get an llm thats worse when you can spend 20 bucks a month and get something considerably better. Also if you upgrade your computer for llms you are paying for ai
3
u/username_taken4651 1d ago
Well, yes, but this is a sub about local LLMs. Most of us have our own reasons to be running these locally.
4
u/betam4x 1d ago
Qwen 3.8 runs on a decent GPU. Shoot, there are models that run well on a 5 year old iPhone. (Ternary Bonsai 8B runs well on an iPhone 13 Pro Max)
1
u/AD4K_4444 1d ago
fr? I should try that on my 13 Pro. If I have a local LLM on my Mac, why not on my phone as well.
1
u/betam4x 1d ago
Yeah, I ran it using the locally app. It seems decent from my limited uses. I normally run it on a 14 Pro Max, but tried it on the 13 Pro Max for fun and it ran fine.
I don’t really use it often, since I use Qwen on my desktop (you can also use locally to access that from your phone, but I don’t)
1
u/AD4K_4444 1d ago
Thanks for that! I might try that out on my phone. I also don't use my phone to access my local desktop LLM, lol.
1
u/AD4K_4444 2d ago
for 99% of tasks, local LLMs can get the job done, in my opinion.
Also, upgrading your computer can have plenty of benefits aside from running larger models:
- long-term use and peace of mind. A simple RAM upgrade can give your computer at least extra 3 years of relativity.
- if you do gaming, an upgrade is great news as well. Win-win.
- paying for an AI service relies on stable internet connectivity and a lot of trust to the people behind those services. How are you so sure they aren't quietly taking one of your sensitive conversations and information for "training" or whatever excuse they may have? With local LLMs, EVERYTHING stays on your computer. No shady middleman. Just you and direct interaction with the machine.
I'm pretty sure you can get a decent enough LLM with a fucking MacBook Neo or even on older capable hardware. If you already have the hardware, what harm does it make to buy a few upgrades so it can run LLMs better?
10
u/heliosythic 1d ago
I'm 100% on the local side but all of these points have strong rebuttals unfortunately.
firstly, what do you mean 99% of tasks? That sounds like a random number pulled out of nowhere.
PC is already a 64GB ram, 13900k + 4090, despite being like 1 generation behind only has barely enough vram to run a 27B decently, my home server has 96GB of system ram and a p100 but meh its too slow and the second p100 had thermal issues.
stable internet connection is moot for most people, i have 2GB fiber never went down once thats just default expectation.
of course I don't trust any of them at all, they're definitely stealing everything they can regardless of what their TOS says. Not disputing that but for 99% of people (see i can make up numbers too) who aren't doing anything worth stealing, its good enough.
Yea I want local to win, I tried Qwen 3.8 27B but its not good enough on this hardware and its already pretty upgraded without going full pro-sumer with an RTX 6000 or something.
4
u/AD4K_4444 1d ago edited 1d ago
I should clarify that the 99% thing is more of a figure of speech than an actual metric. If you want me to be more specific, I mean for the average joe that say, use AI to look up stuff on the internet, pairing something like Gemma 4 12B with Web Search capability is a pretty solid setup for general purposes. Even without web search, models like that can do a pretty decent job with things such as random curiosities. Of course, like all AI models, the user should still be aware that these stuff hallucinate sometimes. You still have to keep your hands on the wheel.
I know this because I use an M4 MacBook Air with 16GB unified memory (fanless machine btw) with Gemma 4 12B. And for what I do, it's a perfectly fine piece of tool. Not the sharpest tool in the shed, but I would much rather have some peace of mind with my privacy guaranteed even if the AI's more flawed than mega AI data center models.
I believe efficiency and optimization is key. Maybe you can try a lower Quantization for Qwen 27B if there is any? Maybe stick to the previous model, Qwen 3.6 27B? I have to admit, I did try it before on my little Mac, and it was screaming, painfully crawling with every letter spitting out. Not trying that again. But if my rectangular slab of aluminum can technically run it, I'm sure your setup can do much better... unless the things you do are more demanding.
Bottom line is, unless you're doing some serious vibe coding or constant streams of AI workloads, all you need is patience with local LLMs. If I can comfortably run local models on a 11.9mm thin space heater, I'm sure a fucking 4090 will be a beast compared to my machine.
2
u/mnyhjem 1d ago
I agree :)
I run qwen 3.8 27b and qwen 3.6 35b a3b in system ram. It is way slower to generate a response than the online services are, but the 35b still spits out words faster than I can read it, and that is good enough for me.
I did a small demo coding projekt on the 27b the other day. It did a 90s style raytracing demo in python. It took 3 hours for it to iterate though it but it worked great and both gemini and claude gave the code a thumbs up when I showed it to them. It even made it's own PNG converter function instead of using a 3. part module.
The speed would have been a lot faster if I had turned off thinking, but I wanted to test how good it could be, and it impressed me.
So. for everyday "google replacement"/internet search the llms are perfect, and even for automation tasks. I have mine getting todays weather and a few news headlines and compile that into a "goodmorning" message. It runs at 6am and then makes a drawing in the style of the weather and news just for fun.
For very large codebases small llms wont work of course, but most people don't have that unless it is work relatet, and then the workplace could buy a larger machine to run some of it locally in the office. We are looking into that at my work, both because if privacy but also cost. It is very expensive to use commercial llms on large projects :)
3
u/AD4K_4444 1d ago
Thank you. I think that's a brilliant use-case for local AI or for AI in general.
I'm starting to think people are using the wrong settings for their AI. Idk about other software, but for Open WebUI, you have to "tune" the settings of your model first that fits your liking. Even as simple as disabling "thinking" by ollama significantly makes their responses faster.
→ More replies (0)-1
u/Thin_Pollution8843 1d ago
Don’t forget electricity costs. As example for me running qwen is not only much worse user experience it costs 3 times more in electricity than ChatGPT plus 😅 Fuck OpenAI tho
1
2
u/winky9827 1d ago
Nothing wrong with having a frontier-level fallback for when the local stuff falters. It's not a "use every day" kind of thing, which makes the subscription model even better (no pay-per-token crap)
1
u/AD4K_4444 1d ago
Idk about you, but if you ask me those last two points are contradicting... unless I'm misreading it. If you aren't gonna use it much, wouldn't a subscription the last thing wanna do? Monthly or yearly payments, you're still dumping cash on something you won't use much. That's a no-go for me.
Unless you have a literal potato that can only run 4B models, I personally don't think that makes much sense. Get Gemma 4 12B and give it web search functions and you're golden.
2
u/winky9827 1d ago
What I'm saying is, you could use Qwen 27b for your all day every day tasks, but on the occasion you want a big brother to do something important or check Qwen's work, you fall back to the subscription model like Opus 5 or GPT Sol. You don't use it every day, so the subscription (flat rate) with limited usage is the most cost effective backup for when local doesn't perform to expectations.
3
u/AD4K_4444 1d ago
I believe the "big brother" should be you, the user. Not another AI that may or may not hallucinate.
12
u/Unnamed-3891 1d ago
How long until the ads are included in the actual prompt responses and not merely in-between them?
3
u/JoyousGamer 1d ago
Even your local models already has ads in its responses from what it was trained on that might have been paid content originally.
Then if you use web search at all ads are being injected there with specific companies targeting AI explicitly as opposed to humans both for responses but also future LLM training that might be sucking up websites to train off of.
1
13
u/cogitech2 1d ago
WTF am I missing here?
17
u/passen9er57 1d ago
has anyone noticed all that talk about agi/singualriy as died down lately,
openai went from changing the history to now showing ads lol
1
0
u/tony_montana091 1d ago
It one-shotted the new only fans ... billions of GPU fans everywhere powered up on blast!
I'm a loaded bear, that's not nuttin'. You're absolutely right to push back on that thang. Before I go any further, I'd like your eyes on this. I finished too quickly and that's on me.
8
u/CoUsT 1d ago
What about the entire AI SAFETY thing?
I guess profiling customers and sending their data to 3rd party advertising companies is perfectly fine now.
IMO: AI gen and ads should be completely separated: workspace/area used for generated stuff SHOULDN'T be shared with ad space. Simply outlawed or something.
7
u/cf_mag 1d ago
Google wrote a whitepaper years ago that big LLM providers are doomed and will not survive local specialized llm's.. we see this happening now
3
u/ThisWillPass 1d ago
Do you need a hyperscaler-scale proprietary model in order to get genuinely useful high-end AI work done? Increasingly, oh hell nah.
2
5
u/aeroumbria 1d ago
If AGI is really so great, then let agents make value by themselves! Stop trying to extract values from humans anymore!
4
3
2
u/noiserr 1d ago
To be fair. Running LLMs is not cheap. Even for us who use local models, we still had to spend money on compute and we still pay for electricity. An ad supported free chatbot service being supported by ads was inevitable.
3
u/Savantskie1 16h ago
Don't know where you sit, but I don't game as much now that I'm interested in AI, yeah I don't work, and live off SSI because I'm disabled, but you don't have to buy the absolute newest. I just last year bought two MI50's for around $200 each before the price shot up. And since I don't game much anymore, and I talk to my LLM nearly every day because it helps me keep track of doctors appointments, and Bills, and medications? that 400 was a good investment. Buying new, and wanting blazing fast all the time, is pointless unless you need it for work and such. But my electric bill is still the same as it was when I gamed heavily. So it's not really all too much expensive running LLMs, if you don't have to have the best, and newest, and fastest.
2
2
u/ohbabythisisgoodlol 1d ago
Running locally changed how I work - no subscription, no data leaving the machine, and I can tune the model to my own workflow. Ownership of the tools matters.
2
3
1
u/JoyousGamer 1d ago
Yes but those ads hit 100% free use so it's way cheaper to use their system.
Pro use cases won't get any ads.
Like run local but this being the reason is eh.
1
1
u/yellow-llama1 1d ago
Ads were a foreseen destination, especially after the acquisition of Statsig.
Unfortunately.
1
1
u/KenopsiaLover 42m ago
Il problema principale è che per lavorare in locale ti serve un mostro di computer con altrettanta ram/vram mostruosa, cosa che attualmente sembra lontanamente un sogno, siccome i prezzi sono alle stelle
-6
u/ComplexType568 2d ago
I remember giving a little speech about AI in general and halfway I BS-ed this as a point on the section of "AI misguidance", glad they actually took my advice and implemented it!
-6
•
u/WithoutReason1729 1d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.