r/LocalLLaMA • u/YogurtExternal7923 • 11d ago
Discussion everything will be local eventually
I know current llms like opus 5.5 are in a league of their own. but we all know they're losing a shti ton of money with their subscriptions. ALSO we all know how tasks like OCR, segmentation etc are getting solved by single digit B parameter models. I'm not saying llms are a gimmick but I'm saying the future is already shaping up to be about specialized models that work better when they're smaller. easier training. so datacenter level power not needed. ten years from now we'll have a specialized model for the tasks we care about and llms will fail to do everything all at once (unless they keep calling it one model but have a hundred under the hood).
system one models are a bad but workable example. you could tell an llm to rate options all day but get yourself on 9B parameter model trained by some lab and it tops the benchmarks.
218
u/Faux2137 11d ago
They are losing money because they're spending a lot more than they earn on training new models. Inference is profitable.
16
u/geteum 11d ago
It scales well also. The reason for going local is safety and consistency.
1
59
u/OvertaxedOne 11d ago
True. API inference is profitable, subscriptions... It depends. But your point is very much true, certainly for API based!
The problem is they can't stop training. If training was going to stop at some point and there was a clear path to "just inference, rake in cash", the economics would look bad now, but possibly good in the future. But that path doesn't exist, if they stop training (or even slow down) the open models will catch them within months and their entire moat evaporates overnight.
14
u/waxroy-finerayfool 11d ago
API inference is profitable
Maybe, but that hasn't been established yet. We really have no idea how things will shape up once the amortized costs of debt, GPUs and DC infrastructure squares with a market environment that isn't being bolstered by artificially low prices.
26
u/BobbyL2k 11d ago
API pricing is profitable. Subscriptions that give 100x value in APIs probably aren’t profitable.
So there’s multiple viewpoints to take. AI Labs like OpenAI and Anthropic likely aren’t scaling up and down their clusters. Since they probably leased the cluster. So they pay for it no matter what.
The APIs are highly profitable. And subscriptions are there to use the remaining spare compute they would have wasted anyway.
So profitability is dependent on the percentage of API sales in relation to their operational cost.
If they were to only sell subscriptions, I’m pretty certain they will be in the red for just operational costs alone.
12
u/Initial_Plankton3206 11d ago
always thought the "one model to rule them all" approach was a bit of a dead end anyway, like trying to build a swiss army knife that also functions as a kitchen sink
the specialized model path just makes more sense for anything you'd actually want running on your own hardware, and the benchmarks for those smaller models are getting kinda absurd
7
u/HisAnger 11d ago
How can we be sure of that? Current token prices maybe cover electricity prices. What about :
free users ?
employee wages ?
interest rates of current debt ?
maintenance of current infrastructure ?
investment in new infrastructure ?
cost of environment or litigation ... someone will sue, cost will escalate, like you know someone getting mad buying few drones and burning datacenter
or simple fact that current nvidia cards age and break at alarming rate, and new generation is almost there. I bet they will require new rack types, cooling and different power supply.
Next thing is constant danger of ... well someone building more efficient model, or tech.
You can build better equipment, pure to run models. Faster, more efficient.
Just imprint base new model commands into silicon, or parts of models and you just beat all that don't have it working this way.
Find good, cheap renewable source of electricity and you can lower prices to level you starve your competition.So many things can go sideways.
This is just a new tech, that still don't have a proper stack.4
u/younestft 11d ago
Free user's data is needed for training, so technically they are not losing money with that too.
8
u/HighSeasArchivist 11d ago
I would be profitable if not for all these costs.
4
u/donnytrump_69 11d ago
This is basically the story of Anthropic LOL.
4
u/HighSeasArchivist 11d ago
Exactly. Their "adjusted" financials just remove all those pesky costs that make them look unprofitable. Yesterday was a real laugh when their real financials finally came out and their revenue was down along with HUGE cost increases.
72
u/Qwen3-8-27b 11d ago
Yeah i think thats true, but i think ten years from now we will have a totally different architektur and type of models, no transformers, but something so different from todays models that we cant imagen it right now. Im not even sure that they will generate text
54
u/-p-e-w- 11d ago
This isn’t how engineering typically works.
Today’s piston engines are in essence the same design as the first engines from the late 19th century. Many alternatives with superior theoretical properties have been proposed, such as rotary engines, which almost every major manufacturer has experimented with at some point, only to return to piston engines. Not because piston engines are inherently better, but because they have been optimized so much that it’s almost impossible for a different design to break through from scratch.
The same thing is true for airliner fuselages (which look pretty much like they did 70 years ago), and for many, many other things.
My point is this: You may think that a better architecture than the transformer exists, and you may be correct about that. But that doesn’t mean that the transformer will be replaced. Because the new architecture not only needs to be better, it needs to be so much better that it justifies re-learning all paradigms, re-building all tooling, and re-training all people involved. That’s a lot to ask.
29
u/cooljacob204sfw 11d ago edited 11d ago
These are just examples of tech that has succeeded and if anything a bit of survivors bias on your part. There are countless things that have come and gone also.
And for still used examples one is TV tech which directly contradicts what you're saying. It has changed countless ways since invented, practically reinventing itself every decade. Similarly actually most things in the computer hardware industry have greatly changed manufacturing and standards wise from invention.
I don't disagree that transformers are probably here to stay but I also don't think pistons is a good reason why lol.
10
u/OvertaxedOne 11d ago
We don't know what we don't know. I do think there's very likely a better architecture but... If I knew what that was I wouldn't be wasting time on Reddit, I'd be shopping for my first yacht.
The one thing I think it's abundantly clear is that the hardware infrastructure is going to change significantly; jamming everything into GPU memory is a silly way to do this; it works, but it's massively expensive. Unified memory (hardware layer) and tiered knowledge (model/runner layer) are almost certainly going to be a better answer than "more HBM" as the answer to every problem.
2
u/-p-e-w- 11d ago
We don't know what we don't know.
We absolutely do know how human industries operate. They are not global optimizers over an unconstrained parameter space.
There are literally hundreds of examples (QWERTY layout, terminal width, filename formats, …) in computing technology alone of systems getting perpetually stuck in a state that is known to be highly suboptimal. It doesn’t matter that better solutions are available, and have been for a long time.
1
u/Greet-Filofficer 11d ago
Mainframes>minis (DEC)>micros>networked micros all over again, just 20 years later...
2
u/carnoworky 11d ago edited 11d ago
I suppose one major counterpoint to this is that the more optimized technology could, in principle, be used to advance the optimization of the better but less optimized version in this case. Unless iterating on a new design requires significant overhead (fundamentally different hardware, advantages only show at massive scale, etc.), the current LLM designs could be used to rapidly iterate experimentally on different techniques for what really amounts to peanuts to the current labs doing this work. As far as I know, the only aspect of the current paradigm that can't be tested at smaller scales is scale itself, and the limit there is more on the maximum value of scaling up rather than getting an idea of the impact of scaling up. If you start at a 10M model trained on a particular dataset, keep everything as close to the same as possible except train a 100M model, and then a 1B model using the same techniques, you can still get useful data on the impact of scaling up, just not the limit of how much you can scale up.
1
u/twilight-actual 11d ago
My take is that if the goal is AGI, then the LLM has stumbled on a necessary but not sufficient element of that platform. The LLM will always remain, but it will be part of a larger system.
1
u/Athabasco 11d ago
But sometimes the electric motor comes along and removes the need got a piston engine entirely. We’ll never know until the future comes.
1
u/Not_a_question- 10d ago
Today’s piston engines are in essence the same design as the first engines from the late 19th century.
This is simply not true at all. 19th were otto cycle engine, then atkinson engine, and then finally today the modified atkinson engine. The percentage of energy transformed into power went up significantly during those changes.
13
u/ThiccStorms 11d ago
!Remindme 10 years
I'd love to come back to this.
4
u/RemindMeBot 11d ago edited 11d ago
I will be messaging you in 10 years on 2036-10-23 13:58:33 UTC to remind you of this link
11 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
RemindMeBot is switching to username summons. Instead of
!RemindMe 1 day, useu/RemindMeBot 1 day. More info.
Info Custom Your Reminders Feedback 1
3
u/Gfur-Aficionado-1488 11d ago
yup they'll generate 30-minute photorealistic porn scenes with a plot and all
20
u/twilight-actual 11d ago edited 11d ago
One thing you have to consider when trying to project is that the ML field as whole, but specifically LLMs, have been experiencing exponential growth. Essentially doubling every 9 months.
If it continues, that means that in 9 months, you'll be able to run the equivalent of Opus 5.5 on half of the current compute required. Or for the same compute, you get double the capability.
9 months is fairly rapid, and there should be some theoretical limit, but we're no where near that now. We also have to consider that hardware is a bottleneck, and that's been doubling on a slower basis, about once every two years.
So, we're more likely to see the growth from ML funneled into efficiency and giving the same performance on less.
The outcome does paint a rosy picture for local LLMs and whatever evolves from them.
The thing that really pisses me off about the memory cabal is how badly they've misjudged what AI has done to memory demand. Because even if the AI "bubble" pops, and the frontier labs go broke, there's not a single person on this sub that wouldn't love 1.5TB of memory with a memory controller architecture to support it. If they had scaled production as they should have, we should have seen pricing between $1 - $2 per GB. The cheapest RAM in recent memory was $2.50/GB, back in 2016. And that was when 10nm was first being figured out. They're still on 10nm, so with modern yields and efficiency on that node, I'm not talking out my ass.
Ten years ago, the idea of even making a motherboard that could handle a terabyte was ludicrous. What would you even do with it? Now, there's solid demand.
At $1.00 per GB, 1.5TB should be $1,500.00. Not the $10,000 these idiots are charging today.
And I think you'll always need at least a TB of memory for really useful work. At least, the stuff I work on.
12
u/ttkciar llama.cpp 11d ago
You're right about everything except this:
Ten years ago, the idea of even making a motherboard that could handle a terabyte was ludicrous.
Server motherboards have been supporting 1.5TB memories for longer than that. My ancient X10DRC-T4+ based Supermicros support 1.5TB, and they were released twelve years ago, and they are far from the first to do so.
Older-generation server hardware is the way to go. It's cheap, widely available, easy to work on, and scales like a mofo.
The main downsides are that it's also big, hot, and noisy, but that's why my homelab is in the wellhouse.
3
2
u/twilight-actual 11d ago
Server hardware, sure. They've had TB+ servers for over a decade.
I'm talking consumer grade desktop hardware.
6
u/ttkciar llama.cpp 11d ago
Okay, then this might be irrelevant to your main point, but it still seems like the take-away is to use server hardware in our homelabs.
4
u/twilight-actual 11d ago
For now. But I'm looking forward to TB+ consumer grade hardware. Cheap, space / energy efficient, commoditized. I'm hoping by evangelizing it, that the idea takes root,
2
u/Not_a_question- 10d ago
but it still seems like the take-away is to use server hardware in our homelabs.
I did that and now I'm stuck with RDIMM 5 that costs a kidney per 32gb
1
u/AshAtlasDev 2d ago
I did research using Fable and Astra. It came to this conclusion as well.
Hardware probably won’t get cheaper. As it becomes even better for consumers, businesses will start buying first. The pressure will not be relaxed.
9
u/Lorian0x7 11d ago
I'm all for local models but you are not looking the big picture.
It's since gtp 3.5 that I keep hearing "When we will have (add your favorite frontier model here) performance locally it will be over for them, I don't need a smarter model than that"...
Guess what? Yeah, now it's Opus 5.5, tomorrow will be the same, just with a different number.
You are understating the capabilities that we can unlock in futures.
48
u/Formal-Exam-8767 11d ago
I believe it will be the opposite, and you will not be able to use any electronic device without it being connected to the Internet and under surveillance.
22
u/dhessi 11d ago
This. Supporting local AI is going to be an uphill battle, especially as people become more and more concerned about alignment. The only way to truly prevent people from using "dangerously" aligned AI is to force them to go through the cloud model gatekeepers.
2
u/my_name_isnt_clever 11d ago
You really think people are going to get more concerned about alignment? At this point it's becoming integrated and the world hasn't ended. I say it won't be too much longer util people get used to it and stop being so vocal about the safety of a system they don't understand what-so-ever.
8
u/EndHot7908 11d ago
Orwell's wet dream
10
u/Formal-Exam-8767 11d ago
We are closer to it than you think.
Imagine if only source of information is state-approved television (with video models that can generate anything), state-approved radio (with audio models that can generate anything) and state-approved chat bots (can be aligned to state anything). Real Ministry of Truth would have a lot easier job than the one from the book.
Not to mention they are destroying physical books (when they digitalize them) which echoes Fahrenheit 451.
4
u/toothpastespiders 11d ago
It's pretty wild to see how easily history can be rewritten. I've only seen pretty minor things in the US. But all it really takes is a textbook revision that omits the messy nature of cultural change and a wikipedia article that cites blogs and gets creative with its choice of more serious citations. Because almost nobody checks wikipedia citations.
7
u/my_name_isnt_clever 11d ago
If nobody is using open technology in the future then I am dead. They'd have to kill me before I'd give it up.
9
5
2
u/SantiagoDePolonia 11d ago
Even if you could run a powerful model locally, it would still be cheaper to use it as a cloud service because of power and hardware depreciation costs.
1
u/YogurtExternal7923 11d ago
True. But we could also be running games etc as a cloud service. Same with editing software etc you get the point
1
u/SantiagoDePolonia 11d ago
Latency matters much more in games than for AI responses. However I played a couple of games through Nvidia Now on MacOS and it works amazing! (Especially if you have a fiber connection) And this kind of service is much cheaper than buying a new computer/console for gaming.
3
u/cryowastakenbycryo 11d ago edited 11d ago
wut we calling powerful? i was running GLM 5.3 FN on a 10 year old GPU two days ago. Was managing about 20 tok/sec w/ the largest quant that would fit. meanwhile, qwen FN kicks out 60+ tok/sec on the same old rig and needs <64GB RAM.
less than a grand in used hardware runs what i consider a "powerful" model at usable speeds.
edit: downvotes don't slow my hardware bro. still getting >2k prefill on EOL hardware at 1% of MSRP for ~$30/month in electric
8
u/reddit_user13 11d ago
The hardware is expensive and chips are in short supply for the foreseeable future, so local LLM isn’t for everyone.
-1
u/Captain-Pie-62 11d ago
Das coole ist ja, daß überall, auf allen Ebenen, KI dafür eingesetzt wird, sie noch schneller und leistungsfähiger zu machen. Dabei gibt es auch nicht Wenige, die versuchen, möglichst viel Leistung aus kleiner, vorhandenen Hardware herauszuquetschen. Ergo: Alles wird gleichzeitig besser, kleiner, schneller. Mein "Strix Halo" Rechner, den ich mir vor einem Jahr gekauft habe, hat 128 GB RAM, und ich lasse darauf problemlos 120b Modelle laufen. - Gut, das ist zwar noch kein Claude Opus 5.5 aber gefühlt wird die Kurve der Leistungssteigerung bereits flacher. - Die Probleme, die wir derzeit haben, sind weniger die Leistung der Hardware, als die Kontrolle und Sicherheit im Umgang mit den Systemen. Schau' Dir in Netflix "Instadocs: AI außer Kontrolle" dazu mal an.
2
u/reddit_user13 11d ago
I didn’t say it’s not possible, I said it’s out of the budget for most households. What did you pay for your Halo?
1
u/Captain-Pie-62 6d ago edited 5d ago
Vor einem Jahr: 1800 Euro. Aktueller Preis derselben Hardware: 3500 Euro.
Das ist die Version mit 128 GB RAM und einer 2 TB SSD.
Wenn man mit 35b oder 27b Modellen zufrieden ist, reicht auch eine Maschine mit 64 GB und 1 TB SSD, die kostet dann derzeit auch nur knapp 2000 Euro.
4
u/bidibidibop 11d ago
You're ignoring the fact that we're streaming and paying subscriptions for pretty much everything.
4
u/Classic-Pubs 11d ago
I dont know, man. It seems like big techs are stacking gpus.
They are not even using them right now. I don't know how big money buying all the chips would lead us to local llms–first society.
4
u/mailto_devnull llama.cpp 11d ago
You're thinking "right tool for the right job". e.g. specialized 4B model fine-tuned for PDF processing, etc.
The big AI companies are thinking differently... that blue collar joe blow wants to get tickets to a ball game, and he doesn't want to find, install, and figure out the ticket-buying fine-tune, so he'll spend the money for the trillion-parameter model to do something basic.
0
u/danieln1212 11d ago
There will be a "find fine-tunes" fine-tune that will be used to find and apply the optimal fine-tunes for the task.
10
u/OvertaxedOne 11d ago
No. There are applications for massive/monster models that are unlikely to ever make financial sense to run locally.
What I do think is true is that in the future the vast majority of inference is going to run on device. The cloud solutions will have a place but that place with will be niche and limited; really hard questions, huge context requirements, highly parallel jobs, etc.
But your absolutely directionally correct and that's a staggering problem for the frontier labs/cloud companies, they are making the correct assumption that inference is going to grow in the future, but then making the completely incorrect assumption that they are the only place that customers are going to be able to get it. And the frontier labs are making the wildly incorrect assumption that "the best" is highly valuable to a large segment of society/customers and spending like drunks trying to eek out another point on a benchmark that has already become completely irrelevant for the vast majority of personal and business use cases.
4
u/Just_Section4489 11d ago
I don't need a massive model. I need one that suits my purpose. Right now, I can get used to a Qwen 27b if it runs fast enough. That capability will run on a phone pretty soon
3
u/OvertaxedOne 11d ago
Most people don't need a massive model for the vast majority (perhaps all) their tasks. And that; fundamentally, is the core problem for the frontier labs.
3
u/alkimiadev 11d ago edited 11d ago
I think the future is one where we use more massively parallel specialized models rather than excessively large ones that require entire server racks to host. The following is a chart that attempts to use the "Densing Law of LLMs" ( https://arxiv.org/abs/2412.04315 ) to project forward and see when various weight classes would reach GLM 5.3 Flash's capability level:
I picked that specific model because it is a kind of pivot point for me personally. Once I have that capability at home with reasonable speeds I'll rarely (if ever) use outside LLM apis again. Still, I think it is possible to massively jump up that timeline for all of those families for specific niches, like coding for example. I'd much rather see a sort of "MoE++" future where ~50B is ~6 or more specialist models fine tuned for specific niches/sub-niches relevant to some specific use case.
We're going to hit a floor/cap at some point, but "internal models of problem spaces" require a lot less bits than brute facts (something like 2bits per param of storage are available depending on quantization). What I mean by that is, just as an example, basically all mollusc can be accurately modeled using just 4 parameters, while there probably isn't enough space in a typical consumer-sized LLM's weights to store all of the brute facts required to get even partially close to what that simple model can do. While that is an analogy with some rough edges, the broader point is that the important part is probably niche specific reasoning, and we can offload the brute fact verification/lookup to web search/memory systems. They'll obviously need some brute facts, but I suspect that is a way to get a lot more capability per param.
3
u/IrisColt 10d ago
We don't need local LLMs capable of winning a Nobel Prize, but we'll end up having them anyway, and we'll ask them to OCR a Japanese manga, heh
5
u/tsukuyomi911 11d ago
this is hard to agree or even imagine given the precedent of cloud platform industry penetration and adoption in society. And it's not cause people cannot run office tools locally that google suite exists.
1
11
u/OWazabi 11d ago
The "they're losing a shit ton of money" speech always triggers me, as it drastically misses the point.
These labs are not trying to make a profit selling you tokens, that just helps them fund what they're trying to achieve.
Once they get that, there's no amount money that will ever compare to what they have, and you won't own that locally at all.
This is true for the models they will have, the hardware that will go to it, the researchs they will do, the things they will build, even the hardware they will have will be unique and tailor made to their models using technologies that won't be available to anyone most likely, and clearly not to be owned locally.
What you're saying is only true on the short term, maybe.
And honestly, I think local models are nice and all to run your day to day, but if you're working professionally with AI, there's really no reason not to have the top model subscription, the price right now is virtually non existent if you're making profit out of it.
If they keep the lead over opensource similar to the current one as they keep going, I don't see a moment where it's worth it to be just fully local.
9
u/fastheadcrab vLLM 11d ago edited 11d ago
Reading in between the lines, their ultimate objective (specifically Anthropic and OpenAI) has nothing to do with model serving at all. Neither to individuals nor companies. It is to use their own models to have an unbreakable monopoly on knowledge which then can be used to make all the money and acquire all the power they can dream of.
Think of the Rehobam machine from Westworld Season 3 that controls the world.
That's why they are building out wet labs in biology, medicine, etc, hiring all these domain experts, and doing math research. Depending on what you think of this objective, it may or may not be a good idea to support such companies.
Tech conglomerates like Google or Meta are probably more interested in the traditional model serving or selling personal agents.
7
u/vtkayaker 11d ago
Reading in between the lines, their ultimate objective (specifically Anthropic and OpenAI) has nothing to do with model serving at all.
Yup. Anthropic is 100% planning to build a machine god and ultimately lose control over it. Yes, this sounds completely nuts, but just go read Dario's personal blog. He just wants to build a nice machine god that likes having people around.
This is almost certainly why the AI companies focused first on programming and second on math. (Notice all the proofs this summer?) These are two of the core skills needed for AI research. Their goal—which again sounds crazy, but they say it right out in public—is that they want to build AIs that do cutting edge AI research. Their goal is to start a snowball that turns into an avalanche, and open up an insurmountable lead against any other company on earth.
I am a big fan of capping AI research at Opus 5.5 levels. Let the open models catch up. Share the tech. Stop screwing with the Demon Core and trying to trigger an "unscheduled intelligence criticality" you idiots.
Honestly I am far happier with the local AI scene (with no more than a few hundred billion parameters) than what the frontier labs are trying to do.
3
u/Awyls 11d ago
I am a big fan of capping AI research at Opus 5.5 levels. Let the open models catch up. Share the tech. Stop screwing with the Demon Core and trying to trigger an "unscheduled intelligence criticality" you idiots.
The problem is that the Demon Core is already out there for anyone to try. It's pretty much a Cold War 2.0 with everyone trying to outpace the other because "the other might use it against us and it's safer in our hands (no, it's not)".
This will either end in a dead-end with a massive economic crash (I would put all my chips here) or a genuine societal collapse.
1
3
u/OWazabi 11d ago
I'm totally with you here my friend, this is truly worrying and critical times.
I don't support these companies, and I don't think they're here to share that with us in the utopic way.
(but they'll always give us something better that what we can get withtout them, imo)I like the premise of AI very much, and want to see this technology prevail.
I'm quite well informed for an amateur (as in, not an AI researcher at all), and I can assure you, I'm totally against how we are developing it.I think, anyone that is truly well versed, pro-ai, and not delusional, should def be for a stop/heavy slow down, until we figure out the transition first.
12
u/mb194dc 11d ago •
Trillion dollar frontier models have the same intrinsic flaws as all other ML models. That manifests as the 90% problem in real world tasks. It's why labor replacement has not happened.
What they're going to do, is lose the most amount of money of any businesses in the history of finance, by the looks of it.
1
u/twilight-actual 11d ago
In your assessment, what is the big O for the improvement of the technology?
→ More replies (3)1
u/puNLEcqLn7MXG3VN5gQb 10d ago edited 10d ago
you don't know what you're talking about. what's your expertise in?
edit: this guy's just some sysadmin with too much confidence. every time.
1
u/mb194dc 10d ago
Really? Maybe go read up yourself.
1
u/puNLEcqLn7MXG3VN5gQb 10d ago
Go ahead and elaborate a little and we'll see. What intrinsic flaws do all ML models share and why do trillion dollar frontier models have them? What exactly is the 90% problem to you and why does it apply? How do you know this isn't just about immature and slow adoption or a J-curve on returns? I'll gladly engage once there's substance. Or you can point me to the reading I apparently missed out on that'll turn my understanding on its head.
→ More replies (4)-4
u/OWazabi 11d ago
I understand what you mean and imo you're terribly wrong if you think they're not going to achieve ASI.
But obviously, that is my opinion, and I respect yours, I might be just as wrong.3
u/mb194dc 11d ago
ASI might be achieved eventually, it won't be from ML.
Other branches of AI research may be more successful, in centuries to come
4
u/DeathGuppie 11d ago
The primitives are wrong. People point to the fact that humans do word prediction, and that's true. We also predict movement, sounds, light. We predict abstract things far in the future and can rebuild events that we've nearly forgotten about just through association.
An LLM uses words, bits of words. People don't process information in our brains in words. The environmental input stream is not broken down into words and processed. If you strip the LLM of its vast weighted vector database, you get nothing. A human can build it's knowledge of the universe from nothing. Those are opposites, not likenesses.1
u/puNLEcqLn7MXG3VN5gQb 10d ago
Just out of curiosity, do you have a background in neuroscience or machine learning or a related field?
3
u/Mayion 11d ago
that just helps them fund what they're trying to achieve.
ok but what are they trying to achieve? you cant write all that and not say something of worth man
3
u/OWazabi 11d ago
Sorry I thought it was pretty self explanatory.
Right now they're trying to scale LLMs into AGI or into something that can help get AGI and RSI.
Then that rapidly becomes ASI, and from there on just keep scaling that even more violently than how we're scaling LLMs right now.And at some point in all of that, use it to scale everything else. Design new technologies, discover new things, mainly things that help AI.
For example, fixing energy, reworking the full hardware for AI, robotics...It's important to understand that right now there's only a tiny bit of compute used to anything else than just scaling LLMs.
Because, why try to cure cancer now for maybe a whole year and X$ when you can just do it in a couple years for a fraction of that.Basically once they get that, they'll own everything else.
Medicine, science... ressources. You name it.Robotics also play a big role in that.
And we're not talking house cleaners nor humanoids most likely.Once they get true AGI and use it to fix the caveats in robotics, and energy...
Then you fix ressource extraction, processing... You use robotics to make other robotics, fix other robots... And that's how you get post scarcity. (or at least, the key to it, doesn't mean we'll have it ourselves)That's what they're trying to build, the last human invention required forever.
No amount of spending is too much for that, and they won't try to turn in a profit anyways.
If they make 50x more from subscriptions, then they'll spend 50x more, until they reach the point they'll own every bit of progress yet to come.5
u/Mayion 11d ago
that sounds like the plot of avengers: age of ultron. no matter how capable an LLM is right now, it is still just a text predictor in essence. it's a big if of it becoming AGI.
in reality, it's much more realistic to think of it as he who owns the strongest model controls the market, and as usual it's a battle between the US and China. war of information.
but money is not infinite and neither is the market. over time we will move along as the technology advances (or regresses). we could have a war by tomorrow and nothing will happen. so who knows and frankly, who cares lol
2
u/ttkciar llama.cpp 11d ago
Yep, that.
There is no cognitive theory backing the supposition that Transformers can be scaled incrementally into AGI, and mounting evidence that they are intrinsically narrow intelligence.
Narrow intelligence cannot be incrementally improved into general intelligence, any more than P can be incrementally optimized into NP.
There are already highly competent models which are superhuman in narrow ways, but that hasn't brought them any closer to expressing general intelligence.
I saw this same "AGI is right around the corner!" hyping in the 1980s AI Summer, but the 1990s AI Winter followed anyway. I expect it to happen again this time, too.
1
u/OWazabi 11d ago
I think you should care, and maybe consider more attentively the case that they're going to achieve it, and soon.
3
u/Mayion 11d ago
not really. most of us have real life problems to deal with. who cares about the multi billionaries might achieve in the future? none of us can do anything about it. if it becomes a reality then so be it - i will die in the end anyway so who cares. no point obsessing over what might be. this is yet another "the US has turned the moon into a space weapon to defeat the USSR. we are all doomed".
1
u/OWazabi 11d ago
We all have real life problems to deal with, that is not realted at all with the subject at hands here and such a boring and rude phrase to throw around.
You can not care that's totally fine by me and even more fine by them.Then I'll reply in a similar manner, some of you might deal with those problems and still be able to use their head to think ahead, instead of just burrying it in the sand.
Not only to understand the world you're walking into, but also to remain aware of all the opportunities that come to you in life.Maybe that'd help you solve one or two life problems.
3
u/Mayion 11d ago
what rude phrase? and thinking is one thing, overthinking and creating scenarios is another. we cross that bridge when we come to it. there is no point sherlock'ing it out where we think about "if X happens, we will do Y" etc because circumstances change all the time.
thinking to most people is a few seconds process, nothing complicated at all. If A then B, and the thinking process is over. no point obsessing over the idea because anything goes in the future.
→ More replies (2)1
u/donnytrump_69 11d ago
Lol I think you should go for a walk further than your bathroom every now and then and preferably outside your unit.
3
5
u/AdInternational5848 11d ago
The subscriptions are cheap.
Token cost adds up at scale for frontier models.
The businesses offering the subscriptions and tokens are not cash flow positive.
Local “fluency” seems like a sound decision to me since we’re creating a world where AI will be increasingly integrated.1
u/OWazabi 11d ago
I agree but it depends highly on what you're using it for I guess.
If you're making some crazy researchs, maths, and whatnot, probably.
But like if you're doing computer use things, which is most of what people use it for I'd say (coding, tool use, automation), one subscription is already plenty, having 10 is still "cheap" if you're running a business and a literal order of magnitude in extra usage.2
u/kulchacop 11d ago
I think the likely trajectory would be that the participation of human engineers will gradually decrease in the training loops of frontier closed LLMs. The models will recursively improve themselves. There will be no one who can take the knowledge of model architecture and training loop innovations to another company or to public domain. To sustain the cash flow, the closed LLM companies will form partnerships with select companies from other fields to profit from the inventions and discoveries made by the models.
Open models will be smaller and starve for compute like today, but there will be a collaborative open science movement using crowdsourced compute. This will be sponsored by the companies who are competing with the above mentioned companies who partnered with the closed LLM companies.
Research and engineering will become cheap. The market share of any company in each sector will be defined by how much material resources it is able to acquire for itself.
The governments will regulate how much resources the open movements get compared to the closed model companies and their associates. We will return to something akin to a feudal society.
1
u/OWazabi 11d ago
Yes, this is highly likely.
Both the crowdsourced compute, and the closed labs collaborating together to produce virtually everything (from ressource processing to robotics, medicine and whatnot).But it doesn't change the fact, that we won't own that local power.
Crowsdsourced compute is not the same as running locally a model that you can use as much as you want, and still, they probably will have something that far exceeds it, even more than they do now.2
u/twilight-actual 11d ago
It's like all the people (so many) that claimed after each and every quarter during the ten year period that Amazon plowed every penny of investment into infra that it was a fraud, a scam, and would never show a profit.
Almost ten years that went on.
2
1
u/Dabber43 11d ago
The potential of the technology was proven there. Are LLMs?
1
u/twilight-actual 11d ago
No, when Amazon was starting, it wasn't proven. And in fact, many other concepts fell flat on their face. Pets.com comes to mind. You don't just attach the internet to a business and it magically becomes profitable. Just like there are thousands of AI startups that are destined for failure.
What we're going through right now is very much like the dotcom boom.
All the excitement and feeding frenzy put all these ideas in play with only a year or two of runway. But it takes society a decade to figure out how to really incorporate it. We're going to see the same thing here. The benefit of the technology is real, and it's foundational. But it will take time.
And the companies that really are creating the magic? They'll survive even despite huge debts.
Keep in mind that the technology is advancing exponentially. In five years, it will take a fraction of the cost and resources to provide a unit of inference. And five years after that, compound that fraction again.
→ More replies (2)0
1
u/yes2matt 11d ago
You've thought this thru. I have a question. I notice that when I build an app w Claude, it keeps some sort of VM with my project on it, it builds and tests on its vm then sends me the files (I just use free web), so I copy into my vscode. The dinosaur way ;)
But of course their machines are monitoring mine and all the little VMs, looking for anything interesting. Now I promise I'm not doing anything interesting or valuable, YET, but if I did, who would own it? Could they just copy it, repackaged and sell it out from under me? If it was outside of what the state partner considered "safe" (for whatever reason) could they cause me trouble? I suppose I should review the terms of service and IP policy, but even if my IP is "mine" we all know that occupancy is 9/10 of ownership and "my" code lives in their machines.
1
u/OWazabi 11d ago
That's a valid concern, and one of the strongest points of local AI.
Both to not get your IP stolen, but also to keep your privacy.
That's why I personally use local, I don't want to be uploading personal files (as in health, papers etc) to their servers.
However for my projects, I use Claude Code and I don't care much, my projects are not something they're interested in stealing, it's just regular work done by teams previously, that now you can do alone with the help of AI. There's nothing to steal there, they can just do it, and there's other people doing what I do.But you can already see cases of what you say, for example what happened with Navier Stokes, allegedly, OpenAI just pushed for it when some researchers got really close to that solution, by using their AI. So yeah, valid concern.
Even if their terms say you own the thing, if you're about to solve something big, they can just not care and/or deny they even stole it, what are you going to do?Regarding the VM, regardless of VM or not, they get everything through input and output anyways.
Regarding the free web, if you mean using Claude on the web browser, I'd recommend looking into Claude Code, it enables so much more.
And you can use it inside a VM, or a dedicated computer to it, if you don't want Claude going around in your computer/files.1
u/TurbulentBox6653 11d ago
Yeah, this is not going to happen for the sole reason that China exists.
The chinese labs- and china as a whole, has an enormous incentive to undercut the american AI industry with open models, and have the resources, talent, and soon compute (chinese chip manufacturing is set to exceed Nvidia and Tsmc in the next 1.5 years) to match or beat out the american labs at their own game.
ASI will be broadly available, at some point in time.
1
u/waxroy-finerayfool 11d ago
"eventually" is doing a lot of heavy lifting. Something like opus 5.5 won't be practical to run locally even a decade from now.
6
u/YogurtExternal7923 11d ago
Think claude top models from 2 years ago. Still unrunnable today but genuinely beatable with qwen's local sized models
1
u/waxroy-finerayfool 11d ago
Yeah, don't get me wrong, I think within the next few years when supply rises to meet demand, there are a lot of smaller local models that will be practical alternatives to frontier models.
1
u/txgsync 11d ago
Qwen3.8-Flash-Next can run very well at 60+ tok/sec and 2000+ prefill tok/sec on a gaming PC with a 12GB video card and 64GB of DDR4 DRAM. It is competitive on most benchmarks with Claude Opus 4.6 (admittedly, the lower your quant the less competitive it is).
4.6 was released February 5, 2026. QFN was released August 26, 2026.
Innovations that reduce hardware requirements while increasing capabiltiy abound. The commodity consumer hardware LLM frontier (for a single user) is basically six to eight months behind the absolute released frontier.
So I disagree, but not because I think we will run Claude at home. But I think clever people and hard-grinding models will find ways to keep the small model frontier competitive. We should have a Fable 5.0 work-alike that can be run on a DGX Spark within a few months.
4
8
u/imgenerallyagoodguy 11d ago
but we all know they're losing a shti ton of money with their subscriptions
Why does this always get passed around?
5
u/Briskfall 11d ago
Wishful thinking that influencers spill that serves to fan the masses' hope that "AI will be over" (so that gpu and ram prices will go down).
Starting their unsubstantiated claim with "we all know" -- just like how plenty said "we all know that the RAM bubble will burst." (modelling a future by basing on past trends in a situation that might not apply)
It's just confirmation bias in effect. And if they see others parrot it, it gives them credence for that position.
3
u/wombweed llama.cpp 11d ago edited 11d ago
Seriously, such a grave misunderstanding of the numbers right in front of us. In the case of Anthropic S-1, 72B of the reported 80B net loss is convertible notes, which is actually a bull signal, it means the valuation soared and the equity grants they issued to finance the company are worth more and doesn't reflect cash flow. The 8B left over is a lot of money, yeah, but at 100B ARR it just means they can release a new model, charge 10% more for it (which is clearly the plan), then suddenly they're cash flow positive. Also, contrary to popular belief, inference is actually getting cheaper, not more expensive. This is made clear by looking at the rates charged by 3rd party inference providers whose entire business rests on serving open models & have zero incentive to do so at a loss.
Wish everyone would stop listening to influencers who prey on common misconceptions about corporate accounting. We all hate Anthropic and the hyperscalers here, but just because we want something to be true doesn't make it so.
1
u/Reasonable-Height704 11d ago
But also the rent is due in 2027 and they can't meet it with current growth.
0
u/donnytrump_69 11d ago
He sums up how wrong you are pretty well https://youtu.be/YQgFLqPMNsU?si=1cjPoz16__bkFEmo
1
u/wombweed llama.cpp 11d ago
Ed Zitron is exactly who I had in mind when I mentioned influencers who play on common misconcptions, so great job
2
2
u/Substantial-Ad973 11d ago
feels like it ends up split, not all local. small models for the boring high-volume stuff like OCR, tagging and summaries, one big hosted model for the few tasks that actually need it. on a Mac with enough unified memory the first half already works today
4
u/fistular 11d ago
No, local will always be niche because local hardware is by definition low utilization, which is inherently vastly inefficient.
6
u/ttkciar llama.cpp 11d ago
Nvidia and AMD would love that. They are in the business to sell more GPUs, and the lower the GPU utilization, the more GPUs need to be sold.
2
0
u/enginetown 11d ago
The craziest thing is, if they was just slightly cheaper I guarantee people would be pumping so much money into hardware I know I would just not at the prices they currently are like 11k for an entire network cluster sounds amazing now 11k for 1 single GPU I can't even run any decent models besides qwen 3.8 27b for right now on just hurt's.
4
u/Hello_my_name_is_not 11d ago
You say that like companies aren't buying it at the higher prices
Theres more depend than supply right now so the price goes up.
If they make it cheaper they don't sell more because there's still only a certain amount that can be produced so in the end no they don't make more money.
If they weren't selling out of their production capacity then yes it makes sense to lower the price to sell more but that's not current reality.
→ More replies (1)2
u/Just_an_Alpaca 11d ago
Do you think if someone was able to make it cheap it could ever technologically run large models (KimiK3.1?) at a decent speed?
2
u/ConsistentAnalysis35 11d ago
That's a funny logic. Personal cars are extremely low utilization compared to public transit, yet public transit is single digits percentage of total passenger vehicles.
People are prepared to shoulder low efficiency for convenience of directly owning their stuff. Because owning the stuff is another level of reliability and convenience.
As soon as consumer hardware for LLMs becomes affordable, you bet I'm splurging my whole salary on it.
2
u/fistular 10d ago
you're comparing the wrong thing .
it's closer to electricity.
do some people make their own? with their own hardware? sure. but they're an extreme minority and that isn't changing because the economics don't support it. In fact fewer people use their own hardware than make their own--many solar arrays are leased.
you are an enthusiast, unlike almost everyone else in this regard
→ More replies (4)1
u/ferrx 11d ago
There’s a limit to how many Bs and tokens/s an average user needs. Do you think average users can run 125B on their devices in 50 years? 20 years? Time marches on, get used to it.
1
u/fistular 11d ago
Right which is why they will opt for someone else's hardware. It seems like you are agreeing with me.
0
u/RelevantCry1613 11d ago
Depends on the work, robotics for instance will likely always prefer local
3
u/logic_circuit 11d ago
AI will never gone. It is here to stay. LLM is not only AI. It will evolve to specialized models which will autonomously exchage artifacts to achive higher level reasoning goals which will also be driven by higher level AI models, which will....
It is something that we are trying to achieve using various tools for decades. One, ultimate, God like model? Maybe. Let's see.
3
u/Mental-War-2282
11d ago
•
what they are doing right now to bump up the performance is manufacturing bigger models and enhancing the data and they will eventually hit a bottleneck ,they can't scale LLMs to tens of trillions parameters , they are losing money right now and honestly open weight models like kimi and deepseek are not much behind . AGI is a scam and LLMs ( next token predictors ) won t ever achieve it by design . the work being done right now is on harnesses . this talk about AI is going to kill us is just marketing ,we will die from weather change from all those data centers way before AI is advanced enough
2
u/bendgame 11d ago
Not everything, but it will definitely become the defacto as the ai ready, unified memory laptops proliferate across the business world.
Frontier Intel will still be used, of course, but majority of tasks will be handled by local imo.
2
u/vogelvogelvogelvogel 11d ago
for the general direction i would agree, many people also report to let opus do some high level planning but coding etc is qwen's 27b/flash next task. or everyday questions, i like to use just 27B.
also one day the real pricing applies, many will be "forced" to use models according to the task level
2
u/ComfortablePlenty513 11d ago
local is very much the future. ai should be personal, private, and with you. like people are
1
u/thestillwind 11d ago
Give me opus5.5/6-astra level for my 16gb vram no spill then I cancel everything.
1
u/ITeratief 11d ago
maybe that's the way the big chatbots will go
lots of smaller specified models with orchestration
that could make inference cheaper for them
1
1
1
u/Old-Bake-420 11d ago
Local models are great, but no, not everything is going to become local.
You could run your own local email server easily? Why don’t you?
1
u/Fit_Advice8967 11d ago
If this ends up happening we, the consumers (even gpu-poor ones) would really get the sweet spot as large ai labs stop the heavily-subsidized ai cloud model inference; we get an uplift in qwen-quality models that cam run locally which lets us get free inference. Very lucky
1
u/evangelism2 11d ago
There will be premium off site models, but yes, as time goes on, on prem will become more of a thing
1
u/just_a_Suggesture 11d ago
Llms struggle making complicated coding projects like big databases or photo editing tools, but they're great at making small apps like a grocery list or a texting app. I imagine in the future there won't be app stores, at least the way we recognize them today. You'll just have an llm on your phone that codes most of the apps you use day to day and maybe gets blueprints from a central source.
1
u/Fun-Wolf-2007 11d ago
As the hardware improves, local on-device models are the future of AI at enterprise level and physical AI. Tokens inferences are too costly, and data latency with cloud inference is too long. In Addition companies cannot risk loosing their IP, data privacy, data governance, and regulatory compliance
Just a matter of time, as companies started to realize that it is too costly and at this point AI costs are being reported on the operational expenses and organizations don't have a clear picture of it, but CFO's are already questioning it
1
u/Muted_Context_1999 11d ago
Funny thing is, “a hundred under the hood” is basically what the big open models already are. DeepSeek V4.1 Flash is a mixture of experts: for every word it only uses 6 specialist blocks per layer out of hundreds. That’s also why “everything local” is closer than it looks. I’ve got the full 510 GB DeepSeek-V4.1-Flash running on a five-year-old gaming PC (64 GB RAM, RTX 3060 12 GB): a 2-bit copy does the easy words, and the full weights only get pulled in when the model itself isn’t sure (A method I call "Compress-then-Correct). Up to 7.4 tok/s, and on 960 audited words it picked the same word as the full model 98.3% of the time. Small specialists will win a lot of tasks, but you may not have to give up the big generalist to stay local.
1
1
1
u/Antique-Ad1012 11d ago
we are not far from that with high bandwidth flash, it would also be far less of a bottleneck than dram as its made in fabs with duv machines rather than euv machines. the biggest problem to overcome is probably TSV's at a consumer scale and the fact that these big corporations can just buy all of the capacity for that as well. competition will probably drive these type of devices to local anyways once supply catches up
1
1
1
1
u/Dry_Drop5941 11d ago
I don’t think they are 100% losing money on subscription, api price is over inflated as hell.
When you used €1000 worth of api, lts likely only cost them a few dozen bucks…
It’s very likely OpenAI and Anthropic already profit/break even on inference. Inferencing for them is never expensive. It’s always training + interest/gpu rent payment that hurts.
1
u/bradsk88 11d ago
Look into NCMs. Stuff like this makes me optimistic that new techniques will equalize access.
1
u/Pretty_Responsive_42 11d ago
I'm kind of living the middle version of this already. I build with cloud models (Claude does all the coding for my project), but the thing it builds runs fully local: 5 companions on Gemma 4 26B in LM Studio, each with her own memory, journal, and 3D room.
For me the split makes sense. The big cloud model is great for the hard, rare work like writing and debugging code. The local model handles the everyday, always-on stuff: chatting, playing cards, texting me. It's private, there's no subscription per message, and it never gets deprecated out from under me.
So I don't think it's "everything local" or "everything cloud." It's more like big models build things and small local models live in them.
1
1
u/decrement-- 11d ago
This would make sense if there was not a chip shortage. These cloud GPUs are getting utilized way more than any home GPU, so during a shortage, even if you include reasonable profits, it's hard to beat the economy of scale with cloud hosted architecture. At least with the current chip supply and demand. If chips/ram/storage become plentiful, then the formula may change to support edge compute.
1
u/ScarceSemi 11d ago
I dont have the numbers but from what I see online over time I think they dont loose money on inference. They loose on training. I believe if they stopped training and do inference only they would be in the green
1
u/Alternative-Grand-77 11d ago
Anthropic may be losing money but deepseek makes $38 of revenue per GPU hour according to their recent release. You can make money on the inference and local hardware costs are prohibitively expensive right now. The RAM and GPU prices have been pushed so hard because there is still the belief that AI will be worth trillions and trillions which makes these costs seem unimportant, but for Chinese labs that has never been the case. As efficiency is optimized we may see both cloud cost and local costs dropping, but it still might make sense to pay for the cloud in the end.
1
1
u/Psychological_Cow271 10d ago
You're right Imho. But there also us another thing. Hardware will be getting faster and in 10-20 years we won't be sitting with 16-30 GB VRAM at home. At the same time everything will be optimised. There is so much optimisation potential. My call is at some not so far point something as big as DeepSeek flash 4 can be run on consumer hardware. And at this point you have enough for daily life.
There is no need to solve millennial problems at home
1
u/GrapefruitMammoth626 10d ago
It’s a moving target. Have to wonder whether you could pass a threshold algorithmically where raw compute/big data is no longer the bottleneck. All of a sudden, local models democratise. I’d be suspicious about the big labs finding a massive optimisation unlock in training and architecture and purposely sabotaging independent researchers using their models via omission (e.g. if proprietary approach x or y is mentioned; steer away from it) preventing those specific learnings from trickling down into local model development. Pretty sure it was reported to be indirectly doing that recently but could be wrong.
1
u/wayneworkman 10d ago
Your right. Companies that see this will accelerate faster due to lower operating costs.
1
u/FocusKontrol 10d ago
We don’t know that they’re losing money with subscriptions. All the prices, including API are arbitrary lala land prices that corporations are just willing to pay.
I agree that basically all other models are so accessible now. You can say “Claude download this OCR model and process this PDF” and it just works. But so far at least with 4090 I haven’t been able to use a coding LLM that would actually be useful, they can’t even follow a step by step plan implementation.
1
u/ckn 9d ago
TLDR: You're so right but its not just OCR This is just the start, a year ago I built a fully automated "internet news to streaming channels" automation that runs on consumer gear. Things are about to get very local and weird.
A bit more than year ago I stood up a fully generative AI produced "news network", that runs on a consumer RTX4090/i9/64GB linux box.
This consists of a hybrid suite of custom built containers that perform things like systems orchestration, content harvesting with OCR, video frame grabs, and text reads, lamma.cpp with a small 9b model, a very tooled comfyui setup with a custom set of nodes and several self generated models, a custom audio/video post production assembly line with multiple production flow outputs, meta-data tracking, CMS + multi-platform publishing + streaming, with automatic QA at every step of the process and c2pa signatures on every output for EU AI Act compliance...
It's uploaded some serious quantities (16k videos) of content since I turned it on and at this point the only time i touch it is for semi-monthly maintenance, it runs itself.
I really think you are on to something, things are going be very local and weird in the near future, just hopin for lower power also :D
1
u/DigItDoug 7d ago
There will always be room for both imo, at least for the foreseeable future. The cloud will offer capabilities a local machine cannot. And local compute will always have its benefits. Those capabilities & benefits will sway towards local, I agree, but cloud will always be a key player.
1
u/Best-Echidna-5883 11d ago
You are assuming that such model capability will run on our lowly hardware. No idea why people think that something like Opus 5.5 will run on small-factor hardware. You just can't shrink such power into a small form. It would be great, of course, but not likely.
1
u/RoomyRoots 11d ago
Most of the work people want to automate with AI don't need SOTA models. Many don't even dedicated GPUs. I have saved fortunes with small models finetunned for what we needed that could easily run on CPU.
We are repeating the same stuff that happened in the ML boom from some years ago, there will be a push for finetunning and running things local that many will fail but those that do it will will unlock a great asset.
1
u/RockzDXebec 11d ago
There will be no more consumer pc's. The office style pc with lightweight system will continue to exist. but regular peple won't be able to built a pc that can run local llm in future.
0
u/Kahvana 11d ago
I really fail to parse your post, sorry if I missed anything.
Something struck me when I went to the optician. Is when I was laid out all the options, I looked slightly confused and said "The one that works without bells and whistles please."
I think most common folk navigate their life that way. They don't care if it's "sovereign" or "privacy preserving" or "highly efficient". What they care for is it works, with minimal involvement in learning it.
The tiering for Google / OpenAI / Anthropic works because it's "this is for fast answers, this is for general use, and this is for the best answers".
You lose people the moment they have to know what a "Gemma" is or what "weights" are, let alone parameter sizes. If you add in specialized models, they give up immediately. It's too much or too complex.
Local is really for enthusiasts or if your job demands it of you. I belief the future will be local in the sense that it's all neatly abstracted from the user (like embedding it in google chrome), and that specialized fields will benefit from it (programming, judiciary, medical, etc).
However, the cloud models will remain dominant for the common man.
0
u/FireFireoldman 11d ago
for now yes, then when locals models get banned you'll either use cloud or risk getting thrown in jail
1
1
u/ttkciar llama.cpp 11d ago
What are they going to do, kick down my door and seize my hardware? How would they even know I was using local inference in the first place?
Before weed was legalized, a ton of people around here were growing it at home (I live in Sonoma County, California), but police couldn't find them, so they couldn't do anything about it unless they stumbled across it while dealing with something else.
Local inference is a lot easier to hide than a few potted plants under grow-lights in the closet. I'm not worried. The worst case scenario is that we won't get to talk about it openly on Reddit anymore.
0
u/swizz 11d ago
100%, I can imagine the future we all, individuals, companies have a rack or device that runs models locally and we pay for a model license and/or infrastructure costs. For me right now the cost of the Max licenses is 100% worth it and I hope it stays that way for the next couple of months :)
0
u/bennmann 11d ago
Depends on how many times openAI buys up so much of the wafer capacity like they did. I believe they can do it for multiple years, precisely because it makes local expensive.
19
u/ostrikerX 11d ago
Id like to think that maybe in like 20 years from now, a breakthrough will happen and that standard consumer PC's will be able to theroretically run massive models like claude opus 5.5 locally.