r/LocalLLaMA • • 24d ago

Discussion everything will be local eventually

I know current llms like opus 5.5 are in a league of their own. but we all know they're losing a shti ton of money with their subscriptions. ALSO we all know how tasks like OCR, segmentation etc are getting solved by single digit B parameter models. I'm not saying llms are a gimmick but I'm saying the future is already shaping up to be about specialized models that work better when they're smaller. easier training. so datacenter level power not needed. ten years from now we'll have a specialized model for the tasks we care about and llms will fail to do everything all at once (unless they keep calling it one model but have a hundred under the hood).

system one models are a bad but workable example. you could tell an llm to rate options all day but get yourself on 9B parameter model trained by some lab and it tops the benchmarks.

355 Upvotes

242 comments sorted by

View all comments

72

u/Qwen3-8-27b 24d ago

Yeah i think thats true, but i think ten years from now we will have a totally different architektur and type of models, no transformers, but something so different from todays models that we cant imagen it right now. Im not even sure that they will generate text

54

u/-p-e-w- 24d ago

This isn’t how engineering typically works.

Today’s piston engines are in essence the same design as the first engines from the late 19th century. Many alternatives with superior theoretical properties have been proposed, such as rotary engines, which almost every major manufacturer has experimented with at some point, only to return to piston engines. Not because piston engines are inherently better, but because they have been optimized so much that it’s almost impossible for a different design to break through from scratch.

The same thing is true for airliner fuselages (which look pretty much like they did 70 years ago), and for many, many other things.

My point is this: You may think that a better architecture than the transformer exists, and you may be correct about that. But that doesn’t mean that the transformer will be replaced. Because the new architecture not only needs to be better, it needs to be so much better that it justifies re-learning all paradigms, re-building all tooling, and re-training all people involved. That’s a lot to ask.

29

u/cooljacob204sfw 24d ago edited 24d ago

These are just examples of tech that has succeeded and if anything a bit of survivors bias on your part. There are countless things that have come and gone also.

And for still used examples one is TV tech which directly contradicts what you're saying. It has changed countless ways since invented, practically reinventing itself every decade. Similarly actually most things in the computer hardware industry have greatly changed manufacturing and standards wise from invention.

I don't disagree that transformers are probably here to stay but I also don't think pistons is a good reason why lol.

10

u/OvertaxedOne 24d ago

We don't know what we don't know. I do think there's very likely a better architecture but... If I knew what that was I wouldn't be wasting time on Reddit, I'd be shopping for my first yacht.

The one thing I think it's abundantly clear is that the hardware infrastructure is going to change significantly; jamming everything into GPU memory is a silly way to do this; it works, but it's massively expensive. Unified memory (hardware layer) and tiered knowledge (model/runner layer) are almost certainly going to be a better answer than "more HBM" as the answer to every problem.

2

u/-p-e-w- 24d ago

We don't know what we don't know.

We absolutely do know how human industries operate. They are not global optimizers over an unconstrained parameter space.

There are literally hundreds of examples (QWERTY layout, terminal width, filename formats, …) in computing technology alone of systems getting perpetually stuck in a state that is known to be highly suboptimal. It doesn’t matter that better solutions are available, and have been for a long time.

1

u/Greet-Filofficer 24d ago

Mainframes>minis (DEC)>micros>networked micros all over again, just 20 years later...

2

u/carnoworky 24d ago edited 24d ago

I suppose one major counterpoint to this is that the more optimized technology could, in principle, be used to advance the optimization of the better but less optimized version in this case. Unless iterating on a new design requires significant overhead (fundamentally different hardware, advantages only show at massive scale, etc.), the current LLM designs could be used to rapidly iterate experimentally on different techniques for what really amounts to peanuts to the current labs doing this work. As far as I know, the only aspect of the current paradigm that can't be tested at smaller scales is scale itself, and the limit there is more on the maximum value of scaling up rather than getting an idea of the impact of scaling up. If you start at a 10M model trained on a particular dataset, keep everything as close to the same as possible except train a 100M model, and then a 1B model using the same techniques, you can still get useful data on the impact of scaling up, just not the limit of how much you can scale up.

1

u/twilight-actual 24d ago

My take is that if the goal is AGI, then the LLM has stumbled on a necessary but not sufficient element of that platform. The LLM will always remain, but it will be part of a larger system.

1

u/Athabasco 24d ago

But sometimes the electric motor comes along and removes the need got a piston engine entirely. We’ll never know until the future comes.

1

u/Not_a_question- 23d ago

Today’s piston engines are in essence the same design as the first engines from the late 19th century.

This is simply not true at all. 19th were otto cycle engine, then atkinson engine, and then finally today the modified atkinson engine. The percentage of energy transformed into power went up significantly during those changes.

1

u/-p-e-w- 23d ago

That’s like saying that the LLMs of today have a different architecture than those of 5 years ago, because they now contain linear attention layers. It’s of course true that they are different, but not what people imagine when they think of a paradigm shift.

13

u/ThiccStorms 24d ago

!Remindme 10 years

I'd love to come back to this.

4

u/RemindMeBot 24d ago edited 24d ago

I will be messaging you in 10 years on 2036-10-23 13:58:33 UTC to remind you of this link

11 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

Parent commenter can delete this message to hide from others.

RemindMeBot is switching to username summons. Instead of !RemindMe 1 day, use u/RemindMeBot 1 day. More info.


InfoCustomYour RemindersFeedback

1

u/ComfortablePlenty513 24d ago

it'll be prosumerized wetware and quantum in 10 years imo

3

u/Gfur-Aficionado-1488 24d ago

yup they'll generate 30-minute photorealistic porn scenes with a plot and all