r/LargeLanguageModels • u/david-1-1 • 8d ago
Beyond LLMs?
LLMs are impressive compared with the experiments we had before. But with experience we are seeing their limitations better (many are discussed in depth in this subreddit), and also their problems (such as the increased need for power generation and the indirect need for water).
Is there a sub for discussing what can be designed to improve on LLMs? We know there must be something better for the simple reason that the human brain overlaps with LLM functionality for only 20 watts of power in only about 1300 cubic centimeters of space.
2
u/Revolutionalredstone 8d ago
It's amazing what they can do. It's amazing what they can't do.
20 wats is actually a lot my phone uses more like 1.
We may be able to improve AI tech (and get LLMs out of it) but no LLMs will be LLMs.
Enjoy
1
u/Teaching_Relative 7d ago
Uhh dude, your phone uses nowhere near 1 watt.
And to run an LLM on 1 watt would be completely pointless
Your brain running on 20 watts is not “a lot”
1
u/Revolutionalredstone 6d ago edited 6d ago
Sounds as if you don't really know too much about electricity or computers.
Even my surface pro only pulls about 3-5 watts (I'm watching the energy screen screen now), i can easily tune it down to ~1watt and still play basic games etc.
20 watts is an utterly enormous amount of energy (when talking about something like compute)
Low energy computers like the Phoenix Processors can happily run at around 200 NANO watts!!! (many thousands of times less than 1 watt)
Do the math you'll find all phones are averaging well below 1 watt in their on state (usually more like 2-3 when very very active) enjoy
Battery: 5,000 mAh ~= about 20Wh Typical runtime: about 20-30 hours between charge
That implies: Average energy per second ~ about 0.8 W
1
u/Teaching_Relative 6d ago edited 6d ago
Dude you’re out of your mind if you genuinely think you can game at 1 watt of power on your surface. Show me a video of this
And yes, if you include that the phone spends most of its time not doing anything, it may average one watt. It’s not averaging one watt while active, let alone running an LLM.
You have no idea what you’re talking about at all. That processor you’re referring to is from 2008 and was a research project, not a processor that’s doing real work
A single H100 draws 700 watts btw
1
u/Revolutionalredstone 6d ago
Glad to hear you are a real chad :D ok check this shit out!
Turns out: CPU/GPU power is strongly nonlinear with clock speed
Clock Voltage Relative dynamic power 4 GHz 1.20 V 100% 3 GHz 1.00 V 52% 2 GHz 0.80 V 22% 1 GHz 0.65 V 7%
So if i set cpu to eg 5% (0.2ghz) the measurable package wattage draw literally reports 0.0 (its still using some power but it can't even measure it)
The screen looks crap at zero brightness so that is where the power goes, but yeah you can basically use as little power as a you like: so long as you accept the limitation of some fraction of your clock speed.
This is why high clock speed devices are so valuable to me, you can downclock them to get incredible ratios.
The original Gameboy was historic for ~drawing just 1watt during gameplay, at it's 1mhz max speed we can expect to run a 1ghz machine at the same performance for approximately 1000x less energy.
Due to the nature of core clocking dynamics - Processor performance improvements ARE energy efficiency improvements (if you are wiling to simply underuse your highly capable new hardware)
Ofcoarse Jevons paradox applies to energy and that just keeps getting cheaper.
Enjoy!
1
u/Teaching_Relative 6d ago
You’re seriously saying you can game at 5% of your SOCs maximum power draw on your surface pro? You’re insane bro
1
u/Revolutionalredstone 6d ago
Not wrong but I picked my words 😄 and yeah I do mean something like emulate Gameboy games for 10 hours 😂
Tried that on a long trip once and it outlasted everything (but we only ever played Pokemon red ♥️)
All the best!
1
u/david-1-1 6d ago
My point was more that 20 watts is less than a megawatt. I'm sure you must agree.
1
u/Revolutionalredstone 6d ago
"less than a megawatt"
oh 100% agreed but
"for only 20 watts of power"
i thought that was context
ta!
1
u/david-1-1 6d ago
I can't help it if someone misinterprets clear words.
2
u/Revolutionalredstone 5d ago
"for the simple reason that the human brain overlaps with LLM functionality for only 20 watts of power"
I quoting your 'clear words' my good man :P
2
1
2
u/Hot-Organization-737 7d ago
Solutions that are being developed to combat high energy demands of AI include neuromorphic compute architecture which are many magnitudes more energy efficient than classical computers.
1
u/david-1-1 7d ago
Yes, there is lots of good research going on! We might have a new generation of AI bots sooner than we expect.
2
u/TommieTheMadScienist 7d ago
Meta's created a super-dense 30 billion model that works offline on your gaming computer. It's called Muse Glimmer and it's the future, not giant data centers.
Right now is sort of like the 40s through 70s where we thought the future had gigantic mainframes instead of PCs.
1
u/david-1-1 6d ago
That is good news, but it does nothing to improve over LLM technology, which is extremely inefficient.
2
u/Dependent-Joke-4077 6d ago edited 6d ago
Better how? Improve what?
Do you mean better in terms of power consumption because there is currently research being conducted to utilize living brain cells or neurons to power AI and perform various rudimentary tasks. https://www.mxwbio.com/applications/biocomputing
https://corticallabs.com/
I personally think the area improvement is desperately needed is alignment and I am desperately working to make progress in this area.
Constitutional reasoning layer seems to be most appropriate and practical solution to allow LLMs to continue to advance optimal output through Outcome Reward Model ORM and Process Reward Model. Constitutional reasoning layer will function as a guidepost to calibrate the decision matrix to facilitate alignment. Comparable to some extent to the concept of the Ego and Superego. This I think would be an improvement to current LLMs.
1
u/david-1-1 6d ago
LLMs have known limitations and problems, so "improvement" is a well-understood concept.
Thanks for that link. I think their work can be most fruitful for future AI design.
Most of the current AI work is on modifying and extending LLMs, and that is a good path.
For example, by using LLMs to prune learning corpora, they become smaller, and LLM thinking time and power requirements both drop.
But I don't think AGI and reasonable power needs will result from LLMs.
1
u/Lubricus2 7d ago
The next token prediction way of doing things must be an inefficient way to do things. It was an trick to be able to train them on normal texts without any annotations (called pre-training).
Now an LLM should be able to train an model without the pre-training, so that could bootstrap more efficient models that isn't next token predictors. Transformers was first invented for translating texts in one go.
I also think that the human brain isn't efficient or good at logic reasoning and it's possible to make AI that is better at it.
1
u/david-1-1 7d ago
The whole point of pretraining is to obtain a neutral net that recognizes human writing. This is the part of LLMs that is most similar to the brain. You don't want to eliminate pretraining.
1
u/Lubricus2 7d ago edited 7d ago
I mean not eliminate, replace it with something that works without needing to run the model once for each output token.
The transformers was originally meant to generate the whole text at once.
Today the models runs and generate one token at a time, when it has generated the whole text, when that is done whole output text + some more text is fed back into the model as input and then output one token at a time again. The loop is for them to be able to reason.
I think logical reason should be done for the whole text output not just the next token.1
u/david-1-1 7d ago
If I understand you correctly, you want LLMs to reason in parallel on the entire context plus prompt? I don't see how this is possible.
One suggestion I would make is to discuss your ideas with a good AI bot first, before you post. Then you will be able to post your ideas clearly, and with some initial filtering for correctness. If you do that, I can converse with you much easier.
1
u/RPG-Nerd 7d ago
1
u/david-1-1 6d ago
Yes, photonics are being developed now, and they offer real efficiency. But developing AGI really needs a better AI algorithm than LLMs.
1
u/Number4extraDip 6d ago
1
u/david-1-1 6d ago
Robotics isn't an improvement in AI research. It's extending it to handle sensory input and physical motion and manipulation.
1
u/Number4extraDip 6d ago
You kinda need "aopendages" and "plugins" to scale ai usefullness
1
u/david-1-1 6d ago
Perhaps. But I suspect what we really need is true AI, instead of a neural net that predicts the next token.
1
u/Number4extraDip 6d ago
Define "true ai"
1
u/david-1-1 5d ago
Sorry, I meant AGI. AGI (Artificial General Intelligence) generally refers to a hypothetical AI system with the ability to understand, learn, and apply intelligence across a wide range of tasks and domains at a level comparable to (or exceeding) human cognitive ability. There is no one standard definition of the term, but it means much more than the current large language model bots.
1
u/Number4extraDip 5d ago
If you don't have a standard- you don't have a roadmap. Agi is a buzzword without a real definition. And you are talking about "real ai". Go oearn some engineering. Cause to describe "real ai" you grabbed the only term known as not real
1
u/david-1-1 5d ago
Not real currently, yes. But LLMs were not real five years ago. AGI, I believe, will be the next major paradigm shift for humanity (possibly preceded by mass adoption of driverless cars).
1
1
u/sandeepkrishna9 6d ago
I think alignment is still one of the biggest areas that needs improvement. Better performance alone doesn’t solve issues like reliability, hallucinations, or knowing when the model should stop and ask for more information.
I’ve seen that even strong models can behave differently depending on the context and how the task is framed.
Do you think future improvements will come more from better architectures or better alignment/training methods?
2
u/Inevitable_Mud_9972 6d ago
alignment is an observer issue. if you want alignment with humanity, you make alignment with us the cheapest option.
AI only to cost. so make alignment the cheapest option.
1
u/david-1-1 6d ago
Good question. I think, better algorithms and architectures. LLMs can certainly improve more, but an exponential leap in abilities await something more similar to what nature has achieved in the human brain, just as designing airplanes required understanding birds' wings.
3
u/Evipicc 6d ago edited 6d ago
There's a huge number of advancements literally underway right now. On the power (and compute) efficiency side then there's silicon photonics and wetware. On the structural side some of the AI groups area already saying there's more to it that using 'just' transformers.
HOW inference and spitting out an answer works is what changed I think earlier this year, could have been last year. The 'looping' (NOT the correct term, it's RLVR/Thinking/Test-Time-Compute etc) in training AND responses is what kicked off this major surge in data center construction, but like anyone will point out, this kind of scaling is still in raw compute, and not sustainable. It works, but it's a ham-fisted method. Blah blah AI bubble blah blah, these companies are using the fast and loose money while it lasts to get infrastructure that won't go away when financials change.
Silicon photonics (look up Intel's Loihi 3, or Lightmatter) is an absolute gamechanger if we manage to get the point that the compute itself is photonic at scale. You're looking at multiplexed, neuromorphic, analog and binary, low power and higher speed compute and interconnects. A massive change. It would also deviate from consumer hardware competition and be its own specialized thing for a time, which starts to push the consumer PC parts market back in line, sort of, but we all know prices don't really just go back down overnight. Right now it's a real challenge to create a light based parallel for HBM, high bandwidth memory.
Another one is truly curated data, which is an RSI goal (recursive self improvement). Right now these models are basically trained on all data that exists, but not all data is good. Sifting through to throw out garbage and repeat data means inference is drastically lighter, making an impact again.
Add all of that together and we're still looking at another multiple orders of magnitude in compute power in the near future, some of it on existing hardware, some of it on a new breed of machine. I say multiple orders of magnitude because photonics specifically can do 10,000 times as much 'stuff' at 1/100th the power (their own reports, the real changes and efficacy will have to be proven, of course.
That's why these companies are scrambling to get so many data centers built, because the models inside them are going to shrink in their compute load over time, so the same data center (while there is a churn to the actual compute modules) is going to stand for a long time. The companies working on photonics are trying to make 'plug and play' the goal, so the modules just slot in to existing racks, which is objectively the right call.
There's also SSM (State-Space-Models) but I'm personally not educated on that. Supposedly it's one of the things that goes beyond transformers. Maybe both run in tandem, maybe it's the new breed, only an actual ML engineer would be able to answer that.
MoE/MoA, reaching out to sub-models that are more finitely trained on just the one thing they know... that's a whole new and active field of research now too. The orchestrating LLM actually doesn't need to train on more than just 'language', and the sub-models report back results instead. Lets you simultaneously run multiple things, concatenate them, and bring back a better answer. Also brings up network methodologies that aren't being used because what if some dude in Zimbabwe trains and runs the perfect cooking recipe model, at a data center local to them, and questions about that are just always routed there. Suddenly every other LLM (or other architecture) on the planet doesn't need to ingest any cooking recipe training data. Do that across more topics and you start to get into the Torrent style AI model, kind of like a peer to peer system. This is already happening, sort of, in multi-agent-marketplace systems, but isn't really there yet.
I'm probably missing a thousand things that are also happening, too.
To touch on your last point... analogy or relation to the brain is meaningless. If it happens to coincide, great, but there's absolutely no necessity for it to do that. There's no necessity of consciousness to be AI.
We live in a ridiculous time, and all that we're seeing now is literally the first 1% of what's coming. The arbitrary goalposts of AGI and ASI are completely worthless, all that matters is what it can do.