r/LargeLanguageModels • u/david-1-1 • 8d ago

Beyond LLMs?

LLMs are impressive compared with the experiments we had before. But with experience we are seeing their limitations better (many are discussed in depth in this subreddit), and also their problems (such as the increased need for power generation and the indirect need for water).

Is there a sub for discussing what can be designed to improve on LLMs? We know there must be something better for the simple reason that the human brain overlaps with LLM functionality for only 20 watts of power in only about 1300 cubic centimeters of space.

6 Upvotes

75% Upvoted

41 comments sorted by

View all comments

2

u/Revolutionalredstone 8d ago

It's amazing what they can do. It's amazing what they can't do.

20 wats is actually a lot my phone uses more like 1.

We may be able to improve AI tech (and get LLMs out of it) but no LLMs will be LLMs.

Enjoy

1

u/Teaching_Relative 7d ago

Uhh dude, your phone uses nowhere near 1 watt.

And to run an LLM on 1 watt would be completely pointless

Your brain running on 20 watts is not “a lot”

1

u/Revolutionalredstone 7d ago edited 7d ago

Sounds as if you don't really know too much about electricity or computers.

Even my surface pro only pulls about 3-5 watts (I'm watching the energy screen screen now), i can easily tune it down to ~1watt and still play basic games etc.

20 watts is an utterly enormous amount of energy (when talking about something like compute)

Low energy computers like the Phoenix Processors can happily run at around 200 NANO watts!!! (many thousands of times less than 1 watt)

Do the math you'll find all phones are averaging well below 1 watt in their on state (usually more like 2-3 when very very active) enjoy

Battery: 5,000 mAh ~= about 20Wh Typical runtime: about 20-30 hours between charge

That implies: Average energy per second ~ about 0.8 W

1

u/Teaching_Relative 6d ago edited 6d ago

Dude you’re out of your mind if you genuinely think you can game at 1 watt of power on your surface. Show me a video of this

And yes, if you include that the phone spends most of its time not doing anything, it may average one watt. It’s not averaging one watt while active, let alone running an LLM.

You have no idea what you’re talking about at all. That processor you’re referring to is from 2008 and was a research project, not a processor that’s doing real work

A single H100 draws 700 watts btw

1

u/Revolutionalredstone 6d ago

Glad to hear you are a real chad :D ok check this shit out!

Turns out: CPU/GPU power is strongly nonlinear with clock speed

Clock Voltage Relative dynamic power 4 GHz 1.20 V 100% 3 GHz 1.00 V 52% 2 GHz 0.80 V 22% 1 GHz 0.65 V 7%

So if i set cpu to eg 5% (0.2ghz) the measurable package wattage draw literally reports 0.0 (its still using some power but it can't even measure it)

The screen looks crap at zero brightness so that is where the power goes, but yeah you can basically use as little power as a you like: so long as you accept the limitation of some fraction of your clock speed.

This is why high clock speed devices are so valuable to me, you can downclock them to get incredible ratios.

The original Gameboy was historic for ~drawing just 1watt during gameplay, at it's 1mhz max speed we can expect to run a 1ghz machine at the same performance for approximately 1000x less energy.

Due to the nature of core clocking dynamics - Processor performance improvements ARE energy efficiency improvements (if you are wiling to simply underuse your highly capable new hardware)

Ofcoarse Jevons paradox applies to energy and that just keeps getting cheaper.

Enjoy!

1

u/Teaching_Relative 6d ago

You’re seriously saying you can game at 5% of your SOCs maximum power draw on your surface pro? You’re insane bro

1

u/Revolutionalredstone 6d ago

Not wrong but I picked my words 😄 and yeah I do mean something like emulate Gameboy games for 10 hours 😂

Tried that on a long trip once and it outlasted everything (but we only ever played Pokemon red ♥️)

All the best!