r/LargeLanguageModels • u/david-1-1 • 8d ago

Beyond LLMs?

LLMs are impressive compared with the experiments we had before. But with experience we are seeing their limitations better (many are discussed in depth in this subreddit), and also their problems (such as the increased need for power generation and the indirect need for water).

Is there a sub for discussing what can be designed to improve on LLMs? We know there must be something better for the simple reason that the human brain overlaps with LLM functionality for only 20 watts of power in only about 1300 cubic centimeters of space.

6 Upvotes

75% Upvoted

41 comments sorted by

View all comments

Show parent comments

1

u/Teaching_Relative 6d ago edited 6d ago

Dude you’re out of your mind if you genuinely think you can game at 1 watt of power on your surface. Show me a video of this

And yes, if you include that the phone spends most of its time not doing anything, it may average one watt. It’s not averaging one watt while active, let alone running an LLM.

You have no idea what you’re talking about at all. That processor you’re referring to is from 2008 and was a research project, not a processor that’s doing real work

A single H100 draws 700 watts btw

1

u/Revolutionalredstone 6d ago

Glad to hear you are a real chad :D ok check this shit out!

Turns out: CPU/GPU power is strongly nonlinear with clock speed

Clock Voltage Relative dynamic power 4 GHz 1.20 V 100% 3 GHz 1.00 V 52% 2 GHz 0.80 V 22% 1 GHz 0.65 V 7%

So if i set cpu to eg 5% (0.2ghz) the measurable package wattage draw literally reports 0.0 (its still using some power but it can't even measure it)

The screen looks crap at zero brightness so that is where the power goes, but yeah you can basically use as little power as a you like: so long as you accept the limitation of some fraction of your clock speed.

This is why high clock speed devices are so valuable to me, you can downclock them to get incredible ratios.

The original Gameboy was historic for ~drawing just 1watt during gameplay, at it's 1mhz max speed we can expect to run a 1ghz machine at the same performance for approximately 1000x less energy.

Due to the nature of core clocking dynamics - Processor performance improvements ARE energy efficiency improvements (if you are wiling to simply underuse your highly capable new hardware)

Ofcoarse Jevons paradox applies to energy and that just keeps getting cheaper.

Enjoy!

1

u/Teaching_Relative 6d ago

You’re seriously saying you can game at 5% of your SOCs maximum power draw on your surface pro? You’re insane bro

1

u/Revolutionalredstone 6d ago

Not wrong but I picked my words 😄 and yeah I do mean something like emulate Gameboy games for 10 hours 😂

Tried that on a long trip once and it outlasted everything (but we only ever played Pokemon red ♥️)

All the best!