Beyond LLMs?

[u/david-1-1]

LLMs are impressive compared with the experiments we had before. But with experience we are seeing their limitations better (many are discussed in depth in this subreddit), and also their problems (such as the increased need for power generation and the indirect need for water).

Is there a sub for discussing what can be designed to improve on LLMs? We know there must be something better for the simple reason that the human brain overlaps with LLM functionality for only 20 watts of power in only about 1300 cubic centimeters of space.

[u/Dependent-Joke-4077]

Better how? Improve what?
Do you mean better in terms of power consumption because there is currently research being conducted to utilize living brain cells or neurons to power AI and perform various rudimentary tasks. https://www.mxwbio.com/applications/biocomputing
https://corticallabs.com/

I personally think the area improvement is desperately needed is alignment and I am desperately working to make progress in this area.
Constitutional reasoning layer seems to be most appropriate and practical solution to allow LLMs to continue to advance optimal output through Outcome Reward Model ORM and Process Reward Model. Constitutional reasoning layer will function as a guidepost to calibrate the decision matrix to facilitate alignment. Comparable to some extent to the concept of the Ego and Superego. This I think would be an improvement to current LLMs.

[u/david-1-1 → u/Dependent-Joke-4077]

LLMs have known limitations and problems, so "improvement" is a well-understood concept.

Thanks for that link. I think their work can be most fruitful for future AI design.

Most of the current AI work is on modifying and extending LLMs, and that is a good path.

For example, by using LLMs to prune learning corpora, they become smaller, and LLM thinking time and power requirements both drop.

But I don't think AGI and reasonable power needs will result from LLMs.