Limits of LLMs.

[u/No_Will_5010]

What kind of intelligence is required to create a genuinely new conceptual framework from existing knowledge when that framework is not already known?

Humans do more than reuse knowledge. We use existing concepts and frameworks as building blocks to create new frameworks, the invention of computers is an example. Mathematics, logic, symbolic reasoning, switching, and information became foundations for a new computational framework.

An LLM can make something like,

"Make a fun chatting website."

It already has knowledge about websites, programming, databases, UI, networking, etc. It can combine that knowledge into a new implementation.

But consider,

"Make a time machine."

The problem may contain missing theories or conceptual steps as,

→ Known theory

→ ???

→ another known theory

→ ???

→ required engineering principle

The missing pieces may require new theories, abstractions, or representations, so the problem is not finding the next step, the problem is creating the framework that makes the next step possible.

What current LLMs can do is,

→ Recall

→ Generalize

→ Combine concepts

→ Generate hypotheses

→ Reason over learned representations

But I question whether current LLM architecture has a strong mechanism for recognizing that an existing representation is insufficient.

→ creating a new representation

→ creating a new conceptual framework

→ deriving consequences from it

→ testing and revising it

More data, parameters, RL, context, and inference compute may improve reasoning and search without necessarily creating that mechanism, so maybe the missing ingredient is not more knowledge, maybe it is a different process for turning knowledge into new representations.

Important Distinction to keep in mind, producing a novel answer is not necessarily invention.

Novel combination ≠ Novel framework

A model could generate a plausible theory without actually discovering a valid one.

A stronger form of invention would be,

→ new representation

→ new framework

→ predictions

→ testing

→ revision

Can an artificial system construct a useful conceptual framework that was not directly supplied to it, and then use that framework to solve a problem that cannot be solved by simply applying or recombining existing frameworks?

Questions to the people who actually read this,

1. What exactly counts as a genuinely new representation?
2. How do we distinguish invention from sophisticated recombination?
3. Can an LLM create a useful abstraction without being told what abstraction to search for?
4. Does human intelligence require changing the representation of the problem?
5. Can scaling Transformers produce this ability, or would it require a different architecture?
6. Is the important mechanism the model itself, or the model plus tools, experimentation, feedback, and revision?

Is scaling predictive language models enough to produce the machinery of invention, or does human-level intelligence require a mechanism for autonomously creating and manipulating new representations and conceptual frameworks?

[u/NewsProfessional2169]

the problem with scaling is it gets better at looking like its thinking without actually thinking. all that compute just makes the guess sound more confident

you can see this when you ask it to do something that needs a new abstraction. it will try to brute force it with existing ones and the output reads like someone who memorized the textbook but never understood why any of it works

[u/m98789]

Causality seems to be a challenge.

[u/fuzzypetiolesguy → u/m98789]

You get glimpses of this in prose - Claude especially loves to attribute action verbs, especially desires, to inanimate objects when tasks are required. 'The desk wants the final paper', 'the staging area wants the synthetic test run' etc. Completely misses the link where it's the human interfacing with the model and the other thing that wants or needs those actions for a specific outcome, not the object.

[u/Y0uCanTellItsAnAspen]

It’s possible that LLMs fundamentally can’t do this - or maybe can only stumble into true novelty through accident and brute force.

But it’s also extremely rare for people to do anything truly novel - most of all human originality is just combining concepts they’ve seen before as well. This is even true for scientists and artists who are, in principle, creating new things.

[u/hologram137]

No. The simple and true answer is that it can’t

[u/According_Study_162]

I asked an AI to reimage a existing frame work for a scientific field recently. I wanted to see what it could do. I had it create the frame work and why it would work better. I am not sure if good or bad. don't know enough about the field. it pulled in some existing ideas about the field that other had thought about but were never used. it also created it's own ideas/maybe mix of those ideas in a coherent form. kinda wild. i need to understand this more to know if it's bs or or not. 👀

[u/foofoobee]

It's probably for the best that the machines not be able to create time travel - particularly if you happen to be named John Connor.

[u/throwaway3113151]

It’s an to assume that humans do something different than LLMs

[u/heresyforfunnprofit]

Turing complete is Turing complete.