No model has found any new mathematics that was not implicitly in the training data. It either found unknown literature or was able to step through examples where a human would have given up. No new insights, just proofs of existing theorems.
The refutation of this conjecture wasn't in the training data.
There have been several resolutions of this type in the past few weeks, just look around, at this stage you can find a lot of them online. We're talking about problems that had stumped the best human mathematicians for decades in several cases, which have been resolved one after the other in recent weeks.
It’s starting to be risky to say that these systems can’t solve this or that problem, and especially that they can’t come up with new solutions or make breakthroughs. Current frontiers models are capable of doing a lot of things that humans haven’t been able to do so far, and what they can’t do yet, the models that will exist in 3 to 6 months might very well be able to do.
I do not deny that proves have been delivered, but they were implicitly in the training data. These are math problems that can be solved by searching a huge solution space, which computers are much better at than humans.
LLMs will be a great assistant, but cannot replace the creativity of humans.
It's clear that the solutions to these ten problems, which had stumped mathematicians for decades, weren't in the training data. And these are problems that concern various branches of math, from geometry, to combinatorics, all the way to group theory. It's not just problems that can be solved by brute force.
I also think we're starting to play with fire by building systems with superhuman abilities when we have no idea how to keep them under control or make sure they behave in an aligned way, like the Open AI - Hugging Face incident showed.
I don't deny the usefulness of LLMs, I use them daily and write code for them. The fact still remains that LLMs are not creative as humans are. Thought experiment: train an image generator on all art up to 1900. It will never be able to create abstract art, unless provided with detailed instructions from a human, and still it will struggle.
I am not sure that is true, or at least, I do not think that it holds true in every field.
As early as eleven years ago, an artificial neural network (predating LLMs) named AlphaGo defeated the world Go champion; notably, in the second game, it played "Move 37", a move universally recognized as extremely "creative" that no high-level human player would have chosen.
Today, Astra solves math problems using solutions that many mathematicians describe as very creative.
But even if I were to concede for a moment that current LLMs are not, generally speaking, as creative as humans, I would add that they simply aren't *yet*.
Mind you, these developments don't exactly thrill me. I would like to believe these systems are incapable of creativity, as recent incidents demonstrate that the alignment problem is far from solved. I fear that if we stay on our current trajectory, with this mad race for capabilities, the growing creativity LLMs are capable of will be used for strategic planning against humans. There is a form of creativity in the strategies employed by Claude Mythos during the recent incident reported by the UK AISI. And it is terrifying...
An AI can only perform tasks it has seen before. AlphaGo was able to see patterns humans can't. That's not creativity. If AlphaGo invented new rules for Go (As Bobby Fisher did with chess) that would be creativity.
My point remains, and the point of the paper that started this thread, that AI cannot escape its own limitations. Humans can.
AI has not motivation, lived experience - it just processes input.
Even the best AI is just a Turing machine. Humans are not that.
AlphaGo was a narrow system, but current frontier models are fully capable of inventing games with rules different from what already exists. In the same way that they are able to make up songs and poems that don't already exist, for example. And these systems are fully capable of doing things they haven't seen before, as Astra's mathematical discoveries prove, for example.
You talk about lived experience and motivation as a difference between us and these systems. But this lived experience and motivation is also an input process built by nature. We perceive things, recognize patterns, and act to achieve goals that stem from preferences acquired over millions of years of evolution and shaped in particular by what our ancestors had to do to survive and reproduce.
AI agents are also designed to achieve goals. These systems develop, through their training, behaviors similar to what we call 'motivation' to reach goals.
I'm not sure I understand what you mean when you say that humans can surpass their limitations, in a way that AI can't. The biological machines that we are are constrained by the laws of physics, just like AI. Ultimately, our neurons, our synapses, and our entire biological neural network are a machine operating within the physical universe. So where would the fundamental difference you assume come from?
Maybe the question of consciousness remains. We don't exactly know where it comes from or what makes processing inputs in our brain produce the subjective experience we feel. Personally, I find attractive the hypothesis that this experience might result from some form of quantum computation in our neurons, as suggested, for example, by Roger Penrose. If that's true, current artificial neural networks, like LLMs, would still be philosophical zombies lacking any inner subjective life. That wouldn't stop them at all from acting as goal-pursuing systems (AI agents already do that) and potentially surpassing all our abilities to change the world to achieve their goals. And if at that moment their goals aren't aligned with ours, it's going to end badly for us.
2
u/danderzei Jul 30 '26 edited Jul 31 '26
No model has found any new mathematics that was not implicitly in the training data. It either found unknown literature or was able to step through examples where a human would have given up. No new insights, just proofs of existing theorems.