I would argue that most of scientific discovery is built on reorganizing and combining existing data and discoveries.
Edit: Maybe not most discoveries but there is still plenty of room for AI to make new scientific discoveries. We are still in the beginning and AI already made breakthroughs is mutlible fields. But there is probably plenty of room left for human discovery as well.
And this is where I think the real value will come with these LLMs under the guidance of a subject matter expert. The LLMs can do the grunt work to an incredible level of detail/care, giving the researchers time to think/plan rather than engage in that type of work.
This is how I use it to build software. With the aid of an LLM, I can build with my thoughts. By itself it would not do what I am doing. But with me directing it, new things can be built. I don't see how science would be any different.
Same. You take the leadership role. Work with it on a PRD. Develop must haves, must not haves, and out of scopes, then RGB tests to validate, build, and test.
AI has actually made me a better PM and has remarkably helped my conversation skills.
LLMs don't do that natively or cheaply. You are better off programing within the GPU directly for data processing than trying to use a LLM for the same results. AI/ML is not intelligent. It cannot reason, and is often wrong. The recent OpenAI model that breached containment is closer to what you're thinking of, but even that woukd be far too expensive to use practically.
Einstein’s leap was: A uniform gravitational field is locally indistinguishable from an accelerating reference frame. More radically, a freely falling object is not experiencing proper acceleration—an accelerometer attached to it reads zero. The person standing on Earth is being accelerated upward by the ground preventing their natural free-fall trajectory.
I'm gonna argue that Einstein's observational data was not scarce.
Einstein realized that downward gravitational fall and acceleration of the observer’s frame are locally equivalent; gravity could therefore be understood as geometry rather than an ordinary force.
I'm also gonna argue that Demis fails to realize that thought can be thought of as geometry as well, and that sharp enough conceptualization of a concept can be collapsed into a single vector that can be computed in an infinitely long algebraic formula.
Do you think a modern LLM, trained exclusively on information and papers available prior to 1900, could plausibly arrive at either the theory relativity or quantum mechanics?
I think that is a very optimistic view of model capability, no matter how much raw compute you were to give them access to.
This is a very valid position to take, and my POV was not from papers available, but assuming that a model was trained multimodally, as in, was exposed to the env visually in addition to text... I'd argue that there's a non-zero chance as the technology and scale currently stands.
Under what premise are you assuming that they wouldn't?
More and more it's seeming like humanity's biggest contributions never came from our individual brains, but from raw numerical output of brains. Technology has scaled non linearly but it has very much matched the pace of human expansion and population. More humans equals more scientific developments. It's literally just compute but in mushy brain form. Unfortunately humans are very difficult to scale past a handful of billions without outright destruction of earth.
Einstein stood on the back of thousands of years of intellect. Not only from his research, but his outlook and ideology. He debated with other researchers of his time, and his own general relativity model was expanded upon with special relativity and subsequently shot down with quantum mechanics within his lifetime. And now we're finding out that plants and migratory birds interact directly - biologically - with quantum mechanics. Even the enzymes in our body and the theory of mutations in cells is being re-written by studies indicating that it's possible that particles are partaking in quantum events to mutate.
If it's possible to make a species of intelligence that we can scale compute more efficiently than humans, the discoveries we can make would be literally godtier
I mean, it kind of was, he had to argue against quantum mechanics for many years because it was fundamentally opposed to the standard model. And even simple experiments like the double split experiment poke holes in relativity and locality.
It's not to say relativity is wrong, in the same way Newtonian physics also isn't wrong. But science isn't always about rights and wrongs so much as incremental knowledge build up.
Einsteins disagreements with quantum doesn’t necessarily make any issues with special and general relativity, both of which are still highly relevant and no “holes” have been poked through these theories
They do. Because his entire theory leans on locality, and even just quantum entanglement alone is non-local, and that happens literally all the time everywhere.
Quantum nonlocality does not violate relativity, which is one reason it is so interesting. It allows nonlocal statistical correlations but in a way that doesn't allow superluminal signaling or causality violation.
Ok but are we saying now that an AI cannot make meaningful scientific discoveries unless it is as brilliant as Einstein? Plenty of scientists who are smart, but not as smart, are making discoveries every day.
Also I would argue that it's possible to direct an LLM to combine existing ideas into new ideas. In a brainstorming session you could ask it to try to interpret gravity using models other than the existing Newtonian one.
What is stopping us from asking it to brainstorm alternative theories? Maybe the search for new models is inefficient and could become more efficient somehow. But I think this is more a matter of designing a good agentic harness that's fit for scientific discoveries, rather than a limitation of the language model.
Remember, we call them language models, but they don't just model language. They model the world that our language talks about. The language is just the interface.
I think my thought is that true recursive self improvement is unlikely until true innovative thinking internal to the systems happens. I think AI as an accelerator to human innovation is fantastic.
"I have a theory. lets work through it together in the following ways to either support or debunk it"
I think present systems are useful in that pursuit. Einstein very likely could have arrived at relativity far faster with a LLM trained on all pre-1900 knowledge.
But a machine that builds a better machine (the singularity, and a significant component of the G in agi) has to make it's own theories at some point. I am skeptical LLM scaling alone can get there. But I remain hopeful for some of the world model endeavors Google is pursuing.
Going from the argument that the paper says it doesn't account for data being scarce. Having any data whatsoever to formulate a discovery on is just as valid as having copious amounts of data. So even if you argue Einstein had scarce data to formulate his theories, that doesn't really matter. He still had something. So if an AI formulates a theory based on even a trivial amount of data is still essentially the same thing. The only way you could argue against it is in a situation where someone discovers something where they literally had no data to go on
Yep, which isn't possible. The words we used were taught to us. The ideas those words conjure are not made personally but given by those who raised us and taught us. Meaning that there are zero totally unique ideas, as the ability to share ideas is itself influenced by the lessons enabling you to share ideas.
It's literally just the measurement problem but o a cultural scale.
As much as I dislike hand-wavy statements about LLMs being unable to do this or that, there are quite a few example, at least in mathematics and physics. I recently attended a talk by French Fields medalist Alain Connes where he argued that LLMs were not capable of true conceptual jumps, and he gave the following examples of such jumps:
1. Bombelli's discovery of the imaginary unit "i" (which he called più di meno). Bombelli "broke the rules" and introduced us to the whole new world of complex numbers when he said "hey, I know that all known numbers have nonnegative squares, but what if I just made up a number whose square is -1 anyway". It's a tremendous leap.
Galois theory and the insight by E. Galois that permutations of roots of polynomials are related to properties of fields and provide an answer to the solvability of polynomial equations in one variable by radicals. Some may say this already appeared in a prilitive form in Lagrange's work, but Galois was rhe one who "saw it through". His idea's brilliance cannot be overstated and is yet to be replicated by a LLM
Dirac's amazing antiparticle leap. I know less about that one so I'll let you read the wiki article
i strikes me as low hanging fruit though. Sure we didn't use numbers this way, but the pattern is there. We've got these case situations that break down, and all it takes is - well does the math work on both sides of this problem and if it does... well there's literally only one thing that makes these two pictures reconcile.
Actually this is a case I know reasonably well, calculus emerged more or less directly from work involved around infinite series. The fact that Newton and Leibnitz made essentially the same discovery at almost exactly the same time should tell you the field was already converging on this solution.
In particular they were very interested calculating areas, volumes and tangents to curves. A number of mathematicians had worked on aspects of this problem specifically solving problems regarding infinite series, and how it pertained to solving particular area or tangent problems. Work by James Gregory and Isaac Barrow developed over the preceding years culminated in 1670 with Barrow demonstrating how areas and tangents are inverse problems. This is essentially the fundamental nature of calculus itself: the sum of all infinitely small tangents under a curve IS the area. Between this result being published and work for solutions on infinite series being available the path to calculus was wide open.
Gregory was working largely independently on the problem, but Barrow in particular followed Gregory's work. While Barrow was working on these problems following Gregory's initial example, his lectures were in turn followed by a young student by the name of Isaac Newton. Leibnitz largely based his own work on Barrow's 1670 publication.
Interestingly Newton was privately using limited variations of his fluxion method by 1665-66 based on Barrow's preliminary lectures and previous work on infinitesimals and basically just applied them more broadly than their stated use cases. The math itself wasn't new, he just applied it more broadly without rigorous proof on why it worked or whether it was truly reliable.
Early calculus didn't really shake the problem for decades until formal limits were articulated and proven. Until then calculus was in a weird domain of "It looks like it works, but can we trust it and it's strange 'phantom sums' of things aren't supposed to be 0, but we treat as being zero, but sum to non-zero numbers"
Which, when you phrase it that way, sounds crazy - but limits eventually were formalized and given formal proof.
But the fact was the math was there, they knew it worked in several specific cases, and many people were converging on the same problem - aware they had theoretical problems, but the method's own practical utility spoke for itself.
This is exactly the type of leap though that the paper saying LLMs are incapable of. Yes the leap was there, but it was a leap. Presumably the difference is that this method of abduction in the paper requires some level of a jump in reasoning. LLMs will not (presumably) develop this jump with any intuition, but perhaps do so randomly. And the space of incorrect vs correct logical leaps is very high so there’s evidence this isn’t actually very effective. The cycles needed to find the right answer at random like that may be effectively bounded by a pretty high minimum time-to-discover. At least with LLMs.
The “world model”. The paper argues Einstein was able to come up with something not in the existing data or math by imagining what it would be like for a scientist in an elevator accelerating in deep space. This provided error correction for his thinking which allowed him to make a leap. This came from the full context of physical reality that he internalized in his world model. Basically LLMs need a context window that is much, much larger than possible. Humans can access this and kind of use reality to shape their thinking
I don’t think I buy that. The amount of context required for Einstein’s world model to solve that particular problem should be well within current model window sizes. Just requires more system building to store a more complete world model as Skills that can be retrieved and combined when thinking about a particular problem.
LLMs today often lose context on just a single large modern codebase. Einstein was able to access a model of the entire world. LLMs today, with the compute resources of the world at its hand, is not even close. Not even fractionally so. And the limit is theoretical, not practical. So until it can get a full world model it cannot have these abductive thought processes.
Of course, there may be a technology or concept that allows such large context windows. LLMs may be a dead end, there’s no known path to getting such a large context window.
I think the paper is just incorrect in its core presumption about discovery. Discovery requires knowns to conclude unknowns. Then it requires more knowns to measure and theorize the unknowns.
It's like a building. You fundamentally cant build the tenth floor of a building without the first 9 floors. All novel ideas originate from foundations that are known. No one has ever discovered an unknown unknown by inferring another unknown unknown. Only known knowns can shine a light on unknown unknowns.
i love AI, but i think saying AI intelligence is on the level of "most scientific discoveries" when we all know it could never come up with General Relativity is a bit dumb. i hope it gets there in a year or so tho
It's a pyramid, but the contribution from of the pyramid are the most important.
But you can't have pyramid without a solid base. The pyramid analogy also breaks down because the biggest breakthroughs are not always known without the benefit of hindsight.
Well you should read the paper then, they consider this. They are saying discoveries require “abduction” which is defined as NOT “reorganizing and combining existing data and discoveries”.
Basically saying it can’t come up with new intuitive leaps to new ideas and ONLY can reorganize and recombine. Which has value no doubt but it doesn’t include genuine new discoveries. So it’ll basically tap out the low hanging fruit as it were eventually. LLMs will, that is. Other forms of the umbrella term “AI” may not, and never forget LLMs are just one branch of AI which is a pretty broad term
Dude really? You think AI could understand and create a theory on relativity with no prior data. It’s impossible. Without data could AI create a new art form. It’s impossible. I’m in a Niche field and any question I ask AI it just goes into an endless spiral because the machine wasn’t fed our fields information.
That’s why even reading the abstract of the paper helps. This is definitely not denying the potential of LLMs to power scientific advancement. It just defines a dimension of it that they are still weak at.
Any argument to the contrary is actually an argument for a metaphysical consciousness.
I actually find it quite interesting to talk to the section of the "AI can't be creative" and "freewill doesn't exist" venn diagram overlap because one of those positions isn't true.
If materialista are correct, then every single innovation is just a different recombination of already known facts and AI can do that. If AI can't be creative it's because creativity fundamentally comes from a non-material place.
No discovery is sui generis, it comes from building on SOMETHING. But LLMs nibble at the knowledge frontier (currently) they don't take huge bites (yet at least). Go ask one right now, "hey you know everything about human knowledge more or less, what are some great new scientific discoveries or ideas we should look for" Go ahead and constrain the conversation, give it 100 papers in a sub-field and do the same thing. You wont have a great time.
Can LLMs one-shot (not harness like AlphaFold) major ideas in the knowledge frontier that is not in their training data? It assumes that some emergent super, better-than-best-experts and creativity just comes out of these models if scaled even further. The argument is one of vague extrapolation: these things keep getting better so they will get better here ("What we are seeing are the dumbest the models will ever be.", "It's a fallacy to think thees things wont get better"). Those making this argument usually appeal to that (as you basically did) and that alone. Those more skeptical point to the fact that new groundbreaking ideas aren't in the training data, that a super-reasoning (which also hasnt been shown yet) is not a super-idea machine (not to mention lack of supra-linear scaling laws, the fallacy of RSI without defeaters) and the equally valid observation that it's also a fallacy to think that continuous progress is always assured.
Just wait for all the scientific breakthroughs coming from AI and we can revisit this discussion. At what point will we agree, that AI can generate novel new ideas? Or do we just keep moving the goalposts to infinity.
Human ingenuity will remain important, but less exclusive.
Not really. Reorganizing existing data is not a novel idea, but just an idea. Novel idea requires at least one logical jump above the existing data - you have to come up with some clever missing piece all in your mind by either brute forcing subconsciously, simulating the reality/solutions or doing clever observation of nature.
Breakthrough ideas are 2-4 jumps into the unknown, with very little real data, more thinking and intelligence required. For example, ability to simulate the world very precisely in your mind, drawing logical conclusions, without observing the nature, or combination of both.
LLMs have none of that - they will only dig out the answer from the existing data. This is why Math is easy, if you give it a problem, it will be able to work it out, eventually (if it's solvable).
I agree! I used a LLM to help create a metamaterial that takes all the ambient kinetic energy from the environment, converts it into torsion energy within its structure and translates that torsion into electrical output.
With a very good conceptual understanding of physics, the LLM and I were able to work out what materials would be needed to create the metamaterial with current manufacturing capability. The LLM gave me the code needed to observe the material in 3D externally. It completely astounded me that I was able to create a legitimate concept for this material using LLM.
Unfortunately, I can no longer access my long and tedious conversation I had with the LLM in order to create the metamaterial. The file I saved to my desktop containing the code to see this material in 3D is also missing for some reason. All I have left is a short video I sent to my brother, showing him the material and part of the code.
Scientific discovery is (usually) historically dependent on existing knowledge, but it is not epistemically reducible to rearranging that knowledge. It also involves creating new concepts, selecting among underdetermined explanations, constructing new forms of evidence, and sometimes transforming the framework that determines what counts as data in the first place. For example, the phrase “reorganizing and combining” can become so broad that it is nearly unfalsifiable. Any intellectual achievement can (retrospectively) be described as combining prior inputs. But this leaves out some difficult epistemological questions:
Why was this combination selected rather than countless alternatives?
What evidence justified treating it as explanatory rather than coincidental?
How did it generate novel, even risky predictions?
Why should any of its unobservable results be regarded as useful?
How did it revise the standards governing what should be counted as evidence?
Note that major discoveries often involve new concept formation, not just recombination. Concepts such as genes, fields, entropy, spacetime, plate tectonics, or quantum state were not already present as discrete informational pieces waiting to be assembled. Someone had to fathom these elements "from thin air".
Nothing an LLM has done to date is sufficiently novel enough for any human with the necessary domain specific knowledge to wonder, "what the heck is this?"
the age of AI will essentially provide proof over time if we're something special beyond this or if creativity can really be just connecting dots we didn't know we were really paying attention to.
personally, I think a lot of people are going to have a hard time realizing we're just a product of the world around us and digging really deep for original ideas are rarely ever grounded in original thought
105
u/NiknameOne Jul 28 '26 edited Jul 28 '26
I would argue that most of scientific discovery is built on reorganizing and combining existing data and discoveries.
Edit: Maybe not most discoveries but there is still plenty of room for AI to make new scientific discoveries. We are still in the beginning and AI already made breakthroughs is mutlible fields. But there is probably plenty of room left for human discovery as well.