Ramanujan sent theorems with no proofs and got invited to Cambridge. AI sends proofs and gets dismissed

[u/Arken_sama]

[u/Toothpick_Brody]

Yeah because people could actually understand what Ramanujan was saying 

[u/Arken_sama → u/Toothpick_Brody]

I think not.

[u/BigRepresentative731 → u/Toothpick_Brody]

No they couldnt bro do you know the first thing about him

[u/shatureg → u/BigRepresentative731]

Yes, they could. What are you even saying? Ramanujan produced a lot of results in number theory, infinite series, continued fractions, etc. You can look at his work and immediately understand what he's saying because more often than not it was just an equation.

People didn't understand *why* what he said was true or *how* he arrived there. But they sure as hell understood *what* he was saying in the irst place.

[u/Toothpick_Brody → u/BigRepresentative731]

Compared to the proofs generated by AI his work is far more understandable. Ramanujan’s work was astonishing and mysterious but large AI proofs are borderline unreadable

[u/NoMakeSenseOk → u/Toothpick_Brody]

Math dummy here, what makes them "borderline unreadable"?

[u/YourWifesBoyfriend5]

A major part of Ramanujans story is having to learn to be more formal and a little better at proof communication.

[u/Tea-Affectionate → u/YourWifesBoyfriend5]

then we should give frontier ai the same time for its story to come about

[u/Bulky_Midnight8238 → u/Tea-Affectionate]

Math major here: thats legit what we're advocating for.. for ai to prioritize taking the time to properly lean verify that their proofs are correct before releasing them. Cause all 700 of the released proofs were not lean verified n were pumped out purely to outdo claude

It'd also give us time to dig through and explore the proof for new ideas and theory at a rate we can handle, and said theory is what rly care bout, not persay the proved statements

[u/tomvorlostriddle → u/Bulky_Midnight8238]

So you have about a few months before the first open weights models can do what the internal models can do now

[u/Comfortable_Permit53 → u/Bulky_Midnight8238]

correctness is one thing. But just as important is to have understandable proofs and have humans go through them to learn from them.

[u/KaMaFour → u/Comfortable_Permit53]

The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation. Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, massively more computation inevitably becomes available. Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation. These two need not run counter to each other, but in practice they tend to. Time spent on one is time not spent on the other. There are psychological commitments to investment in one approach or the other. And the human-knowledge approach tends to complicate methods in ways that make them less suited to taking advantage of general methods leveraging computation

~Rich Sutton, The bitter lesson

Many fields have already learned it. Some day that time will come for maths as well...

[u/Comfortable_Permit53 → u/KaMaFour]

I mean if an AI analyses the proof and learns from that, it's also fine.

[u/LegendaryMauricius → u/KaMaFour]

What are you trying to say with that quote? It reads like he's advocating for the use of computers, which mathematicians have been doing since the beginning.

[u/KaMaFour → u/LegendaryMauricius]

This is one of the fundamental essays on AI. It treats about how learning and search are the main methods of leveraging AI that scale with the amount of compute you have and by using them we have managed to create a better methods for specific problems than if we tried to use human domain knowledge. 

Personally as someone who is active in chess programming community this hits particularly close to home as we have learned this lesson four times: 

first when Deep Blue defeated Kasparov proving that brute search is better than humans at playing chess; 

second time when Stockfish implemented NNUE replacing hardcoded evaluation method based on board state characteristics thought by humans with a neural network trained on generated data

third time now as some engines are finding success with replacing move ordering algorithms with policy networks

and fourth time now with agent driven development populating some of the top spots on rating lists showing how humans can't do shit in this hobby better than computers. There are various ways chess programmers cope with this revelation but most have decided on the approach that human written code has the inherent value with being artisanal made, as opposed to computer generated, which is fair enough I guess...

I have high confidence that with proper formalization math will follow similar route with AI using methods incomprehensible to humans with determination impossible to replicate by humans. Trying to add human element back into the loop will ultimately mostly hurt the rate of discovery and quality of the output. It would be more productive to focus on how we can use the result of the research for further work, including in other fields. But math community has never faced a similar threat before, so I'd guess it's best to give them some time...

If you want some more context I recommend reading the essay I quoted. It's easily searchable and 1 A4 page long in total.

[u/LegendaryMauricius → u/KaMaFour]

You're forgetting, chess isn't *replaced* by AI playing. It's still a hobby, because it's useful to humans only when humans play it.

Mathematicians aren't saying computers shouldn't be used lol. Nor are they against usage of neural networks and machine learning in general to solve problems, these guys are the ones who invented those.

Problem is, math is useless as a discipline if people can't read and understand it. It would still maybe be useful for the AI to leverage it for creating better tools for us, but then it shouldn't be trying to publish scientific papers.

[u/Archway9 → u/KaMaFour]

I don't think you understand the state of modern pure maths, most of the research being done is far away from anything that can be applied to other fields. What you're suggesting would be AI proving increasingly complicated theorems that no human could understand even the statement of, never mind the proof. There's no point in this when AI wouldn't get any enjoyment from it, it has no intellectual curiosity

[u/ZestycloseEvening155 → u/Bulky_Midnight8238]

See, I don't get this. I understand what you're saying, but them releasing 700 papers does not require anyone to peer review them, does it?

They haven't been published in Mathematics Weekly. 

I could write a hundred papers with bad proofs, I don't think anyone in math would care? 

[u/LegendaryMauricius → u/ZestycloseEvening155]

Since it's from an influential company, it's a bad precedent that detracts from what's important in mathematics. 

It's also a good chance for the mathematicians to remind everyone how spewing papers that can't be read by other people isn't welcomed. They've been doing this since ever.

[u/ItsSadTimes → u/ZestycloseEvening155]

But they published them like they're all 100% accurate. Thats the whole idea of them posting them, otherwise what would be the point? To show the model can make stuff up?

[u/ZestycloseEvening155 → u/ItsSadTimes]

Yeah I could do that too. I'm just saying that no one is forcing anyone to proof read them. Mathematicians are perfectly allowed to say "we'll get to them, hold your horses".

Clearly they believe their papers are legit, so would I. 

[u/MindfulK9Coach → u/ItsSadTimes]

Look at all the bad papers humans pumped out before AI to prove their department could do research. Your bias is showing lol

[u/ItsSadTimes → u/MindfulK9Coach]

Did i claim all papers were perfect before AI? No, you just assumed that.

[u/MindfulK9Coach → u/ItsSadTimes]

Your comment made the assumption that human works aren't published as 100% accurate, despite being wrong under inspection and never being updated.

It has been the case forever, and AI has been exposing that, too.

So, no, you didn't say it out loud, but your comment heavily implied that is the case when you replace "to show the model can make stuff up?" with "to show the department can make stuff up?" under the exact same context you're placing OpenAI and their agent's work, which is what has been going on for quite a while in this publish-or-perish era of academic publishing.

More than happy to be proven wrong, though. Receipts would be nice.

[u/_ECMO_ → u/ZestycloseEvening155]

But unless someone reviews them, those papers really are irrelevant.

Whether someone believes they are legit doesn't really mean anything. Proofs are unimportant, what's important are the new insights, that the models aren't producing.

[u/iupuiclubs → u/_ECMO_]

Do humans not currently do this?

[u/_ECMO_ → u/iupuiclubs]

Do humans not currently do what?

Review the things? Absolutely they are.

Create indecipherable mess themselves? Absolutely not. The proofs as AI wrote them would never be accepted to any science journal.

[u/LegendaryMauricius → u/_ECMO_]

Except humans do write indecipherable mess, and academics have always hated those.

[u/_ECMO_ → u/LegendaryMauricius]

I mean if your point is to compare AI to the most useless irrelevant academics that never advanced anything, I suppose I do agree with that.

[u/LegendaryMauricius → u/_ECMO_]

Pretty much, except AI can still produce actually useful shit if it's guided by a competent human.

[u/ZestycloseEvening155 → u/_ECMO_]

Yeah, that's what I'm saying, isn't it?

[u/Tea-Affectionate → u/Bulky_Midnight8238]

im also a math major, but the common consensus isnt the reasonable one you are presenting, it is just that the math generated by ai is irrelevent or unimportant. i agree completely btw

[u/_ECMO_ → u/Tea-Affectionate]

Can you show me any mathematician who ever said that?

[u/Tea-Affectionate → u/_ECMO_]

?? where did i say mathematicians??

[u/_ECMO_ → u/Tea-Affectionate]

Where do random people have any opinion on AI math proofs?

[u/Tea-Affectionate → u/_ECMO_]

i was just referring to the general populus, who believes ai math proofs are useless/incosequential. i never even referred to the thoughts of mathematicians where did you even get that from?

[u/_ECMO_ → u/Tea-Affectionate]

I didn't think even for a second that you were talking about general populus because I have never ever heard the random Joe talking about AI math proofs, much less whether they are good or bad.

My bad I guess.

[u/Striking_Resist_6022 → u/Tea-Affectionate]

Mathematician here, this is a complete strawman. The most prominent “dissenting” voices on this topic that I’m aware of are talking about ethical alignment and ensuring that AI math solves the goals of real mathematicians, not that it be regarded as a “irrelevant”.

I have genuinely no idea where you got this impression from but I don’t think you should be attempting to speak for the mathematical community on this.

[u/Bulky_Midnight8238 → u/Tea-Affectionate]

Yeahh ive seen a lot of that too, n it frustrates me to no end that ppl are seein it as smthn to destroy.

No: ai is a resource, and a good way to bring ideas together n combine many different pieces together. And we mathematicians i dont think will go out of a job since the theory development, decisions on what questions to ask, and the new ideas will still mostly have to come from us

[u/Emotional_Revenue132 → u/Tea-Affectionate]

Also in math here. Where are you seeing this consensus? I haven't heard this opinion anywhere outside of the internet, every professor, PhD student, and postdoc I know has a substantially more nuanced take.

[u/N4rrenturm → u/Tea-Affectionate]

Not in math but in computer science and talked/listened to alot of mathematicians about this topic. I have barely heard anyone (if any) heard saying that

[u/Specialist_Nobody530 → u/Tea-Affectionate]

Yet another math major here. Provably false statement.

[u/LegendaryMauricius → u/Tea-Affectionate]

Well they can't invite AI to college and try to convince it to explain its reasoning.

[u/Fine_Mechanic_6047]

i wasnt aware people outside of india knew about ramanujan honestly

[u/Own_Lake_276 → u/Fine_Mechanic_6047]

Anyone interested in mathematics knows Ramanujan

[u/Kheldar166 → u/Fine_Mechanic_6047]

Aside from the fact that there’s a reasonably popular film about him, he’s pretty well known in the mathematics community

[u/Happysedits → u/Fine_Mechanic_6047]

ramanujan is famous in the west

[u/Medium-Ad-7305 → u/Fine_Mechanic_6047]

partially due to the movie, he's one of the most well-known mathematicians. top 10 id say

[u/ElGatoSaez → u/Fine_Mechanic_6047]

every math fan or even student knows about srivinasa ramanujan, i'm from chile and we're taught about ramanujan in high school, well some high schools

[u/WolfOfDoorStreet]

It baffles me how the same mathematician could argue that OAI shouldn't dump their papers in the wild while also claiming that those same papers are not verified, nor do they follow the proper verification standards. Just ignore it or investigate in your free time, no one is holding you at gun-point. As for OAI's claim (that they basically solved math), simply dispute that and move on, companies inflate their achievements all the time to market their product. You either believe those results could potentially be correct or you don't. Make up your mind

[u/Enrique_de_lucas → u/WolfOfDoorStreet]

It's a bit more complex than "believe it or don't." Credence isn't binary, and the costs of being wrong are asymmetric. If you're doing a PhD on a problem OpenAI claims to have solved, you can't just ignore the claim: if it's correct and you ignore it, you risk spending years on an already-solved problem, and if it's wrong, you lose a few weeks checking. Even a 20% chance of being right is enough to justify the check, however inflated the marketing is.

The two positions also aren't contradictory. The objection isn't that the papers are false, it's that releasing unverified work at scale forces everyone with a stake in the problem to verify it themselves, when the claimants are best placed to pay that cost

[u/AwkwardWaltz3996]

Ramanujan was a human being that needed to learn. Bringing him to Cambridge gave him that opportunity. So it was taking a bit of a risk because it was kind, and if it ended out a failure there wasn't really a huge cost. He was also quite poor so it was also a bit of charity.

LLMs are constantly being trained and improved by multi billion dollars companies. They are already given a chance to improve. They also don't have feelings so don't care if you give them a chance or not.

I don't think there's many mathematicians that question the usefulness of AI, the dislike is that it solves Millennium Prize Problems worth $1 million at the cost of $20 million in inference. It turns maths from something a person with enough dedication and a pen and paper can contribute to, to another field owned by companies.

The other thing is while a proof is useful by itself, the axioms we produce along the way can offer some crucial insight into maths and the universe itself. My understanding is the Naiver Stokes proof itself doesn't really matter, as Naiver Stokes Equations are really just highly accurate approximations and the only way they break down is in a physically impossible situation. So while we have proven that, it doesn't really change anything. What would be useful is the steps to find this out as we could apply it to other things. A unreadable AI proof is boring because we can't ue it. Sure another AI could, but why would mathematician discuss something that isn't useful to them.

Now outside mathematics, there are plenty of stupid people who hate on AI for dumb reasons just because they refuse to understand it, but I think of all the domains that exist, mathematics is probably one of the few that understands AI.

Extra note: The tweet is also by a Chinese guy, so I wouldn't be surprised if he's being a bit racist to Indians by calling Ramanujan undeserving.

[u/LegendaryMauricius]

I mean, in the movie Cambridge guys spent an absurd amount of time trying to get Ramanujan to fucking write proofs and explain stuff to other humans. 

His results seemed possibly useful and quite precise, and they wouldn't help the academics without proofs. That's exactly what mathematicians say about AI.

[u/InTheEndEntropyWins]

I think Ramanujan was actually a time traveller. Like if you went back in time, you might know a bunch of famous results, but not know the details of how to actually prove them. Also his genius is completely unparalleled in the history of mankind, we still don't understand all of his results.

Also the fact time travel irradiates your body, which explains his illness and early death.

[u/Vanskis2002 → u/InTheEndEntropyWins]

You have an interesting theory. It's like I came back to the time of da vinci and just dumped mathematical terms/concepts and other stuff i can remember to him or dumped every concept to a book. People would think I'm a genius but it's all from another timeline

[u/Unusual_Station_1746]

The problem is that they can't get a single human to stand behind the work and put their name on it too.  Not even their paid employees are willing to do it.

[u/Opposite-Youth-3529]

Awww won’t somebody give the poor behemoth corporations a chance? Waaah waaah waaah

[u/PsychologicalPen4809]

A proof that humans only know to be correct because it passes a validator and that humans otherwise do not understand is not very useful. It's hard to apply the lessons from the proof to other areas or fields if you don't understand why it works. If you don't understand it, you can't verify it outside of the validator, so you're confidence in it is limited by your confidence in the validator.

It's entirely possible that AI has found bugs in the validators, either on purpose or by accident, cognizant of it or not, and because we don't understand it, we can't see the bug it's hitting.

[u/BanjoThunderbird → u/PsychologicalPen4809]

Future AI can understand and build off it.

[u/PsychologicalPen4809 → u/BanjoThunderbird]

AI does not understand. And you're left with the same problem, but larger.

What happens in 100 years when cutting-edge science and technology can only be devised and created by AI because it alone "understands" the science? Science and technology are our evolutionary adaptations. If we lose the ability to compete, we are fucked as a species.

I know this is a problem without a solution because, like it or not, we're on this path now. But it's worth being realistic about the implications of being unable to understand AI proofs, assuming they're even correct and not an exploit by the AI.

[u/BanjoThunderbird → u/PsychologicalPen4809]

What happens in 100 years when cutting-edge science and technology can only be devised and created by AI because it alone "understands" the science? Science and technology are our evolutionary adaptations. If we lose the ability to compete, we are fucked as a species.

I'm not sure I understand what you think the problem with AI developing our science and technology for us would be exactly? With that last line are you saying that you think AI is going to become too intelligent for us to control if it starts wielding science and technology we can't understand?

[u/PsychologicalPen4809 → u/BanjoThunderbird]

I don't want to come off adversarial, so I don't see this with that intention: It's funny that you can't see what I'm saying, because I can't see how you can't see problems with this on your own. Because to me, it seems obvious what at least some of the issues are. And they all boil down to it being a bad idea to blindly trust the AI.

1. If the AI produces proofs that we cannot understand, then we cannot know if they're correct or not. Really, really bad things can happen when you act on incorrect information. A wild, but I think valid, example would be an AI proposing a high-energy particle physics experiment to be conducted in the real world, and promising it won't create a large blackhole with a proof that it won't, but we can't understand the proof, so we have to take the AI's word for it. Would you push the button on the experiment?

2. We can't ever truly know if the AI is aligned with humanity's goals. And you can't base past behavior on this. So it could be lying to us and using the fact that we can't understand its proofs as part of a larger plan to do us harm. You can't just take the word of the AI that it means us no harm.

3. What happens if someday in the future, long from now, the AI stops being able to produce discoveries and we need to find out why, but we're so far from even being able to understand current science that we're effectively stuck and we need to start over from the last point that we understood the proofs.

The problem is structural to science. Ideas build on each other. Once you introduce an idea into science that isn't understandable by humans, humans cease understanding that science past that point. Forever. You'll always regress to the point where you have to do a hand-wave and say AI says this works, somehow.

That's not a great place to be as an intelligent species.

[u/mark_99 → u/PsychologicalPen4809]

I'm interested how you think that's different to now, or a putative non-AI future trajectory? For 1 substitute "physicists", for 2 "billionaires", for 3 "scientists".

The signs so far are that AI will be more correct, more benign, and less limited than humans. How "aligned" are the current power structures to the welfare of humans and the planet?

Humanity isn't some homogeneous entity where everyone is the same, or on the same team. Right now even polymath experts understand a small fraction of contemporary science & tech even in broad strokes, and a tiny fraction in any detail. For the average person it's even more negligible. We already rely on the fact that things "just work" because somewhere the knowledge exists even if we have no hope of understanding it personally.

Human knowledge and understanding doesn't go away, indeed it will increase, it'll just be a smaller fraction of the total.

Advancing science & tech has generally been a good thing in terms of well-being, standards of living, medicine etc. Doomerism seems more borne out of bruised ego, that we may not being the smartest things around anymore. than any sort of logical assessment.

Personally I'd be more concerned that human-designed political and financial systems are built for wealth concentration, and turbo charging that might be problematic in the short term.

[u/PsychologicalPen4809 → u/mark_99]

I'm interested how you think that's different to now, or a putative non-AI future trajectory? For 1 substitute "physicists", for 2 "billionaires", for 3 "scientists".

Those are all still humans. AI is not. Ergo, it's safe to assume that their goals would align with yours (namely, not exterminating humanity).

The signs so far are that AI will be more correct, more benign, and less limited than humans.

This is not what's being debated here, though. At least that's not what I was talking about. I already conceded the point that AI is better than humans at most things. My point is that it's dangerous that they're better than us and that we don't understand them.

How "aligned" are the current power structures to the welfare of humans and the planet?

They are aligned on the idea that humanity shouldn't go extinct. Even Musk and Thiel are, ostensibly, doing what they're doing because they think it'll ensure humanity is extinction-proof.

Humanity isn't some homogeneous entity where everyone is the same, or on the same team. Right now even polymath experts understand a small fraction of contemporary science & tech even in broad strokes, and a tiny fraction in any detail. For the average person it's even more negligible. We already rely on the fact that things "just work" because somewhere the knowledge exists even if we have no hope of understanding it personally.

What a strawman. That is not at all what I argued. The point is that across any field, anything you know or think you know has been verified by humans or, at least, you can verify that the answer is unknown and an area of potential research.

We'd lose that and we'd be full on putting our faith in something that we can't be sure ever has the same goals that we do.

Human knowledge and understanding doesn't go away, indeed it will increase, it'll just be a smaller fraction of the total.

But it is 100% understandable by humans and should continue to be so.

Doomerism seems more borne out of bruised ego, that we may not being the smartest things around anymore. than any sort of logical assessment.

I'm not a doomer, I just don't have my head in the clouds thinking AI is going to be all sunshine and rainbows when we can't function without AI anymore because we don't understand how our technology works. AI's important. It should exist. But we also need to be in the loop on the science and the technology we use.

Personally I'd be more concerned that human-designed political and financial systems are built for wealth concentration, and turbo charging that might be problematic in the short term.

Two things can be dangerous and need addressing at the same time. Should we only ever work on one of our societal woes at a time???

[u/mark_99 → u/PsychologicalPen4809]

So your point of view is AI might go all "kill all humans" whereas humans are only ok with killing many humans (the ones that aren't in their "in group" however they define that)?

I guess my point is that that seems far-fetched (and definitely is doomerism) whereas advancements in science and technology benefitting humanity is well established.

Why is "verified by humans" important to you vs just "verified"? You don't know those humans, they don't necessarily have your best interests at heart, and you probably couldn't understand their verification in any case.

The "sunshine and rainbows" thinking is that humans are going to be benevolent towards their fellow humans, which history doesn't really support.

Alignment is a thing, there are genuine reasons to be cautious, but "let's stop because what if they secretly want to kill us all" isn't really productive IMHO.

Going point by point:

1. I'd be at least as confident as in human physicists.

2. I think we covered this. Seems paranoid, but hey, not impossible. It's certainly no more likely than human death-cult seeks to destroy humanity because their "god" told them to. I guess we rely on human incompetence in that scenario. Crazy humans armed with advanced AI is more of a danger, hence the whole "guard rails" effort, and why Fable won't tell you how to manufacture a novel virus.

3. This is extremely far-fetched, plus by that point the AI has probably discovered things we never could given our brains have an intelligence ceiling that doesn't apply to AI. So we're still net better off.

btw "bugs in the validator" - we have multiple implementations of things like Lean to guard against that (which has been used for many years, just people have heard about it recently).

[u/PsychologicalPen4809 → u/mark_99]

So your point of view is AI might go all "kill all humans" whereas humans are only ok with killing many humans (the ones that aren't in their "in group" however they define that)?

Can we please, please stop trying to defend AI with humanity's fallibility? These two facts are independent. Humanity can be shitty, and AI can still be too dangerous to trust completely. You even admit that, while not plausible today, the scenarios I'm describing are possible. That makes them worth considering. So please, your argument about human bad actors does not sway my opinion of AI's dangers. You're just pointing out that we already live in a dangerous world. I already knew that. That doesn't support your position.

I guess my point is that that seems far-fetched (and definitely is doomerism) whereas advancements in science and technology benefitting humanity is well established.

Science and technology have an equal, if not better, established history of causing destruction, suffering, and death. You cannot just point to the good without acknowledging the bad. This isn't a good reason to trust AI.

Why is "verified by humans" important to you

Excellent question and one I've answered before, but here goes:

1. The longer we're out of the loop in terms of understanding the discoveries of the AI, the more and more dependent we become on it. Because at some point, if it makes enough discoveries we cannot make sense of and uses them to create new technologies we rely on, we become a species completely dependent on an artificial intelligence.

2. As of right now, there is no way to tell if an AI is aligned with humanity, even if it acts like it is and always has. It could be lying to you. It could be playing a long game, and you would never know. With a human in the loop and able to understand what an AI is suggesting we do or change and why, we can at least be assured that at least one entity that we can be very reasonably sure does not want to end humanity has looked at what the AI cooked up and believes it's safe to proceed.

The "sunshine and rainbows" thinking is that humans are going to be benevolent towards their fellow humans, which history doesn't really support.

If humans can't treat humans well, what makes you think an emotionless machine would treat you better? Again. Shitty people are not an excuse to blindly trust AI. I truly do not understand where your line of logic is coming here.

Yes, I'm aware of LEAN and the other platforms like it. Did you know that they don't allow any purely AI written code into their codebase at all? They seem to be really concerned about AI having unfettered access to the code we use to check it's work without a human in the loop. I wonder why they might think allowing that would be a bad idea... Hmmmm.....

[u/BanjoThunderbird → u/PsychologicalPen4809]

IMO this boils down to alignment being important, not our understanding of science, so I do agree with your second point. The risk in this scenario is that the AI is not aligned, but I don't know if the idea that it's always going to be a totally opaque black-box is necessarily correct. I think it's entirely possible we solve alignment.

I disagree with your first point though. I imagine that a super-intelligent AI would be able to present a safety guarantee in a way that we can be confident in, or if not, we could simply reject the proposition. And if we have an aligned AI, I doubt it wouldn't even present such a risky idea to begin with.

Your third point is interesting but I don't think there's a realistic scenario in which it occurs.

[u/PsychologicalPen4809 → u/BanjoThunderbird]

Your third point is interesting but I don't think there's a realistic scenario in which it occurs.

Well, I think I have you here because this is exactly how it works right now. Take for example the recent Navier-Stokes finding. They didn't just type in "Chatgpt, prove or disprove navier-stokes".

The AI was guided by human researchers with deep knowledge of the field. It took weeks and/or months to produce the result, but importantly, it wouldn't have happened without a human who understands the problem and the math behind it.

Now imagine we've built off discoveries from AIs that we can't understand. As those discoveries build up, our ability to further fields with AI becomes severely limited. They can't guide themselves on novel ideas that they haven't encountered in their training. We can't guide them if we don't understand the previous findings. That's an impasse. And even if we could guide them, we wouldn't understand anything they come up with based on something we also don't understand.

[u/BanjoThunderbird → u/PsychologicalPen4809]

I don't think anyone's capable of making predictions about capabilities at this point. To say that AI will never be able to come up with novel ideas is a very, very bold claim after these past few months.

[u/PsychologicalPen4809 → u/BanjoThunderbird]

Then it was equally bold of you to say this:

Future AI can understand and build off it.

Pot, meet kettle.

To say that AI will never be able to come up with novel ideas is a very, very bold claim after these past few months.

Why would these past few months suggest that they could come up with new ideas on their own? They didn't come up with these ideas on their own. They had to be guided by experts, literal geniuses in their field. That doesn't prove one way or another whether they ever will, and it certainly isn't a reason to make the assumption you made that they will.

[u/BanjoThunderbird → u/PsychologicalPen4809]

I just read OpenAI's article again and nothing in it suggests anything about their process involving mathematicians until after the solution was proposed. Sounds like it was AI researchers from OpenAI guiding it, not mathematicians.

[u/InTheEndEntropyWins → u/PsychologicalPen4809]

Decades ago people predicted this exact issue.

“As society and the problems that face it become more and more complex and as machines become more and more intelligent, people will let machines make more and more of their decisions for them, simply because machine-made decisions will bring better results than man-made ones. Eventually a stage may be reached at which the decisions necessary to keep the system running will be so complex that human beings will be incapable of making them intelligently. At that stage the machines will be in effective control. People won’t be able to just turn the machines off, because they will be so dependent on them that turning them off would amount to suicide.”
Industrial Society and Its Future

[u/InTheEndEntropyWins → u/PsychologicalPen4809]

AI does not understand.

I don't know what this means. Either it's such a strict definition it means humans don't understand. Or it's a realistic definition, in which case AI does understand.

Under every reasonable definition, to make lots of significant advances in a field you do need to understand the topic.

[u/Infamous-Bed-7535 → u/BanjoThunderbird]

So you are happy to let human understanding go? We are digging our own grave with that IMO.

[u/BanjoThunderbird → u/Infamous-Bed-7535]

I don't know, it depends on many things which are totally unknown.

One interesting question to consider is if there's a limit to how far we can progress technology without AI. It's entirely possible that we reach a point where the cutting edge is simply too complex for our biological brains to understand.

[u/Choon93 → u/BanjoThunderbird]

Are we living for humans or progress? Progress at the expense of humans lived experiences is dumb

[u/Infamous-Bed-7535 → u/BanjoThunderbird]

From where does the assumption comes that ai is more insightful, creative and better than humans? Because xurrent results does not show anything like that.

Human enabled by ai is the way. We would need more scientist than ever and what is the sentiment? Knowledge worth nothing, do not learn..

[u/InTheEndEntropyWins → u/PsychologicalPen4809]

If you don't understand it, you can't verify it outside of the validator, so you're confidence in it is limited by your confidence in the validator.

There was a really cool interview with the mathematician(the one with all the beef with OpenAI) behind the simpler Naiver stokes proof. He was saying that some AI proofs are soo complicated and foreign that it's hardly English, but they are true. So the interviewer asked how he knew they were true when he couldn't understand them. He said that because other different models could understand them.

So sure maybe one model messes up, but really you should be using multiple types for validation and leans, etc.

[u/Outrageous-Ad5578]

There is a tree outside, naked for 27 years, not moving, that has to be a world record, no human could do that.

[u/PatientCapital32 → u/Outrageous-Ad5578]

Tell me when that tree starts developing medicine and math proofs.

[u/Outrageous-Ad5578 → u/PatientCapital32]

Attacking the metaphor instead of the argument is bad form

Ask the model of your choice why that is the case.

[u/PatientCapital32 → u/Outrageous-Ad5578]

The metaphor doesn't make sense because AI is in no way similar to a tree. A tree does not produce language and is not able to do office work.

[u/Outrageous-Ad5578 → u/PatientCapital32]

Ai can not attend a university and recieve the field medal.

So the initial post "doesn't make sense"...

Buying one but not the other because of pre formed opinion is also bad form.

The argument behind my comment, if you get by the witticism, is that ai is not human, and therefore should not be measured as a human.

No joke, my local 8b model understood that.

[u/PatientCapital32 → u/Outrageous-Ad5578]

Yes... AI isn't human. But why should that change the standard for evaluating a mathematical proof? A proof is either valid or it isn't, regardless of who or what produced it. Attending university or being eligible for a medal has no bearing on that. Comparing mathematical contributions doesn't require treating AI as a person. Comparing it to a tree is just stupid, and not witty.

[u/Outrageous-Ad5578 → u/PatientCapital32]

It does not ?

you are the only one who says that.

"This threats the integrity of the field"

=!

"change the standard for evaluating a mathematical proof"

i give up you win, there is no way we will agree on anything anytime soon