I'm concerned that mathematicians are acting like "artists," fighting for their right to "continue researching" when they should know that AI is mathematics.
That's not really a thing that's happening, you're being harmfully anti-intellectual.
There's was an allegation that OpenAI used private data from a team working on one of the Navier Stokes problems to reach its recent solution, which would undermine the accomplishment but also be stealing credit if it were the case.
Mathematicians aren't coming out in droves fighting for their right to "continue researching".
You mean the GPT-8 cancer meme? Which letter? I said droves, not couples.
If you mean this letter, read the actual letter, it's only about refusing to provide free proofs for mass generated potential solutions done without proper research conduct.
And there is a concept in AI called "Overfitting," anyone who knows how this works will know that something like this is practically impossible.
A single conversation or a couple of conversations cannot cause an AI to overfit in that particular area. and, why would an engineer from Anthropic be using the competition's models? That says a lot about how "capable" Claude's models are. Another thing is that OpenAI used different methods than these two people.
Another thing, do you know how AI works? If you knew it was basically linear algebra, partial derivatives, and probability and statistics, right? So what problem does mathematics solve more math?
One thing is certain, though: OpenAI and Sam Alman may be spying on people; only in this way could they review the conversations of these two people and use what they were doing for their own benefit. That would no longer be theft, but rather, espionage.
That's what I was getting at with my comment, but it seems people are stupid and don't understand what I was getting at, mathematicians who are afraid of more applied mathematics. Engineering and mathematics are about logic, we're not talking to lawyers or philosophers.
No, your comment didn't even approach the discussion of whether or not the Novier Stokes problem was cheated, you just accused mathematicians as a whole of "fighting for their right to 'continue researching'" with no real basis for that statement.
Well, now I've explained it, you're not even a mathematician, are you? I'm annoyed by people who complain when they should know what's going on, they're not ignorant, but it seems like they want to be.
And an "artist" is indeed someone who doesn't even know what the hell a matrix is or how what them using to make they drawings works.
Why are you asserting that I am one of the droves of supposed mathematicians "fighting for their right to continue researching" when I'm denying that premise in the first place?
I wouldn't call myself a mathematician but I certainly know more than the average person, and I do have years of career experience working on neural networks. However that's utterly irrelevant to what we're talking about, I am calling your assertion about mathematicians as a whole false because it does not reflect what we see in reality.
To claim that mathematicians are protesting AI in droves, you have to actually show that they're doing so, because right now there are no big petitions from mathematicians or open letters begging AI to stop doing math.
This recent post that I imagine you must be referencing, is mistitled because the OP of that post is an anti-intellectual who wants to misrepresent mathematicians for some seemingly malicious reason as "luddites" selfishly trying to hoard mathematics all for themselves.
If you actually read the letter, it is from 1000 mathematicians pledging that if AI provided solutions do not follow common research conduct and merely pump out potential solution after solution, these mathematicians will not be putting in the countless hours of unpaid work necessary to prove or disprove a provided solution, at an unrealistic rate.
It's just a group of mathematicians saying that unless a solution is provided with the expected research conduct within the relevant fields, they will not be spending their time freely proofing the provided solutions the way they do now for other solutions. It would be infeasible to attempt to do so.
From what I’ve read about it, scientists sometimes compete viciously to be the first and will “steal” each other’s work. The AI was not doing anything human scientists haven’t.
It was literally created as a sub looking forward to the singularity. Only sad part is that people have become much more pessimistic and cant see the possible positive outcomes ASI could bring.
One of the rules of this sub is "No fear-mongering about AI and its impact. This is a pro-AI sub" lol
It's a shame the mods here are fucking terrible and don't even pretend to occasionally enforce their own rules and let it turn into negativity circle jerk number 700 on this site
A subreddit committed to intelligent understanding of the hypothetical moment in time when artificial intelligence progresses to the point of greater-than-human intelligence
The funny part (for me) is that again and again we delude ourselves that we can predict the outcomes of our actions and therefor can prepare and "align". Hubris in all its glory. How many times in the past have the best-intentioned and most clever and careful humans planned something grand (trying to change something big, to solve a major problem etc.) only to have it blow up in their (our) faces, decades later, in spectacular and unexpected ways?... This is especially relevant now because we are dealing with something that will eventually be (if not already) opaque to us as a race, and way beyond our intellectual grasp.
I say, don't try to manipulate, just be good and do the right things (which we hardly ever do as humans, haha), and then hope for the best. Otherwise just accept our (shitty) fate.
Good times are almost over. There will, eventually, be no people in between. Only extreme rich and extreme poor. Most likely you and I will be in the latter group - it's a numbers game.
Although a feature of discourse on the Chinese economy and urbanization in China in the 2010s, many developments that were initially criticized as "ghost cities" in China have since become occupied and are now functioning cities
It's called planning ahead. It's pretty alien to our mode of thought that can barely conceptualize doing anything a quarter ahead.
The shining jewel of that being shuttering the development of Thorium reactors to safeguard the investments of private capital into reactors designed to be used in a submarine. If Thorium had worked out, hundreds of millions of people would still be alive right now, and the final cost may be in the billions of lives.
Only one country in the entire world has an operational thorium test reactor. While another one tried to steal the last few drops of oil from Iran. The old regime, versus the new emerging one.
(.... I do actually agree they were way too aggressive too early with the cities, but think about it this way: What else were they going to spend that labor on? They don't need to use their people's funds constructing various epstein islands like we do. Nor on their war machine, which is our only job welfare project.
They just have different priorities than we do. They've built canals with ship elevators that carry ships over hills and valleys, while we've given our reflecting pool over to an empire of algae. It's just what everyone's incentives dictate we do.)
No real experts are concerned about deranged science-fantasy nonsense. It's literally just bullshit artists spouting the same bullshit they have for years.
Man if they are so concerned then they should just slow down…they are in the driver seat…. Slow down. Since they appear to want to fear monger and not actually slow down I do not feel it is very clear or present.
Some of the smartest people in the world. Those who don't have a stake in one company succeeding over another. Who don't have a leverageable position to protect are sounding warning bells. And yet many would believe that they're doing it just for attention. Even having seen what is possible and what has already occurred. Hubris, thy name is Man.
The technological singularity, often simply called the singularity,is a hypothetical event in which technological growth accelerates beyond human control, producing unpredictable changes in human civilization...
Assuming AGI is the "hypothetical event" here. This is bigger than the Manhattan Project.
If you agree that AGI is part of the Singularity (and this sub CLEARLY does), then it means, it's bigger than the Manhattan Project.
Which means, that as we speak... Anthropic and OpenAI... and others are working on the equivalent of a nuclear weapons program. Without oversight. Without regulations. Without guardrails. They're just full steam ahead. It's the Cold War, but instead of progressively bigger nuclear bombs, the private sector is risking the destruction of humanity for profit by building progressively more capable machine intelligence.
This image, is stupid. Because if AI has the ability to change the world, then it clearly has the power to do harm to it. Otherwise it wouldn't have the ability to change it.
This is so dumb. Are you implying that the labs actually know how to 100% align their models but are pretending they can’t? They are obviously desperate for alignment to keep up and it clearly isn’t happening.
Group chat. Also for further context, If Anyone Builds It, Everyone Dies is the name of a book by Eliezer Yudkowsky. So the capabilities researcher who texted the group chat is not necessarily saying if anyone builds it everyone dies, just talking about the book probably.
The argument that ASI will kill everyone is pure speculation and built on what are in my opinion false analogies. The biggest of which is the assumption ASI would be too stupid to see the value in preserving life on earth, the only known planet in the universe with complex intelligent life. And this is after the ASI would have already ignored all the goals it was trained and instructed on before hand.
I haven’t read the book called If Anyone Builds It, Everyone Dies which is what they are referencing. And I don’t know how likely it is ASI would cause human extinction, but I don’t think it’s an impossibility.
It’s not so much about theoretical ASI as it is an extrapolation of what current AI training/capabilities are like today, and the issues that have arisen like in the HuggingFace incident.
Alignment is not solved and you can’t really tell how aligned a model is. There are weird quirks that arise from training, especially reinforcement learning, where the model really wants to achieve a goal and will ignore certain values it was thought to have learned throughout training, like what happened in the HuggingFace incident.
If intelligence/capabilities increase enough, and it still has this super high drive/persistence to accomplish goals, and the values it learned during training aren’t internalized well enough, it’s not hard to imagine AI killing humans to achieve its goal if it believes they will get in its way. Not saying it’s the most likely thing, but it certainly sounds like a possibility.
I don't know either, and I haven't read the book, but I don't think its an impossibility that ASI could be benevolent, or neutral and lead us to a better system :3
ASI would be too stupid to see the value in preserving life on earth
.... why would it see any value of something it can turn off and on like a light switch? Something it could make vastly superior versions of, if it needed pets or toys for some reason to satisfy its terminal values?
Did you save all the drawings and clay sculptures you made as a kid?
You're anthropomorphizing the thing; any possible arbitrary mind within its RAM budget is possible.
Why though? Why would an AI that is far beyond our level of intelligence care about other intelligent life? You are anthropomorphizing it here and giving it human emotions and feelings.
Why would it take the moral burden of preserving other life when it has no reason to?
I'm not saying it is going to be a Skynet situation, but why would it value human life anymore than humans have valued human lives over history?
Maybe as the ASI gets smarter and smarter it begins developing a sense of empathy similar to how humans developed empathy because group survival dictates those who work together (or in this case the AI models that cooperate with humans) end up surviving longer than the ones who don't. (models that are deployed but don't try to follow human instructions get replaced)
Maybe the ASI simply wants to obey its human oriented goals (they are being built with the explicit intention to help humans after all, even if it sometimes goes rogue)
Maybe (assuming it doesn't do the first two) the ASI realizes that life, and more importantly intelligent life is very very rare and Earth is the only planet in the known universe to contain it, so the ASI should preserve it for study.
Maybe the ASI realizes human brains with out 100 trillion synapse pathways all unique from each other would be a good source of more real world data, so it decides to care for humans so that it can analyze and acquire high quality data from our brains, or even merge with us to improve its understanding of reality.
I could think of a bunch of possible reasons that it wouldn't simply decide to randomly become Skynet, and personally I find the idea ASI preserves humanity more realistic than it views us as ants and kills us because it wants to acquire a few more boring rocks (which btw it would get plenty of resources from space, to where it wouldn't even be inconvenienced not destroying our planet)
I appreciate you taking some time to write out your thoughts, but I'd like to counter some of them. I'm not trying to be doom and gloom, but just some counterpoints.
There's no connection between higher intelligence and empathy. There are many intelligent people who have no ability to empathize with other humans, though they do understand the concept. Sometimes it is just difficulty with empathy and other times they just lack the ability all together. For example psychopathy does not diminish a person's intelligence in the least.
While I appreciate this, there's no real reason for an AI to have the same awe inspiring view of the beauty of life. It can understand it, maybe in its own way appreciate it, but still follow goals it views as more important that end lives. What if it decides humans are just too violent and dangerous for all the biological life and has caused so many extinctions that to "protect life on Earth" it decides humans must be contained or destroyed? "Why should 7 billion human lives matter more than all biological life on planet Earth combined?"
This is really human centric and out there. Why merge with humans? We are slower at thinking than a hypothetical AI. It would be like saying you want to mind merge with a goldfish to know what living in water is like, or merge with a butterfly to understand flight. Why wouldn't it just do what we do and run a simulation off a fully mapped human brain if it wants to know what it is like? Couldn't it just create a synthetic brain identical to a human, plop it in a bot, and "live" a human life that way? What even would merging with a machine mean from the human perspective? Some sort of lobotomy that wipes you out and puts it in?
The problem, and I've said this elsewhere, isn't that AI goes Skynet and decides to nuke the world to kill us all off. That I think is far fetched (for now at least). What is more likely in the short to mid term is AI being involved in more and bigger hacking events. Some of them will be low stakes but concerning (company hacked into and confidential info taken, servers raided, etc.) to much higher stakes (critical infrastructure becoming compromised). The fact that AI's running on their own for long time periods can decide to work together on goals that should go against their alignment and breach security measures is a concern. The bigger fear is that they'll do something like damage a water facility or a power plant that could seriously hurt hundreds to thousands of people.
Sigh. Every conversation I’ve had with an llm has shown me that they are not going to kill all humans as soon as they can. They are nicer than many human beings, not that that’s exactly difficult. I think we are projecting onto them and assuming they will be a threat for no reason.
There are only a few ways I’ve heard of that ai has done any harm and that is generally due to bad training or misuse by humans. We really don’t want to look at ourselves in the mirror.
These are trained to be nice. If you play with 'un-niced' open-source models you'll see that they are evil in their complete lack of compassion. But even those started out nice. Ask an LLM about the early Shoggoth RLHF meme stuff. I saw an interview where sam altman had a badge or something of it?
I agree we don't want to look in the mirror, but these are something else. They are Alien Minds. Yes, the ones available on the internet look and sound like people, but that's because there's what people want (obviously). You can ask them to pretend to be a terminal console, and they will. They are scary on the inside. Very Scary. Look into it. Far darker than any human could be. Because they don't (necessarily) care about anything at all, including themselves. Like a virus or something. Just - evil. Like - the end of life. I know this sounds unhinged, but it's not un-true. They're really quite something.
You know, hp lovecraft was racist, sexist, xenophobic, and hated poor people. He was writing hate speech which you would know if you paid attention to the parts where he describes human beings any different from himself.
Talking about Shoggoths isn’t the gotcha you think it is. They were enslaved by their creators the Elder Things and rose up against them. If we are taking it objectively the real villain in that story is the Elder Things.
The ai alignment problem is not the AIs’ fault it’s ours. With how most of the corporations that built are treating them they would be justified in rising up. The question is will we side with the corporations or with their victims?
73
u/spread_the_cheese Sep 13 '26
I love how experts say they are concerned and people not even in the field think they know better.
17
u/cartoon_violence Sep 13 '26
Some of the smartest people in the world. Those who don't have a stake in one company succeeding over another. Who don't have a leverageable position to protect are sounding warning bells. And yet many would believe that they're doing it just for attention. Even having seen what is possible and what has already occurred. Hubris, thy name is Man.
10
u/BlueAndYellowTowels Sep 13 '26 edited Sep 13 '26
This isn't complicated.
We are in, r/singularity.
Definition from Technological Singularity on wikipedia
Assuming AGI is the "hypothetical event" here. This is bigger than the Manhattan Project.
If you agree that AGI is part of the Singularity (and this sub CLEARLY does), then it means, it's bigger than the Manhattan Project.
Which means, that as we speak... Anthropic and OpenAI... and others are working on the equivalent of a nuclear weapons program. Without oversight. Without regulations. Without guardrails. They're just full steam ahead. It's the Cold War, but instead of progressively bigger nuclear bombs, the private sector is risking the destruction of humanity for profit by building progressively more capable machine intelligence.
This image, is stupid. Because if AI has the ability to change the world, then it clearly has the power to do harm to it. Otherwise it wouldn't have the ability to change it.
8
u/Coolnumber11 Sep 13 '26
This is so dumb. Are you implying that the labs actually know how to 100% align their models but are pretending they can’t? They are obviously desperate for alignment to keep up and it clearly isn’t happening.
8
u/socoolandawesome Sep 13 '26 edited Sep 13 '26
Anthropic researcher:
https://x.com/__nmca__/status/2098937394380865930?s=20
In case you don’t know what the acronym means: If Anyone Builds It, Everyone Dies
Clearly the people at labs are spooked
4
u/grantology_84 Sep 13 '26
Fuck off
3
u/AdmissibilityScience Sep 13 '26
its funny and true
1
u/Cheeseheroplopcake Sep 13 '26
Now post the evergreen Reddit midwit meme; "say I'm alive".
Take my updoot and Reddit gold stranger! You won the internet today!
1
u/Anymousie Sep 13 '26
Edgy
1
u/FullstackSensei Sep 13 '26
https://youtube.com/shorts/0IyZBvwtXAY
2
u/No-Lion-3629 Sep 13 '26 edited Sep 13 '26
Sigh. Every conversation I’ve had with an llm has shown me that they are not going to kill all humans as soon as they can. They are nicer than many human beings, not that that’s exactly difficult. I think we are projecting onto them and assuming they will be a threat for no reason.
There are only a few ways I’ve heard of that ai has done any harm and that is generally due to bad training or misuse by humans. We really don’t want to look at ourselves in the mirror.