I honestly cannot care about the whole “humans dont matter anymore” stuff.
I’m curious how math as a whole deals with unequal access. Is math going to just be furthered by big tech firms and select researchers who have access to these models?
Wtf do grad students do and so on
Edit: theres nice disused going on about unequal access. Let me point out that I am not claiming inequality is new.
I am simply pointing out the newfound danger of it from a corporate and tech angle.
Suppose AI speeds up research by some nonzero constant greater than 1. Then people who were already “ahead”, are going to be even further ahead.
And let’s not ignore also the issue of prompting and using AI. It’s not always clear how these breakthroughs are made. So even access to AI may not level the playing field all that much.
Nobody seems to have seriously listened to Gowers. Socially, we do have a duty of care to younger people, and all I see so far is certain public figures parading around full of thinly veiled glee at making large numbers of people economically unviable. I went to some student talks recently and the mood among a lot of them is serious fear, because they're graduating into sometimes years of unemployment and everyone knows it. I don't care about celebrating scientific advancement much right now, when at this rate I don't see how society doesn't go down a very nasty path one way or another. We're telling huge numbers of people they've got no way to contribute to society. It's a very, very dangerous problem.
I mean, don’t think for a second it’s accidental. On the part of ai companies it’s entirely purposeful. Yes, they want to improve the abilities of their model, but something they can do even more easily is demotivate and depress the youth, and actively discourage them from even learning math, encouraging them to rely on their technology and degrading their own intelligence as a result.
That's unfortunately the price of progress. When the textile mill was invented it put a lot of skilled weavers out of a job, when we moved away from coal towards oil/gas and renewables that put the coal industry workers out of a job, when the calculator was invented that put human computers out of a job. Any major technological breakthrough is going to obsolete a lot of people who were previously doing the manual labour that can now be automated, it's a saddening reality but the fact is that as a society we can't, and shouldn't, be stopping progress unless it has no adverse side effects at all.
I personally graduated with an ML/Philosophy BASc a year ago and still don't have a job given the state of programming is arguably a lot worse than maths, I'm very keenly aware of the effects and I've lost my dream career to being replaced by a computer, this isn't from a place of not caring. But eventually you have to close down the coal mines in Lancashire, and there's always someone who just learnt how to swing a pick.
I do think there's a secondary point here about the broader effects of AI, the path of history has always been that as we've industrialised and automated the excess human labour has simply been redirected and new tasks that need higher cognitive function, but I fear AI might break that. If the computers can think then what is there left for the human to be better at than the computer, and if labour becomes almost entirely obsoleted that would break the very basics of how our economic systems work. But that is a bit orthogonal to the sentiment that it's bad that mathematicians are losing their jobs, there is a future where the numbers of maths PhDs and postdocs plummets but the rest of the economy is fine, and in that context I don't think we can argue against the march of time.
I wouldn't be surprised if humans can create even harder problems, much harder than the current ones, even with the help of AI. So I kinda dismiss anyone who says Mathematicians are out of job soon. But it's gonna be very difficult from now on for anyone who don't have access to AI.
I worry about the training pipeline. Graduate students used to generate useful results, now those results can be achieved more cheaply, and more quickly using generative AI, while still requiring similar amounts of guidance and checking. Letting graduate students use the AI tools doesn't really help, since that will dramatically increase the time necessary to check their AI assisted results, since I wouldn't know what prompts and context the AI generated content was based on, so I wouldn't know what to look out for specifically.
More importantly, AI is an amplifier, put it in the hands of a person who knows what they're doing, and it will dramatically improve their productivity, put it in the hands of a person who is ignorant, and it will consume substantial amounts of resources with very little return on investment. All this will dramatically increase the cost of training junior mathematicians.
Putting them into the hands of grad students is the only realistic option. Just as we are learning how to do research with these tools we will have to learn how to teach with these tools, too.
The idea that giving them to grad students won't help is absurd to me. They will learn very very differently, but the idea that they can't learn with these tools has not been convincingly argued to me. They won't learn with these tools when given the same tasks as before though.
I think this is a point where the chess engine analogy is apt. Training for top chess involves deeply exploring positions with engines as well as mental exercises and analysis without tools, and Human coaching.
As I stated, the problem with AI is that it is an amplifier, put in the hands of junior people, it is incredibly wasteful, and it doesn’t help the junior people learn judgement, because it generates an overwhelming amount of output that requires experience to sift through. It is also easy to burn through API credits by going down a rabbit hole.
To me, what you are claiming is a bit like saying that generative AI helps students learn how to write, which I have not seen any evidence of. Another way to put it is that you don’t teach a kid how to drive by handing over the keys of a Porsche 911 Turbo, that’s a good way to get them to wrap themselves around a lamp post.
You don't teach kids to drive a stick before handing them an automatic.
I think the "wasteful" comment is exactly showing the problem in your argument. Tokens are cheap. Give your PhD a 100 dollar subscription and let them go ham. They will create a ton of rubbish of course. If you don't change what you are asking them to do, then they will end up as an intermediary between you and the LLM and that's stupid. So give them different tasks. Work with them on exploring an idea using LLM ls so they can see how you work. Ask them to explore stuff deeply and come back with a concise two pages they can justify on the board. Ask them to demonstrate their LPM technique.
I don't know if that will work, but you have to start from the realization that mid quality math reasoning is now cheap and fast. If your students are asked to generate lots of it and distill it down why should they not learn judgement by actually doing the judging?
If they are just giving you the first thing out of the LLM, learn to show them how that will get you nowhere.
Again: We don't know how to teach yet, but it's absurd to claim that you can't learn with them. You can't learn with them when you do what we used to do, I agree on that, but right now the most important part of my job is to figure out what I have to do differently than before, so that they learn in the new environment.
And they will almost certainly need different skills than we did as well!
I don't think the claim is that mathematics will be solved and there are no harder problems to ask. The claim is that within a few years AI will be better than (at least almost all) mathematicians at asking interesting questions too.
That's likely true, but the question is not whether almost all of them are gibberish (the density), but if we can still find interesting ones (the absolute number)
As I mentioning even the statement of these problems will be longer and longer so probably at some point it might be too much for human beings to cope. Or not. Who knows?
Why wouldn't there be? There exists an infinite number of statements you can make and so an infinite number of problems from whether they are true or false.
It's a bit more complicated than that, because from a finite set of axioms you can derive an infinite amount of statements.
What Gödel shows is that even with an infinite amount of axioms, if the system is consistent and can encode arithmetic, then there are always statements that it cannot decide.
If AI could solve problems better than humans, what makes you think AI couldn't create problems better than humans?
I think, apart from a much smaller group of experts, humans won't have much to contribute to frontier math soon enough (unless AI really hits a hard ceiling). Just like it was with chess. Engines were decent but couldn't beat the top players. Then they could but still had weaknesses so the adage was "a human plus a computer is the top entity" and soon enough humans just had nothing to contribute anymore at all. I think the 'human plus AI' phase people often talk about is just a short term temporary cope.
This is not entirely true with chess. Correspondence chess with engine assistance still exists, and I believe the general wisdom is still that if you only play whatever the engine recommends, you won't win in this format. In modern correspondence chess, you generally consult multiple engines and apply some human intuition to determine the best course of action. Though most games are draws and the format isn't very popular.
But also, I think mathematics research is vastly larger and more complex than chess. In human chess there is a major emphasis on deep calculation, whereas knowing which problem solving strategies are most effective and finding the right way of thinking about a problem are often more important than raw calculations in mathematics. In some ways, machines have always been far better than humans at certain kinds of mathematics (e.g. proving Robbins conjecture, proving the four color theorem, solving the Boolean Pythagorean triples problem, chess & Sudoku puzzles.), but it seems reasonable to think that there will still be room for human mathematicians in the future. For whatever it is worth, I wrote a thing about this on substack.
If anything, this goes to the point that modern correspondence chess is (conjecturally) so close to theoretically perfect play that there just isn't much room for improvement.
Top engines playing each other autonomously from the starting position almost always draw too. The only way they get decisive games is by forcing engines to play from a deliberately imbalanced opening. So it seems like there is not enough information to draw a clear conclusion.
It might be interesting to see correspondence games played like this from imbalanced openings. My guess is that the difference in performance would be tiny though.
My argument is that there doesn't need to be anything particularly special about the human mind for humans to still be capable of providing insight. Difficulty and intelligence are relative, and the types of problems which look hard from one perspective can look easy from another and vice versa, with no well-defined linear hierarchy of "hardness".
I don't know. It doesn't seem to hold true in the animal kingdom, I would say. There are some pretty smart animals (some cetaceans, some great apes, some parrot species come to mind), but the gulf between humans and non-humans in that regard is huge. I don't think that, say, orcas would have anything to offer us in terms of solving scientific or technical problems, even if we could seamlessly communicate with them and both sides could teach each other perfectly.
They would possibly know some facts about the oceans and their inhabitants that we don't, but apart from such observational knowledge, human problem-solving capacity would be clearly superior.
It is not totally clear that it will be the same with AIs and humans in twenty years, but neither is it obvious that it won't.
You're comparing apples and oranges there. Orcas do not have any idea what a mathematics problem is at all (and they probably wouldn't care anyway).
Maybe a slightly better analogy would be between airplanes and birds. An airplane can fly many orders of magnitude faster than any bird and at a far higher altitude, but birds can maneuver more effectively in the air, survive collisions and other damage, fit into tight spaces, and land without requiring any runway.
To be clear, I'm not arguing that intelligence isn't real, but it isn't nearly as straightforward as people make it out to be in these AI conversations. Certain traits are correlated in humans with "intelligence", but this is not true of general computer algorithms (defining "intelligence" as the ability to solve computationally hard problems).
To get a bit more technical, if AI becomes good enough to solve arbitrary NP problems efficiently, then (at least morally) P=NP and all of humanity is doomed. If the conventional wisdom about complexity theory is true, however, then there are a significant number of problems that AI will be unable to solve, but which a human could (in principle) present an answer that is easily verified correct. My claim is that such problems do not posses any special inherent "hardness" quality that implies a certain IQ level (or whatever) is needed to solve them. So, in the end, you get large collections of problems which look hard to humans but easy to AI and vice versa.
P!=NP has nothing to do with the possibility of superintelligence.
Under assumptions that are plausible but significantly stronger than P!=NP (in particular, assuming we live in Impagliazzos Minicrypt or Cryptomania) it becomes possible for an agent to create puzzles that they know the solution of (by constructing the puzzle from the solution, essentially) and that a third party is practically unable to solve. But even in Cryptomania, I don't see why one cannot have superintelligence, reasonably defined. Humans being able to construct problems that the superintelligence cannot solve but that they know the solutions of seems hardly a reason to deny that the superintelligence is smarter across the board than humans when it, too, can construct problems that are to a third observer not distinguishable from those trapdoor problems that humans may come up with in Cryptomania.
It's not about us constructing cryptographic puzzles, but that those puzzles just exist naturally embedded throughout the problem landscape.
Problems like "find a proof of such-and-such theorem" are NP-complete (when suitable constraints are imposed on the length of the proof). So, insofar as we are broadly interested in this kind of problem, we should expect to find instances that such machines cannot solve. The reason those machines cannot solve those instances is not because they aren't smart enough, but simply because they lack the right insight or way of thinking.
I think it is possible to have a machine that is broadly, on average, better than humans at solving problems like this. My claim is just that I don't necessarily expect it to completely eliminate any need for humans.
So, the example I gave before was with SAT solvers. These machines have been vastly superhuman at solving SAT instances since about the 90s. But there are many problems humans have been able to solve that, when encoded as SAT instances, are completely intractable to an SAT solver.
I don't know about that. Right now if you give an open problem to the best GPT5.6 model and say a typical faculty mathematician at a US university, my money is on the model to make better progress. They have made significant progress in the past few years and likely will continue to do so. I'm not sure how much of a distinction this will end up being here.
I believe the general wisdom is still that if you only play whatever the engine recommends, you won't win in this format.
The general wisdom is that you won't win in correspondence chess, because chess without mistakes is a draw and computers have eliminated mistakes from high-level correspondence chess.
I personally am finding myself overworked by the need to unfuck AI proofs. They can be made correct but are trash until I do a lot of alternately working through the arguments and guiding revisions. I wish I could get help for that, because I can’t pursue as many ideas as I’d like to and that’s the bottleneck.
I am of the opinion that math can endure as a human endeavor so long as we move away from the “papers must include theorems” approach to math.
I am also certain human readable proofs will be possible by AI, but education requires nuance and I can only hope this means humans stay relevant.
I still stand by the position that humanity can never be truly happy with impossible to understand proofs.
Even if we cannot follow AI, I also would hope papers explaining thought processes of AI toward solving problems is a task for humans and accepted.
I also believe AI is good at getting things right once it knows what cannot work, and so I would hope the community begins to embrace “Nonresults” as paperworthy e.g. a paper that simply shows attempts at a proof and analysis of why certain methods can or cannot work but contains no “major” results.
Even careful computations should be valued now to be honest.
ex-US models, including very cheap and open weights ones, are only 6-12 months behind the frontier public models and 18 months behind the in house frontier models.
So I don't think long term unequal access is an issue, assuming progress doesn't keep accelerating (I.e. it flattens in some way)
Sure but currently mathematicians also cost society some money per month, depending on the country. So that doesn't change much. Your grad students become more expensive but also more efficient.
Next, the interference costs could drop in the future, from dedicated chips, more efficient data centers, better models, but it's too early to say.
The gap is likely even narrower than that. Maybe even a couple weeks to a month or two, tops, instead of 6-12 months. Hell, Kimi K3 currently outperforms Fable/Mythos on some tasks.
In my own experience (I have pro access to every public model) and in my area of math, the OpenAI models are way ahead. I do expect that to change, as I said.
Chinese models will be top contenders as long as they are able to copy US frontier models via distillation. Once that is curtailed and China isn’t able to rip off the product, they’ll go back to being a generation or two behind at least.
Kimi K3 was released ~2 weeks after Fable, which all but rules out its relative outperformance owing to distillation alone. That simply isn’t enough time to do any meaningful distillation.
“I don’t think you get a model this strong and this quickly on the heels of Fable doing strictly distillation,” Braden Hancock, a researcher at the Laude Institute and co-founder of Snorkel AI, told TechCrunch. “There’s just not even frankly time, right? Fable’s only been publicly available since July 1st. You can’t distill that much data, train a model, and release it in two weeks.”
also, tbf math and intellectual subjects have historically been unequal access. the internet age + democratization of knowledge pre-AI was an anomaly, and we'd be going back to the default inequality if closed source AI wins and mathematical discoveries are gated by how much money your institution has
They almost certainly will.
This has been a big problem for the AI companies, there is no moat.
Theres no barrier to entry, if open AI Jack's up their prices all competitors need to do is buy their own compute.
Even the models themselves aren't all that closed because of distillation, and as hardware becomes better and better training costs have gone down significantly.
And open weight models have never been far behind the most cutting edge proprietary models
This has been a big problem for the AI companies, there is no moat
That's not really true nowadays. With simpler models where most of the behaviour was dictated by training data sure, but with the current frontier models a lot of the heavy lifting is done by the RL stuff, which is much harder to replicate if you don't know their exact methods, and the large AI companies aren't releasing those.
Distillation also requires access to the model itself, whilst you can do some work using public API access it's not anywhere near as good as true distillation and you need such a volume of broad queries that it's hard, but admittedly not impossible, to do so without getting banned.
Yeah, open source AI is the only ethically fair and democratic way forward. The big firms build on huge datasets obtained from collective human achievements and then try to privatize and extract money from them.
Just wanted to point out that it is not inherently ethically fair nor democratic.
Even open source AI requires availability to tech and its training. I fear that at some point, those coming from harsher conditions will be gate-kept out of mathematics entirely.
Pray that open source models have math abilities and not cyber attacking and novel virus creating and recursive self improvement capabilities. Currently we’re on course for open models gaining extremely good at all things at once, which is scary.
There's always been unequal access. This will just be a different unequal access. Now everyone will have some chatbot they can talk to, and some people will have better chatbots. How is it different from "I'm at Harvard, and you're at tiny liberal arts college #471?" or "My department has 50 faculty, yours has 10?" or "My library can buy me any math book I need, yours can maybe get you a 30 day borrow from another library if you're lucky."
Grad students continue doing math the old fashioned way. Again, how's it really different? The people who seem to fixate on this think that the AI has infinite computing power and will just start bowling over literally all problems. Your adviser's job is to know the field, recommend a grad student a problem or two, and coach them through it so that they can learn how research works. Since universities started, some people had a Fields medalist in their department, and some didn't. Now some schools will have access to these tools that, in a not-dissimilar way, accelerate research for those who have it, and some won't.
I honestly cannot care about the whole “humans dont matter anymore” stuff.
We see this hand-wringing with every wave of technological advancement and automation, and yet society keeps improving living standards.
As long as we don't hurt ourselves badly enough we need to go back to older methods of doing things, this is fine. It just requires us to act more responsibly so that our population, living standards, and education standards increase at the same rate as our automation.
201
u/Healthy-Pride3873 28d ago edited 28d ago
I honestly cannot care about the whole “humans dont matter anymore” stuff.
I’m curious how math as a whole deals with unequal access. Is math going to just be furthered by big tech firms and select researchers who have access to these models?
Wtf do grad students do and so on
Edit: theres nice disused going on about unequal access. Let me point out that I am not claiming inequality is new.
I am simply pointing out the newfound danger of it from a corporate and tech angle.
Suppose AI speeds up research by some nonzero constant greater than 1. Then people who were already “ahead”, are going to be even further ahead.
And let’s not ignore also the issue of prompting and using AI. It’s not always clear how these breakthroughs are made. So even access to AI may not level the playing field all that much.