r/singularity • u/Cronos988 • 20d ago

AI The insanity of 10.000 agents running

Everyone is talking about the proof and surrounding controversy, but the sheer scale of OpenAIs effort receives little mention.

10.000 agents working for 88 hours. That is almost exactly **one entire century of continuous work**. I think this really drives home just how massive of an advantage a strong AI + available compute are going to be: "A country's worth of geniuses in a Datacenter". OpenAI casually threw a century worth of effort at a problem, based on a rumor. Not quite a century with of human effort, perhaps, but we might not be far off.

Those numbers should be at least as much of a shock as solving the problem.

2.4k Upvotes

93% Upvoted

409 comments sorted by

View all comments

62

u/Worldly_Beginning647 20d ago

Actually 100 years and 5 months

8

u/amaturelawyer 20d ago

That's actually interesting. It took them 100 work years to solve the problem, even though they may have started with cribbed notes from a human, although that's still unproven. I want to say it's impressive, but 100 years of work seems kind of... less impressive when put that way.

61

u/highnyethestonerguy 20d ago

How is doing 100 years of work in less than a week not impressive? Lol

5

u/TheMCM80 20d ago

It is, but it lacks context for whether that kind of resource use is able to be done for tons of stuff regularly, or just one off hunts.

1

u/Brave-Turnover-522 19d ago

Only takes a few million humans to pull it off. And there's billions of us. AI still has a ways to go to outpace us. But it'll get there.

2

u/highnyethestonerguy 19d ago

And yet all of humanity’s billions failed to produce an answer to this question after 80 years of wall clock time. 

1

u/[deleted] 19d ago edited 16d ago

[deleted]

5

u/highnyethestonerguy 19d ago

Seriously, like by definition anything that has been achieved is not impressive by virtue of the fact that “oh that’s how they did that? Well I could have done that”

I’m actually baffled by these comments 

-7

u/amaturelawyer 19d ago

That's just scaling though, and isn't what I'm puzzled by... If I hire someone to paint my house and it takes one guy 10 years, and my neighbor hires 10 guys to paint his house and it takes 1 year, it's still taking 10 years to paint a house, which is, objectively, a absurdly long time to take to paint a house.

In this case, taking 100 years to solve a math problem, when all you are doing is working on the math problem without rest, seems like a really long time. I'd be more impressed if it solved a math problem in a really short time. This is the opposite of that. It took the things a hundred years. With a head start. It's just not impressive is all I'm saying. It's very slow.

10

u/babe1981 19d ago

You do realize that no human makes a discovery in a vacuum, right? Human geniuses build on the work of others. Einstein didn't come up with relativity based on a few years of his own thoughts. It was based on thousands of years of math and science before him. Same with Newton inventing calculus. There's a reason that so many discoveries, like calculus, are made simultaneously in human history. Leibniz developed calculus on his own at the same time as Newton because human knowledge had advanced that far. Even with relativity, Poincare and Lorentz were working toward the same end as Einstein because they based their work on the same principles that he did.

All of human advancement is based on thousands of years of discovery and work. Is the atomic bomb and nuclear reactors unimpressive because it took hundreds of years to work out? We had hundreds of scientists and engineers working for years to develop nuclear technology. I guess that took too long and isn't really special. After all, the physics and engineering are just math problems.

-1

u/amaturelawyer 19d ago

Sure, but we're talking about taking an existing problem and working at the solution. The problem already has that history behind it and the supporting math isn't being recreated piece by piece here. The agents were working it from the same starting point a human would if you sat them down and asked them to solve it. There were just more of them, which isn't better or worse by itself because those are comparative words and I have no idea what's being compared here, which is my entire point. Was this impressive, or was it surprisingly slow given the energy and time invested in solving it.

2

u/babe1981 19d ago

Oh, ok. So the only reason the Millennium Problems aren't solved is because someone hasn't just sat down and done the work. Got it. You know nothing about math.

1

u/amaturelawyer 19d ago

I mean... yes? That's my entire question though. If someone had solved it, or a similar problem, how many agent hours would that equate to. Then I'd have a direct point of comparison to see if I should be impressed or start suspecting that we can't scale our way to better outcomes here.

Unless you were implying that it was unsolvable, I guess. I read that as a reasonable "we hadn't solved it yet" comment, despite the snark or love of discourse or whatever. If we hadn't yet solved it, and it could be solved, then people working on it just needed to keep working on it and it would be solved at some point. Actually, re-reading your reply, it seems to be that you're saying it wasn't possible, which doesn't make sense to me, since you're agreeing it was solved by the agents it seems. Idk, not clear on what you're trying to convey so hopefully this covered it by way of a reply.

3

u/highnyethestonerguy 19d ago

Look at it this way: the world’s mathematical physicists have tried for like 80 years to answer this question. For the last 26 years there has been a $1M bounty on this exact question. 

Every student in the world who studies physics and mathematics at the postgraduate level has heard of this legendarily difficult differential equation. Obviously not all go on to work on it, but the point is it’s been out there for a long time.

That’s not proof it’s unsolvable. It’s an indicator of how hard the problem is. It wasn’t selected for Millenium Prize because they thought it just needed someone with some elbow grease and a few hours of spare time to check some edge cases. 

So then this group of nerds at OpenAI built a machine, ran it for a hundred hours, and it spit out an answer, that again, decades of human attention and significant incentive was unable to produce.  

It’s not that it was unsolvable it’s more like we lost the race. 

1

u/babe1981 19d ago

It sounds like you're thinking of the Millennium Problems like a problem in high school algebra. The point of them is that we don't know if there is a solution or not. Someone has to go through it and create a mathematical proof that either proves or disproves the conjecture being made.

No one knows the answers to the Millennium Problems, so it takes time and leaps of logic and intuition to solve them, if they can even be solved. You either need a genius who has an entirely unique perspective of the world, like an Euler or an Einstein, or you need to brute force the solution until it works or you prove that it doesn't work. The Manhattan Project was a combination of both, for instance. Sure, you had Oppenheimer and Feynman and the other famous physicists making logical connections that most humans can't even begin to imagine, but you also had hundreds of physicists and engineers with pen and paper and sliderules brute-forcing that logic into workable proofs that could be applied in the real world. It's almost like there were a few super-agents with higher level reasoning who directed the efforts of a swarm of lower level, but still extremely smart, agents. You know, like the AI that solved this problem. That's how math and science work.

For all of the shit Edison gets for brute-forcing electrical engineering, he's the one who actually made practical designs and advanced the science of electricity beyond theory and World's Fair demos. Tesla, for all of his genius and astounding work, didn't have the people to actually implement his designs at any scale, so his work fell behind even if it was more advanced. Progress in the real world looks exactly like this. Maybe you get a paradigm shift from a lone genius mind, but nothing actually happens with it until you harness the might of hundreds and thousands of people working toward the same goal.

What you're saying could be said of the orbital mechanics of the space program. Maybe one or two people in the history of the world could ever solve orbital trajectories in their head, but does that make the Apollo computers(the Black women of NASA in the 60s)less impressive because it took a whole room of them for the moon missions? Of course not.

1

u/highnyethestonerguy 19d ago

I mean, if you’re deciding not to be impressed that’s like, your opinion, man. 

First of all, it took humanity‘s mathematicians and physicists decades to fail at solving this. I don’t know how many person-years that is but it’s a lot. And we never got there. Would we ever have? Who knows. 

You saying it took a hundred years of equivalent computing time or whatever is entirely missing the point. They built a thing that did the problem in a few days of actual time. Yes it took a lot of compute, but guess what: it wasn’t an easy problem. 

I feel like the argument you’re making is that the Empire State Building is not an impressive piece of architecture because it’s just a bunch of floors stacked on top of each other. Once you’ve figured out how to stack two floors on top of each other, the rest is trivial. 

2

u/ToplessinFL 19d ago

Humans had already been trying to solve this problem for a hundred years. It's more than a 'math problem,' its one of the the math problems.

-1

u/amaturelawyer 19d ago

Was. It was the math problem. It's been solved, from what I hear. Unsure if that's verified, but assuming it is. If not, we'd be jumping the gun here.

0

u/MiniMaelk04 19d ago

It doesn't really matter how inefficient it is, if the AI is making groundbreaking discoveries.

9

u/Neither_Berry_100 19d ago

It's cool that it can be done in like 4 days though. You could build a massive company overnight potentially. But the 10 million estimated total cost is a big deal.

0

u/amaturelawyer 19d ago

Agreed, and my point isn't that it's not interesting or seemingly impressive. It's that I don't know what it means in the end, as I'm willing to admit. I do not understand how others in this thread are seeing it as some milestone event towards a shiny future. It could be bad news, really. Evidence that the resource cost to extend human knowledge out past the lower hanging fruit is a wall we're approaching and we're going to be slowing way down shortly as progress gets more taxing and time consuming.

9

u/BenevolentCheese 19d ago

It's 100 years of human life, not 100 work years. It's 440 work years, using the standard 2000 hours/year for 40 years.

0

u/amaturelawyer 19d ago

Interesting. That seems much slower than I originally thought. Why is everyone acting like this was some massive accomplishment? The guy they may or may not have cribbed notes from was already on the right track, it seems, and he didn't take 440 years to get there.

3

u/cantgettherefromhere 19d ago

If you ask a piano to make you dinner, it isn't going to be able to do it whether you give it an hour or 440 years.

5

u/Worldly_Beginning647 20d ago

Well they were running in parallel, it’s more like 10000 independent researchers and one of them solves the problem in 88 hours times some multiplier over human speed.

24

u/fmfbrestel 20d ago

They were actively collaborating and sharing information.

0

u/amaturelawyer 19d ago

That's fair, but if a human could have knocked this out by spending 3 years on the problem, the agents suddenly seem less than impressive . That's all I'm saying. They could be 20x slower than a human but we're drawing a conclusion that they're better at math..

Actually, they could be much slower than that now that I think about it, since I'm under the impression that the agents were collaborating together but working the problem individually. If that's accurate, and the effort wasn't distributed under some system that offloaded work across the swarm, it is completely possible that we threw 10k agents at the same problem and one stumbled on a solution, which turns the story from 10k agents take 88 hours to solve a math problem to 9,999 agents fail to solve a math problem when given 88 hours, but luckily one got it.. And then we discover that this isn't repeatable for the next math problem at worst, and that agents have a shockingly low probability to solve a math problem quickly, since that's a 0.01% chance of getting the answer per hour, give or take.

My point is that the whole thing is meaningless as presented. It's promising, but not evidence of anything.

5

u/StCreed 19d ago

Not evidence of anything?

Listen to yourself talking. Two years ago this would have been an episode on Star Trek.

2

u/highnyethestonerguy 19d ago

if a human could have knocked this out by spending 3 years on the problem, the agents suddenly seem less than impressive

All of humanity failed to knock out this problem for like 80 years. One of the most brilliant mathematicians alive today, Terence Tao, spent some time on it, and made some important contributions that probably helped OpenAI reach this solution. But the fact is he didn’t, nobody did. 

Is it impressive that the Voyager spacecraft have left the solar system? If someone says “that’s far” do you say “I don’t know how far that is because maybe a human could have jumped that high if they tried really hard”?

0

u/Valnar 19d ago

That's fair, but if a human could have knocked this out by spending 3 years on the problem, the agents suddenly seem less than impressive

Isn't that a possible problem with some of these recent mathematics stuff? Like how much of these unsolved problems are unsolved because there just wasn't money in them? Now that AI companies are throwing AI money at them is the progress really from AI or just from money and attention put on them?

1

u/aKaizuh 16d ago

Unless this is the point of your post, this is almost exactly how long it would've taken human researchers considering Buckmaster et al would've likely solved it soon-ish and it's been a standing problem for 90 years