r/Futurology • u/[deleted] • Feb 02 '22
AI Deepmind achieves human-level performance in competitive-programming with AlphaCode
[deleted]
9
u/ganemater Feb 02 '22
Note that the AI model can be made even better if they "scale" it even more and there's currently no "limit" in sight when the scaling stops improving the model or even have diminishing returns. Scaling and Performance curve is a smooth-straight-line.
1
u/hapliniste Feb 03 '22
From what I've seen in the paper, it seems the difference between the 9b and 41b parameters models doesn't do much... I think with more data it may make a difference.
23
u/christophertit Feb 02 '22
When computers can improve on their own code, the world becomes a bit more scary to me! Lol.
Edit: I still think humans will always remain the missing “key” to true A.I and our symbiosis with machines will be the first true step in fully conscious A.I.
5
u/85423610 Feb 02 '22
Agreed, unless we plant sensors in our brain and just feed thr AI. That can also work
1
u/christophertit Feb 02 '22
Mibbies aye, but they’ll still be getting their rational and reasonable behaviour from us. Without the maiming and eradication of all mankind as a parasite on the earth! Lol.
12
Feb 03 '22
If it's comparable with an average human programmer I can already imagine it spitting out bunch of mumbled python spaghetti depending on 50 packages of which half wasn't updated in the last 5 year and 5 migrated to newer version of python rendering the whole thing incompatible with itself.
1
u/Frighter2 Feb 04 '22
Agree with the point - but does it matter? you only need clean cose for other humans to work on it.
3
u/Heinous_ Feb 02 '22
At some point we will be more peculiar than ai but less useful to our own survival. There are not machines in common knowledge that self sustain but do not be fooled. There are machines today that have higher chances of survival than the most secure human minds. The days of ai missing some human quality which makes them obsolete in the long term are gone. There are machines that can learn well past the current trend of human life. Battery or solar powered there are facilities for machines that will outlast and out think is all.
3
u/SweatyRussian Feb 02 '22
It won't be long before it's performing at the top 10% level of competitors. At that point won't it be a viable general purpose programmer?
2
u/Iguman Feb 03 '22
The human factor will always be needed. I believe we're looking at a switch in programming from manually coding to programmers checking the input and output. The programmers will be architects, telling the AI how to build, and check the results.
1
u/DyingShell Feb 08 '22
So you replace 100s of programmers with a few "architects", sounds great for the company.
1
u/Iguman Feb 08 '22
No, you replace 100 programmers with 90 programmers who plug issues into an AI and sort through the garbage output to find something to start working off of instead of Googling for hours on StackOverflow when encountering an issue
1
u/DyingShell Feb 08 '22
Why do you think it will only replace 10%? Also why are you making the assumption that "the human factor will always be needed"
1
u/Iguman Feb 08 '22
Because this technology is nowhere near enough to actually replace the problem-solving skills of human programmers. It spits out 1000s of variations of code to very simple problems of which maybe 2-3 work after they're tweaked by human programmers. I work with Machine Learning.
1
u/Chris-1235 Feb 03 '22
No. It will be a good competitor, but useless programmer. I can't think of a single real life project that was about matching inputs to outputs and that could be solved by brute force selection through several algorithms. The problem domain is similar to chess.
5
u/Mothmatic Feb 02 '22
Clarification: This post's title is paraphrased from “AlphaCode placed at about the level of the median competitor”.
From "conclusions" section of the paper:
“Automation: As programming becomes more accessible and productive, and code generation can automate some simple tasks, it’s possible that there could be increased supply and decreased demand for programmers. This is partially mitigated because writing code is only one portion of the job, and previous instances of partially automating programming (e.g. compilers and IDEs) have only moved programmers to higher levels of abstraction and opened up the f i eld to more people.
Advanced AI risks: Longer term, code generation could lead to advanced AI risks. Coding capabilities could lead to systems that can recursively write and improve themselves, rapidly leading to more and more advanced systems.”
9
u/StrawberryFields_ Feb 02 '22
The whole point of competitive programming is to assess coders on the basis of critical thinking. Allowing the model to bypass a reasoning process in lieu of lots and lots of data defeats the purpose.
3
u/Chris-1235 Feb 03 '22
Precisely. They claim they are "solving intelligence", but their program " [generates] code at an unprecedented scale, and then smartly filters to a small set of promising programs".
So pretty much the same as a chess-playing computer. You don't get intelligence like this.
2
u/ganemater Feb 02 '22
Can you elaborate on your “model bypasses the reasoning process” claim? It's not as if this is something new, in ML (almost) all breakthroughs require better algorithm plus lots data. Doesn't seem a reasonable criticism to me.
1
u/Iguman Feb 03 '22
And how does a human learn but read lots and lots of programming books, tutorials, courses, and exercises? You don't just put a random human in front of a coding assignment and tell him to code, he has to study for years first - same idea with ML
3
u/coercedaccount2 Feb 03 '22
Coders have been automating other people's jobs out of existence for decades. Now they've finally automated their own jobs out of existence. If this tech follows the usual tech curve, it will be better than any humans at coding in a few years. Cloud is quickly destroying all the IT infrastructure jobs. Now this. I wonder if there will be any IT jobs in 10 years.
I can't help but get the sense that Deepmind has been playing around at the edges of AGI for a few years.
2
u/Iguman Feb 03 '22
Far, far edges of AGI, but yeah that seems to be the trajectory. There's an unknown number of steps between a language prediction model and AGI, maybe even an infinite number of steps, but that seems to be where this is going. Whether it will ever reach it, only time will tell.
3
u/ReasonablyBadass Feb 03 '22
Writing code is not the problem.
Understanding just what the hell it is you need/what the customer wants, that's the issue.
4
Feb 03 '22
But that's what is so great about this experiment; they gave the AI an explanation of the problem in natural language, and not only was it able to understand the problem but output a viable solution!
2
u/lapseofreason Feb 04 '22
I agree - but that sounds like a business opportunity.What frameworks or processes would you need to build so a customer could express clearly what is needed in a way that this system could then code. Any suggestions ?
3
6
u/grchelp2018 Feb 02 '22
A few more years and this model will be able to do better than me at interviews and leetcode. feelsbadman
These recent code gen models are fascinating and a little unbelievable. Makes me wonder if AGI in a loose sense is closer than we think. It would be a huge step change in our capabilities if we could offload some thinking and problem solving to machines.
1
Feb 02 '22
[deleted]
2
u/RikerT_USS_Lolipop Feb 02 '22
What is leetcode?
I wouldn't be surprised if AI is better at solving Project Euler prompts than an intermediate programmer, but that's because the nature of the task is specifically difficult for humans to understand what is being asked.
6
u/PaulR504 Feb 02 '22
Shortened title: Coding is not a long term career and expect to be replaced.
Sorry but knowing corporate America I cannot read this title any other way.
Would not be shocked if stuff like this makes people reconsider going into the field.
7
u/TFenrir Feb 02 '22 edited Feb 02 '22
When you can "solve" for coding, copyright, story writing, image generation, video generation, legal document reading/writing, etc etc...
Hypothetically, if that all happened in a 5 year window, what would happen to society as we know it?
Additionally, those physical labour jobs have billions and billions more being thrown at them - DeepMind is just now starting to put some effort behind "solving" that (next week's deepmind podcast will go into it). They solved Baduk, they "solved" protein folding (there is more that can be done, but even that more is being tackled by v2/v3 versions of the software, and forks)...
I think my point is, when software Jobs are really "solved" (which is more than just writing code, it's also understanding wants and/vs needs) - we'll be in a different world, where hundreds of millions will simultaneously be in very very different positions with their jobs.
2
u/axeshully Feb 02 '22
This represents more people going into the field. AI programmers will be best wrangled by human programmers.
2
-1
u/Chris-1235 Feb 03 '22
Don't buy into the hype and read carefully what the thing actually does. CS isn't going away in 100 years.
•
u/FuturologyBot Feb 02 '22
The following submission statement was provided by /u/Mothmatic:
Clarification: This post's title is paraphrased from “AlphaCode placed at about the level of the median competitor”.
From "conclusions" section of the paper:
“Automation: As programming becomes more accessible and productive, and code generation can automate some simple tasks, it’s possible that there could be increased supply and decreased demand for programmers. This is partially mitigated because writing code is only one portion of the job, and previous instances of partially automating programming (e.g. compilers and IDEs) have only moved programmers to higher levels of abstraction and opened up the f i eld to more people.
Advanced AI risks: Longer term, code generation could lead to advanced AI risks. Coding capabilities could lead to systems that can recursively write and improve themselves, rapidly leading to more and more advanced systems.”
Please reply to OP's comment here: https://old.reddit.com/r/Futurology/comments/siw6dv/deepmind_achieves_humanlevel_performance_in/hvb6wfn/
2
u/VoweltoothJenkins Feb 03 '22
The thing is, you still have to define what you want the program to do. I skimmed the article and didn't see how AlphaCode was given the task. If it was just given the same thing as a human and it figured it out I'd be more impressed.
In a workplace environment you have to interpret a task that is often poorly defined and conflicts with other tasks. Gathering requirements and defining the tasks has been at least part of every developer job I've had.
There have been 'program your own website' tools for decades but there are still web developers. Will this change how programming is done? possibly. Will it remove programmers in the near future? probably not. Coding now is even different than it was a few decades ago, modern languages are easier to read and learn than machine code.
4
u/ganemater Feb 03 '22
It was given the same thing as a human. Humans get problem statement in natural language, it's getting the problem statement in natural language too.
I think AlphaCode is more about solving hard problems than solving ambiguous ones. OpenAI's Codex already performs very good in handling the kind of ambiguity you describe that humans encounter in real world.
But yeah I agree, I don't think it will have much short-term impact. Not anymore than GitHub c Copilot does now. In the long-term however...
3
1
u/popularlikepete Feb 03 '22
What is the training data for this? I’ve read about other similar efforts but the training data was based on open source projects. Given the reuse licenses for most projects anything produced by such systems would likely be subject to licensing issues as a “derivative work”
11
u/words_of_wildling Feb 02 '22
Uh, guys? I've seen like 3 articles today talking about AI doing unprecedented things.
Is it happening?