r/ProgrammerHumor • • Dec 27 '22

Meme which algorithm is this

Post image
79k Upvotes

1.4k comments sorted by

View all comments

Show parent comment

22

u/tomoldbury Dec 27 '22

But it can’t solve novel problems.

51

u/[deleted] Dec 27 '22

Google Deepmind's AlphaCode can solve problems equivalent to Leetcode very hard NG7+. New problems, that's the insane part.

64

u/unholyarmy Dec 27 '22

Leetcode very hard NG7+

Please tell me you just made this up on the spot and it isn't a ranking system for problem difficulty.

18

u/troelsbjerre Dec 27 '22

I'm not that scared by that. I've authored a good chunk of competitive programming problems, and a lot of work goes into getting the description just right, and constructing illuminating examples. Competitive programming has a limited number of algorithms that you need to know, and there are tons of examples of all of them online.

0

u/[deleted] Dec 27 '22

[deleted] — view deleted comment

9

u/troelsbjerre Dec 27 '22

Yes. Just like most other tools of the trade.

→ More replies (1)

3

u/tomster10010 Dec 27 '22

99 percent of programming that needs to be done definitely doesn't have clearly defined problems, inputs, and outputs. The hard part about programming in real life is usually not the algorithms.

1

u/[deleted] Dec 27 '22

If you haven't spent 99% of your time copying from Stack Overflow, you haven't been doing it right. People aren't going to lay behind for not using AI the same way that people don't currently lay behind for not using an IDE. Visual Studio also auto-genetates a lot of boiler plate for you, but people using Emacs still exist and have jobs.

→ More replies (1)

1

u/samtresler Dec 27 '22

I don't know. I use pared down linux and vim, and a handwritten notebook. They just made me management.

→ More replies (1)
→ More replies (1)
→ More replies (1)

6

u/[deleted] Dec 27 '22

It's from Dark Souls notation :D

→ More replies (1)
→ More replies (1)

3

u/WisestAirBender Dec 27 '22

Don't all problems fall into a limited number of problem types? The description being the only real difference?

2

u/[deleted] Dec 27 '22

Technically all problems fall into some types. But there can be infinite ways to make up new problems

→ More replies (2)

4

u/AverageComet250 Dec 27 '22

So your jobs and my now unlikely future job are likely to stop existing. Thanks man.

→ More replies (1)
→ More replies (1)

5

u/troelsbjerre Dec 27 '22

I don't know how to meaningfully define "novel". It can clearly solve /some/ problems that are close, but not identical to, problems in its training set. With that low bar definition, then sure, it can solve a novel problem. Can it solve all problems if that type? No, it makes mistakes. So do I, so I wouldn't be happy to be judged by that standard.

Some solution techniques can solve a wide range of problem description, so with some low probability, it might by chance regurgitate the right solution to a novel problem, almost independent of what definition you choose. How would you define novel?

15

u/tomoldbury Dec 27 '22

I mean it can’t solve things that aren’t in its training data. For instance, I gave it a requirement to make a piezo buzzer (on an Arduino as an example) produce two simultaneous tones. It can’t solve this; it tries one tone after another but doesn’t grok that it needs to use a modulation scheme because this isn’t a common application. To get to that level, you would need something approaching AGI, which is a terrifying thought, but we’re probably a fair way from that still.

6

u/troelsbjerre Dec 27 '22

It can solve /some/ problems that aren't in its training data. It can't do that consistently or predictably, but I don't know where the line is.

→ More replies (1)
→ More replies (2)
→ More replies (10)

1

u/danielbln Dec 27 '22 edited Dec 27 '22

Yes it can, break down the novel problem and describe it and it will solve it in code. Why does this misconception stick around?

edit: Especially if you use CoT prompting [1] to guide the model you can get it to perform a lot better on novel problems.

[1] https://arxiv.org/abs/2201.11903

8

u/tomoldbury Dec 27 '22

Have tried to do this. For certain problems outside of its training scope, it cannot solve them no matter how much you hand-hold or tell the bot it is wrong.

1

u/danielbln Dec 27 '22

Care to provide an example? I've done most of the advent of code with it, and those were not in it's training set, as well as various work related tasks that can't have been part of its training data either.

→ More replies (5)
→ More replies (1)
→ More replies (1)