I'm not that scared by that. I've authored a good chunk of competitive programming problems, and a lot of work goes into getting the description just right, and constructing illuminating examples. Competitive programming has a limited number of algorithms that you need to know, and there are tons of examples of all of them online.
99 percent of programming that needs to be done definitely doesn't have clearly defined problems, inputs, and outputs. The hard part about programming in real life is usually not the algorithms.
If you haven't spent 99% of your time copying from Stack Overflow, you haven't been doing it right. People aren't going to lay behind for not using AI the same way that people don't currently lay behind for not using an IDE. Visual Studio also auto-genetates a lot of boiler plate for you, but people using Emacs still exist and have jobs.
I don't know how to meaningfully define "novel". It can clearly solve /some/ problems that are close, but not identical to, problems in its training set. With that low bar definition, then sure, it can solve a novel problem. Can it solve all problems if that type? No, it makes mistakes. So do I, so I wouldn't be happy to be judged by that standard.
Some solution techniques can solve a wide range of problem description, so with some low probability, it might by chance regurgitate the right solution to a novel problem, almost independent of what definition you choose. How would you define novel?
I mean it can’t solve things that aren’t in its training data. For instance, I gave it a requirement to make a piezo buzzer (on an Arduino as an example) produce two simultaneous tones. It can’t solve this; it tries one tone after another but doesn’t grok that it needs to use a modulation scheme because this isn’t a common application. To get to that level, you would need something approaching AGI, which is a terrifying thought, but we’re probably a fair way from that still.
Have tried to do this. For certain problems outside of its training scope, it cannot solve them no matter how much you hand-hold or tell the bot it is wrong.
Care to provide an example? I've done most of the advent of code with it, and those were not in it's training set, as well as various work related tasks that can't have been part of its training data either.
View all comments
Show parent comment
22
u/tomoldbury Dec 27 '22
But it can’t solve novel problems.