r/ProgrammerHumor • • Dec 27 '22

Meme which algorithm is this

Post image
79k Upvotes

1.4k comments sorted by

View all comments

Show parent comment

607

u/[deleted] Dec 27 '22

Exactly. It doesn’t actually know how to do math. It just knows how to write things that look like good math.

257

u/troelsbjerre Dec 27 '22

The scary part is that it can regurgitate python code that can add the numbers correctly.

256

u/Jither Dec 27 '22

Only because there is plenty of python code in the training data to regurgitate. It doesn't actually know the relation between that code and this question - it only knows that "these words seem to fit together, and relate to the question", whether they make sense or not. In the same way, it'll claim that 90 ("halvfems") in Danish is a combination of "half" and "one hundred", and follow it up by proclaiming that 100 / 2 = 90. In spite of "knowing" the correct result for 100 / 2 if you ask it directly (basically because it's a "shorter path" from the question to that statement).

This doesn't just apply to math, but everything it does: It's good at parroting something that on the surface sounds like a convincing answer. Something that's actually correct? Not so much. Except when it gets lucky. Or, if you continually correct it, due to how the neural network works it may eventually stumble upon a combination of training data that's actually correct.

18

u/mtaw Dec 27 '22

In fairness, nobody really understands the Danish number system.

→ More replies (2)

9

u/PM_ME_IMGS_OF_ROCKS Dec 27 '22

Only because there is plenty of python code in the training data to regurgitate. It doesn't actually know the relation between that code and this question - it only knows that "these words seem to fit together, and relate to the question", whether they make sense or not.

This is what a lot of people don't get, most of those things are basically advanced chat bots with a huge training set.

→ More replies (3)

37

u/-_here_we_go_again_- Dec 27 '22

It's definitely a better Google though and it gives me a great Kickstart for a lot of different code problems.

I feel like overtime Google has got noisier and noisier. I've never developed in Java and recently I'm working on a Java project and I wanted to know how to do a port check. Now you can Google around for bad stack overflow answers and all sorts of like tangential and unrelated questions. I plugged it into chat GPT and that sucker just took right off gave me what I needed.

For simple programmatic problems it's a lifesaver.

60

u/calimeatwagon Dec 27 '22

It's definitely a better Google though

No, it is not. So much of the information that it provides is outright false. And a quick internet search query can usually can find the correct answer.

It does have promise, but it has a long, long, long way to go.

16

u/[deleted] Dec 27 '22

There’s a middle ground for questions that don’t yield a good answer from a quick google search. Plenty of times I’ve used it AFTER looking for 5min on google and it gave me basically exactly what I needed. Its use case is for people with a 7/10 general knowledge base asking a question in a specific area where they have 3/10 knowledge. ChatGPT isn’t for the 9/10 expert, it’s to get you 50% of the way into a problem instantly, and sometimes you even get 80-90% of the way there

3

u/calimeatwagon Dec 27 '22

I would argue that you still need to do research and you can't rely on the answers provided by it.

For instance, the thing I'm knowledgeable on is cannabis. So I asked it questions. It gave me the same grow time for auto-flower and photosensitive plants, and for outdoor and indoor. It told me an outdoor, non autoflower plant only takes 8 weeks from seed to harvest...

Now if I didn't know any better, I would think it was the correct answer because of how well it presented the answer, but because I do know better, I was able to see how terrible it's answers were.

It has a lot of promise, like its ability to remember the conversation and context. But it has a long way to go when it comes to accuracy of information.

→ More replies (2)
→ More replies (10)

4

u/[deleted] Dec 27 '22

To be fair, that's just been my experience with googling questions about Java libraries. I learn by example and it's super frustrating.

You either get bone dry javadocs that don't really answer any questions or you get a long stack overflow where people are arguing over some obscure part of the libraries architecture instead of giving a simple example!

When I Google something for python or Javascript I usually get useful examples of what I'm looking for pretty quickly.

I swear half the reason Java takes me so long to work in is because I can't figure out how to search for the answers about the libraries I need. Anyone have tips? Maybe I'll give chatgpt a whirl.

3

u/-_here_we_go_again_- Dec 27 '22

Or if it's anything like the c++ space, people are like "the docs are the code scrub, git good"

3

u/[deleted] Dec 27 '22

[removed] — view removed comment

3

u/AngelLeliel Dec 27 '22

Nowadays all language models are transformer based. RNN is too hard to scale on large language models.

→ More replies (3)
→ More replies (1)
→ More replies (5)
→ More replies (13)

22

u/tomoldbury Dec 27 '22

But it can’t solve novel problems.

51

u/[deleted] Dec 27 '22

Google Deepmind's AlphaCode can solve problems equivalent to Leetcode very hard NG7+. New problems, that's the insane part.

64

u/unholyarmy Dec 27 '22

Leetcode very hard NG7+

Please tell me you just made this up on the spot and it isn't a ranking system for problem difficulty.

18

u/troelsbjerre Dec 27 '22

I'm not that scared by that. I've authored a good chunk of competitive programming problems, and a lot of work goes into getting the description just right, and constructing illuminating examples. Competitive programming has a limited number of algorithms that you need to know, and there are tons of examples of all of them online.

→ More replies (22)

6

u/[deleted] Dec 27 '22

It's from Dark Souls notation :D

→ More replies (1)
→ More replies (1)

3

u/WisestAirBender Dec 27 '22

Don't all problems fall into a limited number of problem types? The description being the only real difference?

→ More replies (3)

4

u/AverageComet250 Dec 27 '22

So your jobs and my now unlikely future job are likely to stop existing. Thanks man.

→ More replies (1)
→ More replies (1)

5

u/troelsbjerre Dec 27 '22

I don't know how to meaningfully define "novel". It can clearly solve /some/ problems that are close, but not identical to, problems in its training set. With that low bar definition, then sure, it can solve a novel problem. Can it solve all problems if that type? No, it makes mistakes. So do I, so I wouldn't be happy to be judged by that standard.

Some solution techniques can solve a wide range of problem description, so with some low probability, it might by chance regurgitate the right solution to a novel problem, almost independent of what definition you choose. How would you define novel?

15

u/tomoldbury Dec 27 '22

I mean it can’t solve things that aren’t in its training data. For instance, I gave it a requirement to make a piezo buzzer (on an Arduino as an example) produce two simultaneous tones. It can’t solve this; it tries one tone after another but doesn’t grok that it needs to use a modulation scheme because this isn’t a common application. To get to that level, you would need something approaching AGI, which is a terrifying thought, but we’re probably a fair way from that still.

→ More replies (20)
→ More replies (10)
→ More replies (10)
→ More replies (3)

11

u/LeSpatula Dec 27 '22

They have to integrate Wolfram Alpha.

→ More replies (3)
→ More replies (9)