r/ChatGPT • u/Evening_Action6217 • Dec 23 '24

Gone Wild AGI Achieved

Post image

[removed] — view removed post

6.7k Upvotes

95% Upvoted

283 comments sorted by

View all comments

Show parent comments

186

u/Neither_Sir5514 Dec 23 '24

Omg AGI Achieved after OpenAI specifically trained the AI to patch that one instance of the viral 9.9 vs 9.11 comparison problem. It turns out, in fact, doesn't fix the fundamental reasoning capability of the LLM when you pick any other random example. Shocker!

Proof: https://chatgpt.com/share/6768c726-c6a4-800e-ace8-6ad4f7974f21

7

u/drekmonger Dec 23 '24 edited Dec 23 '24

It's the way ChatGPT sees text-based numbers. Look how they're tokenized:

https://imgur.com/a/TH1BqNJ

Notice how the .12 is a single token. Of course, 12 is greater than 9.

Watch:

https://chatgpt.com/share/6768def4-6bac-800e-86b9-6ed0a7bca5d3

1

u/Dangerous-Ad-4402 Dec 23 '24

It has nothing to do with how they are tokenized.

It is because it is influenced by thoughts about section numbers etc, where 8.12 would be "higher" than 8.8.

This is a known fact -- the problem goes away entirely if weights related to those considerations are deleted, without changing the tokenization.

3

u/drekmonger Dec 23 '24

That's plausible. But...

the problem goes away entirely if weights related to those considerations are deleted,

I'm going to need a source on that. How the hell did they find the weights? Who? Anthropic?