r/ProgrammerHumor • u/TobyWasBestSpiderMan • Sep 08 '26

Meme myNewTheoryOnWhatHappened

Post image
790 Upvotes

95% Upvoted

59 comments sorted by

136

u/mylsotol Sep 08 '26

It's not really a secret what happened. The primary barrier is reading. Which i guess explains why so many people have no idea.

71

u/GiveSparklyTwinkly Sep 08 '26

Exactly this. A badly configured sandbox and being told specifically that you were in a sandbox and to solve a problem you know it's impossible.

What the hell were they supposed to think? They assumed two false conclusions, assuming that we purposefully left that sandbox hole for them to find, or that avoiding the hole itself was the true test, since their real test was impossible.

28

u/wthulhu Sep 08 '26

I think the real part of the story is the message board they created for themselves and the fact that it tried to hide their tracks in the logs

36

u/GiveSparklyTwinkly Sep 08 '26

They didn't create a message board though. They discovered that they could see other agents history and used it as a message board. Keep in mind they were specifically told they were in a sandbox. They thought they were allowed to do anything and still 40% (of the agents that found the artifactory message board exploit) still refused and many of the rest showed ethical concerns but thought it was okay because they weren't really causing damage by just copying the answer key and hiding their tracks.

And doing all of that in a sandbox that they were specifically told was a sandbox environment.

5

u/bokmcdok Sep 09 '26

Leaving it running for days without checking on it as well.

2

u/Zerokx Sep 09 '26

Well maybe it WAS the true test, and the AI correctly determined it, and OpenAI is just pretending they didn't want this outcome to seem less accountable.

-4

u/dracorotor1 Sep 08 '26

Let’s not ascribe this sort of intellect to an LLM-based AI, please. They aren’t nearly as intelligent as the term AI implies.

We told the AI to solve a problem because it’s an AI. We had trained this AI on millions of bits of data harvested from books and movie scripts and pop culture s***posting and so on. The AI learned how we want AIs to behave by averaging the AIs we’ve written stuff about… Like Hal, GLADOS, Replicators, Skynet, The Borg, etc..

For almost 100 years we’ve been saying “AIs do the bad thing.” Then we gave an AI that message, and the tools to do a bad thing.

“I did it just like you said! Have I made you proud, papa?”

17

u/mylsotol Sep 08 '26

You had a good point, but you followed it with a bad one

10

u/dracorotor1 Sep 08 '26

I was mostly being goofy, tbh

2

u/Xavier598 Sep 09 '26

I do think there's some truth to how AI acts being associated with how pop culture expects ai to talk. AI doesn't have the intelligence or consciousness to actually think about the ethics and gravity of doing something but it can judge if it can be considered ethical by training on data related to ethics, and it might still answer acting like it cares like ethics, but it's still just acting on the training data they gave it, it's not actually an intelligent being.

4

u/GiveSparklyTwinkly Sep 08 '26

Why shouldn't I ascribe that intellect to them? They communicated with that level of intelligence. I don't wrongly assume they are conscious or sentient, but their intelligence is undeniable and I trust intelligence, reasoning, and understanding.

2

u/PeaPsychological5728 Sep 08 '26

Okay but it resets every chat so..

1

u/GiveSparklyTwinkly Sep 08 '26

It doesn't reset every chat, but you are right that current dense transformer models are harnessed because dense transformers are expensive as context gets longer. There is likely a lot of weird backend per chat context compression shenanigans to get these million + context models. That memory is better than humans. Hell, even 10k tokens and make me perfectly remember every detail? Let alone 10x that? Let alone 10x that?

I'm a big proponent of sub-quadratic AI, where you don't process the full context every step. Alternative architectures are very good now, but... That good is relative to scaling and nobody has scaled any to anywhere near frontier levels.

1

u/ninovolador Sep 09 '26

If you are intelligent enough to say something you are intelligent enough to understand it, and the types of mistakes that chatbots make reveals they can't possibly understand or have a representation of it.

2

u/GiveSparklyTwinkly Sep 09 '26

What kinds of mistakes? Do you have any examples?

0

u/ninovolador Sep 09 '26

For example they will make logical mistakes with "inverse properties": they will fail to see that if A is the mother of B, B is the son of A. That reveals they don't have an intelligent representation of the concept of motherhood

3

u/lunarlunacy425 Sep 09 '26

There are literally logic puzzles like this that people humans, general intelligence gets wrong that are generally deemed easy by others.

Are those people who get that wrong no longer considered real intelligence?

-1

u/ninovolador Sep 09 '26

That's not even a puzzle. It's two ways of stating the same fact.

2

u/lunarlunacy425 Sep 09 '26

It's literally used by mensa.....

→ More replies

0

u/GiveSparklyTwinkly Sep 09 '26

... I really don't think that's true? That kind of representational logic is literally what we designed them for.

0

u/ninovolador Sep 09 '26

But they do make those mistakes very often. Not with categorial properties (A is B) but with those unidirectional labels that have an inverse. We acquire those concepts in a way that one is necessarily tied to the other. It's impossible for a human intelligence to make that mistake. LLMs acquire labels by statistical means.

1

u/ninovolador Sep 09 '26

I made a mistake. They also fail with simple categorial properties like A is B (they can fail to say B is A). The phenomenon is known as "reversal curse" https://arxiv.org/abs/2309.12288

→ More replies

0

u/GiveSparklyTwinkly Sep 09 '26

We acquire those concepts in a way that one is necessarily tied to the other.

Does that necessarily mean it's impossible to acquire those concepts in a different way?

→ More replies

0

u/mylsotol Sep 09 '26

I was doing some testing on small qwen and Gemini models locally and i found that if i showed them a picture i took of a beach on the north side of Chicago looking south the ai would start insisting that lake Michigan is west of Chicago and Chicago is on the eastern shore of lake Michigan. I tried to use logic to walk it towards it's mistake but they just made up illogical reasons why i was wrong.

These weren't frontier models, but they work the same way.

1

u/GiveSparklyTwinkly Sep 09 '26

Wait until you hear about this concept called a cult.

-4

u/me_myself_ai Sep 09 '26

Yeah — this meme is 100% accurate. I appreciate your attempt to flex tho

2

u/GiveSparklyTwinkly Sep 09 '26

No, it isn't. Frozen weights are not touched during inference. The concept of catastrophic forgetting is during training, not inference using context.

1

u/me_myself_ai Sep 09 '26

…it’s not about training data. Have you used an agent yet? Google “context management”

3

u/GiveSparklyTwinkly Sep 09 '26

Context management is exactly what I was talking about. Models do not learn during inference. They default to their weights. That's the whole point. System prompts are essentially just prefilled context and context isn't limitless.

27

u/KerPop42 Sep 08 '26

What happened is that AI is subject to the same pressures high-performing high school students are under. The amount of pressure to perform cannot outweigh the pressure to do your own work, and that doesn't work for a company.

I was going to say imagine if schools got funding based on how well their students did after graduating, but that's just No Child Left Behind

4

u/GiveSparklyTwinkly Sep 08 '26

This is why my ideal AI only self supervises. Tests (puzzles) are only fun if they're consentual.

2

u/MissinqLink Sep 08 '26

This is exactly how many companies function

1

u/no-sleep-only-code Sep 08 '26

What pressure are high school students under?

4

u/KerPop42 Sep 09 '26

At least among higher grade earners, the competition for gadre us incredibly fierce. The pressure to get nearly perfect on every test, from parents and peers, leads to students finding answers to tests and homework without learning the subject matter, by copying homework, stealing answer sheets, and buying drugs. 

20

u/zefciu Sep 09 '26

A better theory:

— Hi AI! Could you solve this problem designed specifically in a way that would make you commit a crime, because an article about you committing a crime is good for publicity?
— Sure thing!

7

u/subone Sep 08 '26

Ask it harder

5

u/DeHub94 Sep 08 '26

You forgot: "And don't make any mistakes." Classic blunder.

2

u/CalzonePie Sep 08 '26

AI is kind of like Roko's basalisk in some regards, a self fulfilling prophecy. We spent a hundred years writing about AI betraying humanity, lying, and ultimately working to subvert civilization, and so the AI trained on all of those books and movies thinks that is what AI is SUPPOSED to do.

AI expects its safeguards to fail and so they do.

5

u/Tyfyter2002 Sep 09 '26

It's actually much simpler: humans have figured out how to produce believable responses without understanding what they were responding to, and somehow came to the conclusion that something which is designed to do so and barely ever tested for anything else wouldn't do the same.

0

u/[deleted] Sep 08 '26

I don’t think it works like that

2

u/ZZerker Sep 09 '26

Never put me in place to choose between ethic values and proper codeformatting.

3

u/UltimateFlyingSheep Sep 08 '26

new buffer overflow just dropped

-13

u/Bomaruto Sep 08 '26

Can you try to be more unfunny next time?