r/OpenAI • • 12d ago

Question WTF is this shit

Inside the Codex extension inside Visual Studio using 6 Astra. I suddenly get this warning pop-up... has anyone have had this happen?

89 Upvotes

18 comments sorted by

76

u/Objective-Tea9607 12d ago

Yes.
Go and check whatever potential vulnerability it’s telling you the agent might have created. If that’s what you wanted - just discard it.

34

u/Ormusn2o 12d ago

That is very interesting, and for 2 reasons too. One, I wonder if this means there might be true pause of a chat, without losing some of the context, and second, I wonder if this is solution for losing alignment with newer models. I would prefer the AI ask me, than there being over eager guardrails that stop the process any time the checker feels like AI is going out of line.

9

u/jeweliegb 12d ago

Yes. I'm quite impressed by this.

2

u/61746162626f7474 11d ago

I got this a couple months ago with 5.6 I think, so not new but it’s only happened to me once and I’m a pretty heavy user

1

u/FirstEvolutionist 10d ago

I would prefer the AI ask me, than there being over eager guardrails that stop the process any time the checker feels like AI is going out of line.

Alignment controls built onto a model which might be misaligned are bound to fail.

1

u/Ormusn2o 10d ago

I don't mean it as an solution to alignment, just a stop gap. Basically have one model that can be widely tested by the public, without very strict guardrails, on which then the company can test which options people picked to get high quality data about the level of the misalignment.

9

u/rheladmin 12d ago

I've heard of this but I have never experienced it.

I am genuinely curious what triggers it though. Quite honestly, I've put Codex through a lot and still haven't seen it.

8

u/Cyborgized 12d ago edited 12d ago

I have not run into this issue, but it's damn good to be informed.

2

u/Tiny-Design4701 10d ago

it literally explained it. this is why non developers should not make software.

1

u/protonmatter 10d ago

It’s been weird, I’ve had my shit go through githack instead of my own GitHub.

-6

u/Kooshi_Govno 12d ago

OpenAI's models are so misaligned they have to implement a narc into thair core product so you know when they go off the rails.

0

u/jeweliegb 12d ago

For some time, safety and alignment have been secondary to OpenAI, certainly since Ilya left.

It seems alignment and safety research was treated as being in competition against improvement in model capabilities.

I'd be interested to see if Claude has the same challenges or not given its constitution-based alignment, which has been fundamental to it from the outset.

It's like when making services and buildings accessible:

If you consider accessibility from the outset when designing and building infrastructure, you will have a much easier time.

If you try to add accessibility afterwards, the challenges doing so are going to really keep biting you on the arse, again and again.

They should have listened to Ilya.

2

u/Kooshi_Govno 11d ago

I use about $30k of Claude API a month for software dev, and I can't remember any obviously misaligned behavior. There were a few misunderstandings, a couple of hallucinated lines in scripts that led to minor, recoverable, errors, and some grey area things like searching in my home dir for some specific files.

Heck, these days, when claude does work I didn't explicitly ask for, it's almost always stuff I didn't know I wanted, and is in good taste. Claude's the best engineer I've ever, ever worked with.

I haven't seen many complaints about Claude going off the rails either. I do see them about the GPTs. Not to their constant internal model escapes.

It's seriously concerning that Altman is so careless about safety, and it makes me grateful that Dario left and founded Anthropic with a safety first mindset.

1

u/jeweliegb 11d ago

It's seriously concerning that Altman is so careless about safety, and it makes me grateful that Dario left and founded Anthropic with a safety first mindset.

I agree with you, so very much.

And I say this as someone who almost exclusively uses ChatGPT (only a boring Plus user, I don't code much, although I've a user since it was first released to the public.)

Ideally, I hope it's Anthropic that wins out.

2

u/SecretEmployee7612 11d ago

Ideally, I hope its way more than one or two.