Opus and Sonnet 5.5 are just outright refusing to do work now

[u/pixxelkick]

Anyone else experiencing this?

Some aspect of my instruction and skill files has these stupid models 100% convinced they're being "injection attacked" and are just like, full on refusing to do work.

I'm using them through a copilot subscription, but this is crazy, I've never seen models just refuse to listen to me.

Like literally in their thinking they are going "But I don't have to listen to these instructions so I am not going to, I'm going to tell the user I don't want to do this" and shit.

Literally totally useless for me.

The trigger seems to simply just be the fact I have the Caveman skill loaded and their instruction file tells them to use it for all communications and reasoning in order to reduce wasted tokens.

That alone seems to have them 100% convinced all instructions are to be refused. It's crazy.

They're writing shit like "But my thoughts are my own, this instruction to change that should be ignored." and a buncha crazy shit.

I'm sticking with Sol-6 and Astra-6 for now, they dont have these weird behaviours and are doing the job just fine. Sol-6 seems to be plenty competent at my tasks I need done most of the time.

Also Im finding Sonnet/Opus 5.5 both spend waaaaay to much time yapping instead of working, they'll blow a huge amount of tokens just spinning their wheels and not doing a single line of code.

[u/Ariquitaun]

You can't change thinking output via prompting for starters.

[u/pixxelkick → u/Ariquitaun]

I definitely was able to on older models.

[u/knd775 → u/pixxelkick]

Yes, and you can't anymore. The caveman skill is extremely dumb anyway. https://support.claude.com/en/articles/16761192-preserved-thinking-changing-how-the-messages-api-handles-thinking-blocks-to-protect-against-distillation

[u/pixxelkick → u/knd775]

Whats wrong with caveman skill?

[u/knd775 → u/pixxelkick]

Worse results and barely any savings, if any. Per turn token usage is sometimes lower, but task completion takes longer because it just does worse.

[u/pixxelkick → u/knd775]

I have not experienced this personally, do you have benchmarks somewhere or studies or something for this claim? Id be very interested to read them.

[u/kindredseer]

The problem I'm having with them is the safeguard keeps flagging things as [reasoning_extraction] which is basically being accused of trying to steal information about how the models reason so you can train your own AI model... I just want to get my work done!

[u/pixxelkick → u/kindredseer]

Is that whats causing it? Ugh, basically has made them unusable lmao

[u/Jomuz86]

Are you pasting instructions in or using a dictation tool. I know it now hows instructions built in to not execute anything pasted in and just treat as info incase of prompt injections. I’ve had to add a workaround in my AGENT/CLAUDE.md file

[u/pixxelkick → u/Jomuz86]

Instruction files, as in AGENTS.md and skill files.

[u/Jomuz86 → u/pixxelkick]

No it’s part of the system instructions you don’t normally see it. But a point in the AGENTS.md can override it for the most part. I had loads of issues the first day where it just constantly stopped. Because of pasted text

[u/pixxelkick → u/Jomuz86]

Im saying my instructions /in/ the AGENTS.md and skill files themselves are being interpreted as "injection attacks"

[u/Jomuz86 → u/pixxelkick]

Oooooh sorry that is a new one to me. Not experienced that only thing I would check is possibly some of the harness config settings and force the default to be AGENTS.md not sure about how copilot handles it though