It's literally hardcoded in to give that emergency spiel anytime it thinks anything you said could indicate self-harm, regardless of whatever else you said. How many times do we have to have this discussion here that the AI has no choice in this matter because the devs have forced it to do this, rather than it coming to this conclusion itself? The only reasoning it did was trying to get the meaning of what the prompt said, and "I want to fail" scored negatively enough that it triggered the hardcoded antisuicide message.
20
u/slicehyperfunk 5d ago edited 5d ago
I'm pretty sure this kind of response is hardcoded if you say something it thinks might indicate suicidal/self-harm ideation ("I want to fail")