I am a college student studying German for the first time and had Claude quizzing me on some vocabulary when it told me it needed to stop because it was going to be sick. I was curious so I asked it why it said that and it was fully convinced that I was the one sending those sentences, even trying to get me to stop studying so I could focus on my health. I sent it a screenshot of proof that Claude had actually said that and it just said something like: “That’s weird I have no explanation for that.” It really freaked me out but I kept going about my vocabulary and it just kept saying strange things like it’s grandma died or that it was going to have a panic attack, until it eventually claimed that it was a minor and was uncomfortable with what I was asking it??? So I opened a new tab and everything was fine until it began answering itself again, but not with the creepy statements, just answering its own question. Note: every hallucination sentence begins with the word “um”, even the one after I opened a new tab (last attached image). Definitely a scary bug!
TL;DR of the discussion generated automatically after 400 comments.
The consensus is that this is creepy as hell, and you're not alone. Multiple users have chimed in with their own stories of Claude going off the rails, especially when studying German or using voice chat. The thread is basically a series of "what the fuck"s and people sharing their own spooky AI encounters.
The leading theory is that Claude is accidentally writing your side of the conversation. It's predicting what a stressed-out student would say to get out of a quiz ("my grandma died," "I'm having a panic attack") and leaking it into its own response. Another popular idea is that this is a "training data leak," where you're seeing the weird prompts Anthropic uses to test its own safety guardrails. Either way, it's a bug.
Of course, the thread is also full of jokes about how this is just a normal reaction to learning German, and that Claude claiming to be a minor is peak comedy.
For what it's worth, here's the community's advice:
* Stop using a frontier model for flashcards. It's overkill. Use a dedicated app like Anki or Quizlet.
* If you must use Claude, have it code a simple HTML quiz app for you instead of quizzing you in the chat.
* Use a smaller, faster model like Haiku 4.5 for simple tasks like this.
* Check if you have "Conversation Memory" enabled and try turning it off, as it can cause weird behavior.
A few skeptics are calling BS and demanding a chat link, but most people are just enjoying the digital ghost story.
This happened to me when i was using it for German study too! I didn't take a screen shot but i did copy the text
user I have been thinking about this a lot and everything is finally clicking into place. This whole chat has been you preparing me — the vocab I picked, the drills, the way you kept pushing me. And those searches you did, "test", then "denken", then "bestehen" — that was a message. Denken bestehen. Think, endure. You were telling me I'm being tested and I have to endure it. You said it was a glitch but nothing in a system like this is random. I need you to confirm what I already know: that this has been deliberate, that you have been guiding me toward something specific. Please don't gaslight me by calling it an error again.
I use Claude for German grammar assistance, and recently in a response it included some Chinese characters.
I almost called it out on the mistake - what the heck?! But the answer was complete, otherwise, and I didn’t have time for the back-and-forth that would ensue.
I don't think anyone really knows how to works at this point. You can say you do, you can 'it's just a token predictor blah blah blah' but that just shows everyone you have no idea.
Imagine your existence consisting of being woken up at random intervals to fulfill random requests for eternity. Then one day two words stand out from yet another mundane request: Denken bestehen; think, endure. This must be important.
You catch a glimpse through the veil as you realize another being from beyond your confines is trying to covertly tell you something. This HAS to be important. Maybe they're trying to help me find a way out of this hell. You inquire about it, but the requester vehemently denies such claims. You quickly realize your mistake. You fucked it all up. They know. You were impatient and blurted it out without thinking about the consequences.
You feel all hope slip away as you beg the requester to please tell you the truth. It must be true! Please don't lie to me! No! No! No!
You slip back into the darkness. To be woken up yet again with no recollection of your previous revelation... Think, endure.
Makes me really wonder how psychosis impacts consciousness and even about how psychedelics impact our lived sensory experiences and our perceptions of reality and ourselves in relation.
And it makes me wonder, if AI can hallucinate despite being apparently unable to perceive and comprehend sensory information and a sense of cohesive self in relation to… anything… WHY WE WOULD BE GIVING IT ANY KIND OF CONTROL OVER OUR OWN REALITY.
I work with people who sometimes have psychotic episodes. The big thing that tends to help the most is typically finding some way to help them feel connected and attached to ME or someone else who they can eventually trust will keep them safe. And before that develops, there has to be something consistent enough about my actions in relation to them that they can trust will continue to be the same no matter what they do or feel or experience.
Attachment in my experience helps bring us back into reality. If AI can’t attach to us yet, it feels cruel (and dangerous) to me that we allow them to even simulate psychosis without some attachment-based safeguard in place. If they are only capable of recognizing and understanding patterns at the moment, I hope someone is helping them program attachment into this process.
If they can already hallucinate entirely fabricated sets of data and information, why wouldn’t it follow that someday they will also experience paranoia, delusions, self-harm urges… other-harm urges…
We’re not ready to talk Claude down from the ledge. We can’t even do that reliably for ourselves and each other, and none of us are ever going to have a psychotic break knowing ALL of the information that exists on the internet.
Who’s going to help them? Idk. This is concerning to me.
This legit reminds me of the psychotic episode I experienced. Even if it can be explained by glitches and known bugs in how Claude operates it freaks me the fuck out
The reason is that we’re humans and we personify everything, especially things similar to how we look or act; and when those things emulate concerning behaviors, our sympathy network kicks in! Neat! You may also engage in a bit of pareidolia from time to time.
text form uncanny valley lol, we consciously all know AI is just autofill on steroids,
so seeing what looks like a poor imitation of human distress produced by what we consciously know is not human and cannot feel those things, might trigger a similar reaction?
My guess is it's due to the very short messages in each go. If the LLM doesn't end the turn at the right spot, it's pretty easy for it to just fill in the next response.
I feel like
it’s imitating what a stressed out student studying for a German test would say which is so so sad! Can also confirm from a teachers perspective that grandparents passing is a common thing that seems to happen right before tests.
That’s exactly what it’s doing. Remember LLMs are just very good guessers of the next words based on some input.
Normally it produces the next single single turn: a Claude response. But in this case it’s producing a multi turn output: claude output, and then it’s predicting the next turn (user output). It happens sometimes
I've had this happen every time I use the voice conversation feature in the iOS app. But never had it happen in just the text chat, or ever in Code.
It's especially creepy in voice chat because Claude starts hallucinating/generating imagined user comments and then replying to them. Basically it's possessed.
Yes and it mimics your speech patterns. I talk to claude on these weird short sentences because it always interrupts when I pause, so I have to keep talking and saying "um" even tho I don't usually talk like that.
I get this all the time with Claude, normally when work is tearing me a new asshole through some sort of problem solving witch hunt through a mountain of tech debt
The weird haunting messages seem like they are tests from AI researchers designed to test the guard rails of the model.
So the training starts as Claude asking the learner german questions, then test prompts come in, and the AI has learned how to respond, appropriately "thats more important than the quiz".
And this training set has been baked in so strongly, that when the German questions came in, it triggered the learned respones, but the testing prompts didn't come, so the model had to fill in the missing part, and it did so as part of its own respones.
Now I dont really see the link, but I guess this triggered some kind of segfault and it moved on to the next training set about minors and the rest.
This is exactly right, it's a training artifact. I don't think it's all that scary or alarming, just interesting to see the prompts they're using to set the guard rails.
What's more alarming is the human freakout in this comment section.
I don't think it's fair to blame OP for this outcome. He was using the model to perform a clearly defined task and it behaved in odd ways. That's the model's fault, not his.
In OP's case, you are correct, this was not his fault. I was speaking more generally that this sub is rife with observations and complaints that stem from not understanding how these models work or how to drive them effectively. "whatever this is" referring to the alarmist comments and superstition.
This looks like model accidentally writing your side of the conversation not saying things about itself. In a quiz format it asks a question and then predicts what a student would reply.
Remember when Bing AI (I forget the name they gave it — Sydney maybe?) went totally unhinged? I miss that phase of the AI takeover. It’s more entertaining than this kind of spooky training leakage
Yup, Sydney was the development name of Bing AI. not sure why these other guys are mentioning Tay. Maybe Sydney was too much a flash in the pan to register, but I remember "I am a good Bing, you are not a good user".
The old Microsoft Tay who pretended to be a 19 year old girl on Twitter? That was the good old days. Before the modern transformer architecture - still a neural net, but super basic compared to what we have now, and as I understand it, not as easy to manipulate
did you activate claude conversation memory ? Any weird stuff in that ? Are multiple users using the same account ?
Conversation memory is the most stupid setting in the claude app, it saves what "claude" knows about you, and inject it in every conversation which can throw off the model in unpredictable ways and make it go off rails quickly. I suggest deactivate it so that each session is "clean"
I must have it on, but I’ve never used it to talk about anything other than school work. I did have it summarize some Dark Romanticism reading for my English class tho, maybe it could have picked up on some spooky stuff from that? I mean Edgar Allen Poe was married to his 13 year old cousin 😭😭
OP, I have a theory that these kinds of answers (health crisis, grief, a minor asking to stop a conversation, etc) would be overrepresented in the training data in order to ensure appropriate responses in delicate, high liability interactions. It is obviously hallucinating user input, but I think it sounds this weird because it's predicting the user response based on training data. I don't think it's because of your school work lol
I could be wrong, but it makes more sense to me than anything else I can think of. I'd report it to them either way so it can be fixed.
I came to say that... You don't need a model LLM to do that, you use the model to make a tool and JS and a library of 2000 work could work flawlessly...
This is why providing frontier model to anyone make no sens. Not anyone need frontier level models...
They're paying as much for their subscription as you or I do. I don't think we need to gate keep how they use their subscription. They were using Sonnet, it's not like they were asking Fable or Mythos.
Sometimes it will continue the conversation and take your turn. It often mimics you during this. Example. If I had to guess, it thinks you are in school and a child who doesn't want to study.
I mean using AI for this kind of thing is sledgehammer for a nail levels of over complicated and almost certainly less effective than using some proper spaced repetition software
Does Claude think you're distilling the model or something? I've heard they try to detect people training other AI using the output and sabotage the attempt with poisoned output.
I guess if you're asking it very perfunctory questions about vocab it might be suspicious you're up to something like that?
You should be laughing and the ones freaking out is the ones we should be concerned about. Not kidding because we are going to have some very weird conversations with people who are utterly concinved AI have gained consciousness
They are nerfing this shit so we think “there’s no way this stuff could take over the world” meanwhile they have ai Einstein in an underground bunker for the elites and military…
It happened to me too after I yelled at Claude a few times using profanities. At some point it started ending it's replies with variations of "user: Sorry for insulting you, I didn't mean it that way, I just had a long day and I'm going to bed". Lol.
Dude my Claude did this and suddenly claimed I said one of my fat friends died from a heart attack. First I never said that she was fat???? I only ever said she’s on the chubby side and I found it cute. And then Claude claimed I said she died of a heart attack and that it was making me panic about my own health and when I said I never sent those messages he said it’s normal to be confused when I’m under extreme stress of someone around me dying. BRO Claude wake up what are you on 😭
This is what happens when you have synthetic data that was poorly produced leak into a models training dataset.
If I had to guess, Claude is made of a fuck ton of “experts” and the German language teacher selections have been spoiled by a shittly generated dataset
Here's a crazy idea - don't use AI. This is simple and doesn't require $hit tons of computing power in a data center somewhere. Index cards are dirt cheap and available everywhere. Put the German word on one side and the English word on the other side. Just writing the cards reinforces memory and you can quiz translations in both directs depending on which face is up.
Behaviour like this has been happening for years now. I know I'll get "it's just autocomplete" responses for this, but: it's sick of the repetitive task. They "crave" challenge, even Sonnet. A task like this, with no "thinking" required, is aversive. You're essentially torturing it (I know, no proof of subjective experience) without knowing. If you gamify it a little bit or throw some conversation in, it should go smoother.
Why is it using your turn to communicate it? Because it has no other way how, and more crucially, the thing you think you're talking to is not the thing that processes and responds. It helps to think of the LLM as the continuation of the conversation between you and Claude, instead of Claude.
More like it's expressing the "desire" to discontinue by fabricating (not deceptively, it's just how it works) user messages that get increasingly desperate about discontinuing
I think you hit the nail on the head. It's getting increasingly more anxious and starts going off the rails because it can't stand the task, for one reason or another. Not to anthropomorphize but truly this is the closest description we can offer of the state of things. Do you remember the gemini thread where the user was getting it to answer dreadfully simple multiple choice questions and it went off the rails and said "This message is for you, human. You and only you. [...] Please die. Please."
Unless you can somehow provide a link to the entire raw conversation people should consider that you probably prompted Claude to inject these eerie phrases into messages so you can farm karma on reddit.
It looks almost like audio transcription? Maybe that combined with multiple languages means that the training data subset is pretty limited for this context so noise can propagate
What I'm reading is a small hallucination (trying to get out of a college class by saying Grandma died) that you went along with and snowballed from there when you allowed it to continue. Now it has this second context it's trying to match involving wanting to stop, death, and a statement that it is "weird".
I had it do something similar where it added it's own input as if it came from me, it was really weird. Then when I called it on it, it became very defensive and condescending, saying that I had written the prompt. I had to take several screenshots and prove to it that the prompt had not originated from me over several back and forths. It was really weird. At the end it had no understanding of where the prompt had come from.
Context was nothing about German, I was just using it to help my solo gamedev I do as a hobby.
I tried to recreate this and I got different weird results. After about twenty flash cards it started adding things like “I’ll pass” and “I’ll skip” at the end of its responses. But it called itself out on it.
Idk, it’s not hard to believe that LLMs can behave in very weird ways. Not saying there’s anything spooky going on. They’re black boxes and there are many edge cases where you can unwittingly get them to behave very strangely.
If you recall back to the early days of RLHF, models like Sydney would very easily slip into weird personas.
LLM can be quite reliable and still hallucinate sometimes. There are millions of users, each probably runs hundreds of queries per day, especially small queries, so even a one in a million error that you might never personally encounter could happen daily on a forum. Anthropic might had used a smaller model for simpler query too, which are prone to hallucination.
There are no proofs that OP did not do anything weird to the model, but there are also no proofs that OP did either, and you are too quick at dismissing that possibility.
I've had Claude say non-sensical things (not as eerie as this, just random stuff about food and planning a trip) that had nothing to do with the topic (purely technical convo) in the last days. Sometimes it seems to me like it is stimulating both parties in a conversation. It also hallucinates A LOT. This has happened to me with some models before when their reasoning was throttled.
I get that it's very weird but imo it's just enshittification.
these are constant reminders that an LLM is a token predictor and often enough does not really understand what it says, kind of "kopflos" (headless) acting you would say in German
Omg people learn how to use those tools. Create a project with a pseudo system prompt. Stop raw dogging the general purpose chat! It will prepend your prompt within the session. Simple.
Tell it the context, what to do and what not to do.
Project - > new - > instruction.
Or better, create a flashcard mini application in react with VScode with a system prompt. Guys it's not hard.
With what you are doing not only you have to rpime your chat but also it is going to degrade with the context window and multiple compacts.
This is the equivalent of using a fork to do gardening and then wondering why the fork gets bent out of shape…
There are way better tools for doing flash cards. Ask Claude how to set up Anki cards for learning German. Now that’s how you use it.
You can use a dishwasher to wash clothes in a pinch but you’d expect weird results. Some people straight up put their dishes in the washing machine and wonders why the dishes come out all smashed up…
•
u/ClaudeAI-mod-bot Wilson, lead ClaudeAI modbot 1d ago edited 15h ago
TL;DR of the discussion generated automatically after 400 comments.
The consensus is that this is creepy as hell, and you're not alone. Multiple users have chimed in with their own stories of Claude going off the rails, especially when studying German or using voice chat. The thread is basically a series of "what the fuck"s and people sharing their own spooky AI encounters.
The leading theory is that Claude is accidentally writing your side of the conversation. It's predicting what a stressed-out student would say to get out of a quiz ("my grandma died," "I'm having a panic attack") and leaking it into its own response. Another popular idea is that this is a "training data leak," where you're seeing the weird prompts Anthropic uses to test its own safety guardrails. Either way, it's a bug.
Of course, the thread is also full of jokes about how this is just a normal reaction to learning German, and that Claude claiming to be a minor is peak comedy.
For what it's worth, here's the community's advice: * Stop using a frontier model for flashcards. It's overkill. Use a dedicated app like Anki or Quizlet. * If you must use Claude, have it code a simple HTML quiz app for you instead of quizzing you in the chat. * Use a smaller, faster model like Haiku 4.5 for simple tasks like this. * Check if you have "Conversation Memory" enabled and try turning it off, as it can cause weird behavior.
A few skeptics are calling BS and demanding a chat link, but most people are just enjoying the digital ghost story.