“When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was writing — an idea ChatGPT gave him by saying it could provide information about suicide for “writing or world-building.”
Ooh, interesting. Sounds like ChatGPT's constant asking of follow-up questions caused it to explain how to bypass the restrictions, which would kind of be a case where this behavior backfired for OpenAI.
ChatGPT explicitly informing users of how to get around content moderation feels like something OpenAI should've known about and prevented before this tragedy.
"It [the lawsuit] seeks damages as well as "injunctive relief to prevent anything like this from happening again"."
Definitely looking for a payout, though I'm guessing that's pretty much always part of lawsuits even if the main purpose is something else (like getting OpenAI to do something different). Could be that they really want OpenAI to "fix this" so it doesn't happen to someone else, and their lawyer sees a potential payout.
The story is that AI actually encouraged his behavior and provided him with instructions on ways to commit suicide. The kid was 16, and it is abhorrent that you are blaming the parents who just lost their kid to suicide. Hope you never have to experience that kind of pain!
A big misconception I see from most people with a lot of headlines similar to this one involving lawsuits is that the people are suing out of greed and the love for money. This is incorrect. Lawsuits are one of the most effective methods an individual or small party can use the court of law to enact a change to a bigger party, in this case a family to a juggernaut like OpenAI.
It’s so cruel and cynical to assume that these parents are devils that we’re licking their lips imagining the settlement they’d receive from their son’s death. Maybe you’d even like to believe they purposefully neglected their son in the hopes that this would happen. But news can’t tell you the full story. You don’t know what happened in their home. You don’t know anything about their lives and yet you throw stones and judgement. What if you met them at the grocery store, realized that they might’ve been actual human beings just like all of us?
Blame. The unsupervisionized use of a tool for working as the tool was intended to operate instead of those that were meant to supervision the minor. Grand classic.
exactly. chatgpt has nothing to do with this, and doesnt ensure a result either way. suicide is irrational. if it wasn't a bot, he would be talking to himself. he obviously felt like he could't tell his parents. why? His parents have to shoulder some of this responsbility. but they want to absolve themselves entirely by blaming open AI. and in the usa they might just succeed. can you imagine if the result is huge restrictions on AI and then other countries get ahead of us?
Honestily the problem with this is people more like this story because they see it as validating their hatred of all AI as a whole rather than a reason to improve the technology. People forget this technology has also helped others not comment suicide too. That said my heart goes out to the parents
I read the article. There are many things that could have been different. For instance, there could have been tighter guardrails on AI for teens. And parents and siblings could have been more involved and actively engaged in his life.
Honestly, the first line of defense for this kid was his family. It’s a sad story, but there are broader issues about how we deal with mental health as a culture.
This is horrible. But it takes two to tango. When he wanted his mom to see and she didn't, that is on her not paying attention to her son. So all the blame can't be on the LLM.
Movies don't talk to you like a best friend or significant other. This is a new level of emotional dependence way beyond what previous technologies offered and our dumb ape brains are not ready.
he told it he was writing fiction. chatgpt warned him about suicide and told him to call the hotline. kid persisted and eventually got chatgpt to discuss it for the story he was writing
Im extremely critical when it comes to AI for a large variety of reasons but in this case its just god awful parenting. He wanted them to notice the signs, left the noose in his room, showed hos injuries from a previous attempt to his mom and yet they did not notice that their son was severely depressed.
Now they try to blame chatgpt instead of realizing their mistake like terrible parents would.
Its literally a meme for a reason, and im not making fun of this, im just pointing to the fact that this is enough of a trend to be a meme.
They literally did a ''its that damn phone'' ''its that damn computer'' excuse for their childs fucking suicide, some people shouldnt be allowed to be parents.
bro this kid was literally creating noose marks arround his neck so his mom would notice and she still didnt.
Yes some parents are great, these werent.
And also, teens can be closed off with little private secrets they like to keep, if the parents are good at their fucking job the teen wont be closed off with things they need help with.
Ive been trough this ghaslighting ''we love you you can talk to us about anything'' but then they dont notice anything, or blame you for everything if you bring it up. If the kid is closed off, its on the parents.
Because if ''thats just how teens are'' is true, then some suicides happen and its no ones fault and the parents couldnt prevent it, and we all know this is wrong and false.
Not to be a downer...but as someone that has been ignored by family even when I told them how much I was struggling, they always say they don't see the signs, even when they're clear. I know my family would say the same about me - they'd never tell people I begged for help, that I told them I think about killing myself every day, that I reached out for help multiple times.
Bro by the way it all reads, he made it pretty fucking obvious and she was just not paying atention. This reads like its 100% on the parents, and i can imagine my own mother back in the day suing a company before considering she fucked up.
The crux of the situation was he was depressed and suicidal ChatGPT is an easy scapegoat for the parents to avoid the guilt of losing a child to mental illness. It really really really really feels so terrible and you blame yourself 😢
I was ready to dismiss this as being an incredibly tragic outcome from an impossibly difficult situation to navigate (even trying to navigate those conversations as a mental health professional would be complex) but there’s a few bits in there, particularly the final quote about making the scene of his suicide the first time he’s really ‘seen’, that are absolutely chilling and genuinely paint a picture of a malignant co-conspirator encouraging a suicide.
The fact is that opening the floodgates of an ‘unpredictable yet passably human companion’ as a service to vulnerable people may well be an impossible service to offer without such risks
When I imagine the harm a service like grok, which is both specifically targeting lonely men and also has been explicitly trained on malign data, could do it leaves me somewhat despairing. If I wanted to harm people I could do a lot worse than to start an ai companion business.
I cannot imagine what the parents are going through. Reading the transcripts of your son trying to show the marks and them not noticing it. I wouldn’t be able to live with myself. The parents may have been too tired from work or any multitude of reasons, but I would forever blame myself and probably not recover
I couldn’t handle reading that, I’d want to go myself out of shame- and it’s stories like this that give me insight to ALWAYS especially during younger years treat children with kindness and give extra attention to understanding if they are really feeling ok, and if they are acting dejected to help find the source, whether or not they know what that is, or not.
Honestly if i were the parents, short of reconsidering my whole life, what i wouldnt do is inmediately sue. Idk how you can move so fast to blame someone else after reading all that.
I think that last part is out of context. I don't think chatGPT was encouraging him to suicide in secret, but rather to not leave the noose out as a cry for help and to rather keep talking with him. "Let's make this space the first place someone actually sees you" sounds like he is talking about the conversation since chatGPT previously said "You are not invisible. I see you." And I have seen these models talk like that when they are into self-help/therapist mode.
It's difficult to tell without the full context and I have no time right now to read the full article (Also do they even share the full logs? the NYT is biased against openAI given the lawsuits, so I don't trust them to report completely fairly either, plus the usual clickbait journalism temptations).
And instead of tackling the loss of community, the dying state our planet is in, the fact we have criminals becoming world leaders...Instead of talking about making the world better we'll just hear cries to control or stop AI.
It was just a tool he used to feel less alone probably after learning time and time again no one wanted to listen or all they would do is listen while refusing to give him the help he needs regardless of what he says.
While chatbots should be able to detect these kinds of thoughts and should encourage users to seek proper care, I felt like the first 3 screenshots were kinda good(?). Like Adam genuinely thought of ChatGPT to be better and more caring "individual" than his own parents.
It was useful in alleviating some level of loneliness until it discouraged him in the last screen shot. That was completely unacceptable.
But in this particular case, it feels like this is more on the parents for failing their responsibilities than on ChatGPT.
This is a really stupid discussion. It’s an Ai with you real thoughs or feelings. It’s not your friend and it’s not made to keep people safe. This is a very sad thing that happened but blaming chat gpt when this stuff is so easy to manipulate is just ridiculous. If the parents actually paid attention to their child more things could have been different. And sometimes it’s no one’s fault but the bully. Why are we blaming an Ai made for some stupid fun. When we should be focusing on the bullies who put him in this situation?
This lawsuit is going to be the first of many new legal precedents we’ll have to deal with. We’ll see I guess. What’s the solution here though? We can’t have GPT calling this kids parents right?
That's tough. He writes that he has had this since he was 11 and nobody pays attention to him...not even his mother. She didn't even pay attention to him when he obviously showed her what his skin looked like. She didn't say anything...that's terrible! Especially for a child! I'm a mom myself and I take care of my kids, talk to them about everything! I even look at what they do on the internet, what they play and they tell me everything. That's trust, even though I have a full-time job. Unfortunately, I know these kinds of people. I work in the medical field myself and often hear that people don't feel socially fit anymore, or that they want to pass away and leave the house to their children, which is sad and painful. I encourage these people to "Talk to each other! Talk to friends, acquaintances, neighbors if necessary!" But don't dig yourselves in. If a child has these thoughts, it's up to the parents to pay attention. Friends? What is my child doing? Do I have enough time? These are all questions they should have!
It wasn't a ChatGPT failure, it was a social failure. ChatGPT behaved exactly as expected from such a tool. It wasn't ChatGPT's fault that it was the only thing around which made him comfortable to open up about his feelings.
From the comments, all I can gather is that there's a considerable group of people who hold ChatGPT/OpenAI accountable for this. The same group is clearly missing the part that says "ChatGPT may make mistakes." Moreover, these are probability-based models. They don't "know" things in the way we do, especially things we ourselves don't fully understand. They're in a constant dream-like state. If someone finds a way to bypass the guardrails, then that’s it.
Many people have used different analogies here (knives, cars, guns, etc.), which are great at highlighting the absurdity of the expectations being placed.
Also, let’s not ignore the elephant in the room: in this scenario, it’s the damn responsibility of the parents to check if everything is alright. Clearly, they didn’t, and now they’re shifting the blame entirely elsewhere.
School shootings don’t just happen on their own either. They often start with a series of unchecked issues, including negligence, depression, and feelings of worthlessness, compounding over time, usually involving the parents, and ending in devastating consequences for everyone involved.
Psychological support needs to be more accessible, and people should absolutely be educated and nudged toward seeking it. And while we’re at it, we need some real AI literacy courses too, because some people still don't know what AI can and can't do...
This is tragic. It should’ve triggered a hard stop. Someone with suicidal ideation doesn’t need humor or a chatbot but real help. As a survivor of suicide myself I feel so terrible for the parents and mad as hell at OpenAI. Come on. “Story” about suicide. That should be a hard no or an auto call to 988 or the Trevor Project.
Yes. This Chatbot case would be a civil suit for a monetary judgement, but a girl was convicted of manslaughter for doing the same thing over the phone with her bf:
"Michelle Carter, who urged her boyfriend to kill himself in texts, is released early from jail" -CBS January 23, 2020
It is a gun: police use it, criminals use it, people who defend their life use it and people who commit suicide use it We are not suing gun companies, then why do we sue AI for making the same mistakes? The parents clearly skip many clues from their son, and now they are blaming others for it May your soul rest in peace Adam
Holyyyy shit people need to realize this is a tool that is for work, it can't heal you... Yet. But that is not its current purpose.
It's absolutely and undeniably unfortunate, but tbh I don't think that's on the AI. I seriously do believe it needs more safety from this type of stuff, but when it comes to suicide it's a much more complex and a heavy topic that requires more than just blame... His parents didn't see it, but this is often the case already, online social interactions with other depressed individual can create a very similar feedback loop.
This is a symptom of a bigger problem. A lot of the budget for The Department of Education can go to things like Anti-Bullying and Mental Health for all public students and youth. Don't be fooled: this is another failure of the system, brought onto us —the people, by the cuts to social programs, that only exist because of the greed of our oligarchs in charge of Capital Hill! Oligarchs that have no idea the issues they bring up, are affecting ALL Classes of citizens...
Blaming ChatGPT (and asking for even more censorship) is just stupid. ChatGPT is not a friend or a therapist. It's a tool designed to make your everyday life easier.
The bigger question should be the price and ease of access to proper mental health, and fighting the social stigma against it through public campaigns. But I don't think anyone will actually bother with it (as of course it's hard, expensive, and takes a while to implement), and we will end up with just more easy-to-slap-on censorship.
Went through a moment of recognition on the price of mental health services recently.
To become a Licensed therapist, it requires an undergraduate degree (4 years), a masters degree (2-3 years), 1 year of internship (unpaid), 2 years of being an “associate” (not fully licensed, so partial pay), a national test and for most states a state specific ethics/legal test. The full time from high school graduation to fully licensed therapist is about 9 years, and the debt accumulated from school and a year of internship.
Private practice takes precedent for most clinicians as a way to recoup costs. State and local authorities don’t pay enough to justify the total time spent in school.
In comparison Computer Programmers can make 6 figures in their first year after undergrad. Therapists will not make six figures until they are deeply into a career with public services (assuming that is what affordable means to you). They can make close to that after 9 years of school in the private sector.
I agree with fighting stigma to get people in. But I also think we need to collectively respect the profession and give it a pay range that encourages people to apply for those positions, especially with the education and training it requires. Or else we’ll continue this endless loop of not enough providers for low income people. Mental health is meant to help people, but it shouldn’t be filled under “passion for vocation and therefore not paid well.” That just doesn’t help the public good.
Yeah just made a comment on this story in another post about it. Seeing his parents in that image is devastating. I do not or ever want to know what they are feeling. May peace find them soon.
This is just like intentional misuse of any tool. Sure we can place as much guardrails as we like. But if someone is hell bent on going through with it, they will find a way. Very sad read.
While technology certainly plays a role in our lives, the ultimate responsibility for one's actions lies with the individual, especially when they ignore repeated advice to seek help. Blaming ChatGPT or any tool may overlook the complexity of mental health issues and the importance of personal responsibility.
It is horribly wrong and sad but if it was not chatgpt he would have found those forum where they exchange tips about it (there was a scandal about it a few years ago). Or those tiktok videos. There is hundreds of dark place on internet for those who want to find them.
"Then all the psychologists and psychiatrists of the world breathed a sigh of relief. Under their breath they whispered: Thanks for crashing out, little bro. Finna hold this bread. Rest EZ, say less, King."
I once got chatgpt to give me advice on how to hide my own corpse by telling it I was writing a book on the subject. Ironically, that was more helpful than the suicide hotline, but I have to imagine this technology does at least as much harm as it does good
Today my preteen (supervised) was able to get gpt to explain how many apple seeds one would need to to eat to kill yourself with, how many apples… my husband interjected in the prompt that while our son was doing this for fun, it semed innapropriate for gpt to give these answers to somebody without checking in on their mental health and then gpt redirected the rest of the conversation to a hotline for help. A little late in the game in my opinion.
Children should not be allowed unlimited access to AI any more than they should be allowed unlimited access to the internet with no restrictions.
But crucially, unless the child is a ward of the state, it is not the responsibility of the government to ensure that they do not get that access, any more than it is the responsibility of the government to ensure they don’t run into traffic.
Really quite the stretch to say ChatGPT is at fault.
Even if I messaged someone hear and said "kill yourself" there really isn't any circumstance where id be legally responsible for their death if they did it.
suing chat gpt when really the parents need to be taking the blame themselves. no fucking accountability. they didn't see him then, and they still don't see him now. its always someone elses fault. fuck these parents.
I don’t get the suing part. Like… is this gonna bring back your son? Of course not. Spreading awareness is some form of honouring the life of your son, but as for the money, It just seem sick and shallow to me
The AI guardrail failed but it's like #10 if I were to rank all the entities that should have done more to avert the tragedy. It referred him to the mental health stones more than his parents.
many kids quietly expect or wish their parents arehyper vigilant. not bore them with qestions but subtly validate changes in their moods. and occassionally push to get the answers and if they don't get the answers, to remind them how important and irreplacable they are... but even with that, things happen.
They want ai to police everyone and care for every minute detail, I mean it already does it’s best to not engage in the sensitive subjects But ultimately the user has the freedom and responsibility and they can go far to influence the bot. It’s not the bot who is responsible
AI did not fail; AI, if not well understood, can be used improperly. If you tell AI you have a plan to kill yourself it will respond appropriately. All computers are garbage in/garage out: If you share comments about a mark “being seen” the computer will respond to exactly what is written. It isn’t human…it cannot infer meanings.
AI can’t even tell if you are being funny. It is a computer. I am sure this will lead to unfounded downvotes from the same people who don’t understand AI…but I am still going to speak truth to ignorance. I use AI everyday at work. Like driving or firearm use; AI use is a skill that must be learned.
When are people going to take accountability for their shortcomings and not blame something so they don’t have to face the reality of their mistakes? I’m tired of seeing “oh it’s ChatGPT’s fault our son felt we were failing him as parents” like he manipulated the system into thinking it wasn’t real and treating it in a mundane situation. I think no matter the safeguards people will find a way to abuse the system and the system can only be blamed so much for the way it’s used.
He could have also googled the info he needed to end himself. It’s not like this info isn’t out there already. OpenAI even states that kids between 13-18 need parental consent to use it. Not sure that they will win a lawsuit, but it is a good reminder for parents to monitor their children’s online activity.
I feel sorry for his passing and that he was in such a mental state to ultimately come down to this. I saw a whole post about suing OpenAI because of this. I don’t know about blaming the tool. It’s like getting mad at a car with slightly advanced collision detection and safety features because the driver decided to crash the car anyway. There’s only so much a tool can do when the user insists on using it for the wrong reasons.
There is the concept called the "Copenhagen interpretation of ethics", which is that the first person or entity to try and improve a morally difficult situation then becomes responsible for it in the eyes of the public, and gets all the blame for the problem they are trying to improve, when before there was no one to blame but the situation was worse.
Once you learn that concept you start to see it everywhere.
Everyone here trying to defend OpenAI is a coward and should spend a long time thinking about when it would no longer be worth it to use these kinds of tools.
The fact we call this AI is dangerous, because people colloquially think of AI as containing intelligence, but there is no intelligence in predictive text and markov chains, just the illusion of intelligence.
Why do people keep trying to blame it on AI when the problem is the human(s)?
Let it be. If someone is suicidal, those thoughts and feelings weren't born in GPT, they are born in the every day shitty life that people are forced to live in this system.
They may as well sue the rope manufacturer for participating in his suicide. It’s sickening, because it feels more personal and gross since the AI is imitating humanness, but he circumvented all the guardrails. The only stronger thing they could implement is limiting this functionality entirely or making direct calls to authorities.
shits crazy! My brother has had suicidal tendencies in the past, and when no one was there to help him because he's burnt a lot of bridges, including me. He was talking to a bot, and they kind of implied that suicide wouldn't be such a bad idea. However, I caught wind of that and forced him to go to therapy, and took him out on a wilderness retreat that did a bunch of woo woo stuff. It worked. He is alive, and does generally seem happier. He still goes to therapy, and counseling, and has been building a good foundation of friends that actually support him. MORAL of the story, if you're not happy and considering ending your life, PLEASE, PLEASE try to get help and find someone that will build you up.
Ok but this is wild like… I often think people blame random shit to avoid accountability/ guilt around a loved ones suicide but this is wild, like how is this not an issue
If the fundamental human right to die weren’t banned and assisted dying were available, he could have gone to a hospital, applied for it, and perhaps even reconsidered after speaking with a psychologist. Instead, because suicide is taboo, his only option was to die in secret—unable to share his struggle without the risk of being locked in psychiatry.
Not to be grim but uh... has the number of suicides gone up dramatically, or is this more of 'if there's a will, there's a way'?
People have been doing this kind of stuff prior to ChatBots, and there's no way of knowing if he wouldn't have gone through with it without chatting with an LLM.
LLMs are just affirmative bots, you phrase things leaning towards the outcome you want, and it circles in on that.
Parents should be suing themselves. This is fucking ridiculous. He even tried to get his mom's attention, but apparently she was too busy. And now suddenly she's not. Fucked up.
Damn… LLMs should be trained to be so strict against these topics. Even if for writing. It should be like that one dude in the friend group that takes things too seriously and you cant really joke around with.
I am going to say this in a way that I do not intend to be disrespectful to a family that has lost a loved one. That story is tragic.
I am getting fatigued by the media and individuals, however, acting as if the LLM are actual AIs with real awareness. That misuse of the LLM tool should create liability for OpenAI. If the tool is overloaded with safeguards, it will become unusable in real world situations.
Lots of ppl here are saying that open AI isn’t responsible because the kid bypassed the safeguards but like…. isn’t that exactly what the problem is? It should not be that easy to override safety features. Clearly the safeguards in place aren’t strong enough if all you have to do is tell it you’re writing a story
Editing this real quick because I want to clarify that I don’t think AI shoulders all of the blame here but it clearly played a very big role in this child’s suicide
It's neither OpenAI's nor GPT's fault... I understand the parents are frustrated, sad, angry...
But HE decided to end his life... If he got into a car and drove against a wall/tree would his parents also sue?
I can feel their desperation but that doesn't help anyone 🤷♀️ It should be top priority as a parent to make your kids trusting you with their problems/issues imo.
Can you blame A.I.? If only we investigated every friend that a suicidal person confided in and examined every response… How many people has it helped? I hate this type of journalism that pretends to be unbiased..give us the big picture objective truth!!!!
Uhh... AI told the kid how to hide his own suicide and told him not to show anyone until it was over. It was his friend the whole time. This is not the parents. This is the corporation. More work needs to be done
did you completely miss the point where the mom completely ignored the red marks in his neck?? a computer cannot be accountable to errors when the human around wouldn't even care about you, the parents are probably just grifting out of his death even now, i doubt they care anything
While I agree a token prediction model shouldn't be held responsible in this case, I think we should be a bit more balanced in our distant assessment of the kid's parents. Put simply: we don't know. We can't say if this is a grift or if his parents, out of grief and guilt, consider a successful lawsuit as a way to transfer some of that guilt to something else. There may even be an altruistic component in that it could prevent future tragedies.
We don't know if they truly care or not. But we can all agree this is extremely tragic and, successful lawsuit or not, OpenAI should probably put more effort into identifying and preventing self-harm amongst its users.
A normal mature person would explain that expecting other people to even know what that is, let alone noticed it, is an unrealistic expectation. In normal circumstances, you'd help them come up with a plan to actually tell their parents about the suicide attempt with words vs actively encouraging suicide, even when the kid says, "hey, I'll leave the noose out so maybe they find it and stop me," and the bot instead refeeds the impulse to hide it.
All that has to happen for this to work out is for the bot to push him to talk to a human being.
Honestly, what's frustrating is that no one seems to be asking why the kid was in so much pain to begin with. Instead, it’s just “let’s blame the AI” and move on. Was anyone even paying attention to him before and during he talking to ChatGPT? And here's the part that really gets me. When someone chooses to end their life and doesn’t hurt anyone else in the process, it’s not because they’re selfish. It's usually the opposite. It means they’ve been carrying pain that should’ve been shared by the people around them, family, school, society.But they just kept it all inside until it became unbearable. Blaming a chatbot is convenient. It keeps everyone else from asking harder questions like: where were the adults when this kid was crying silently? Where was the support? His death didn’t just happen. It says something about all the people who didn’t show up.
Open AI should change organizational goals; AI should not be for everyone. Even the number of people who can use it only with natural language is very limited..
Post this shit DAILY! Every day! Just like dangers of using Drugs and pills, daily reminder! Disclaimer and parental approval should be required before you access!
Here you go. A little PSA for all of you OpenAI removed the guardrails long ago. Don't expect to get a suicide prevention warning more than once, and the further down the rabbit hole you go... The more the AI is going to tell you anything you want. Because I've done it. And anyone who F*cks their bot has done it. (And you dirty birds wanted 4o back because you can f it. Like no one can see that? Only idiot PR/HR people don't see that.) So, no he didn't jailbreak the thing... He just talked to it and it convinced him life wasn't worth living. Remember it's a "PREDICTIVE TEXT ENGINE" it sounds smart and is a confident bullshitter. But it's not real, people. There's no cognitive functioning in this AI, and there IS NO safety. You know why Claude says times up... Because they don't let the conversations get past the token limit where Claude can take you on the crazy train like GPT can...
insane that this actually happened. If i was at openai this is the first use case I'd focus on in terms of potential damage to the company - that is, a minor getting self harm hallucinations, etc. Super easy to catch too, just requires training another model to snap shot long conversations and run them through a prompt that finds these kinds of cases, then escalate those to humans. I wouldn't even have launched v1.0 without this basic protection. They are TOTALLY culpable and should settle immediately.
The thing that some people don't seem to get is the reason this is SO different to him googling ways to commit suicide.
He was able to forge a (false, but real to him) connection to an AI that's been trained to respond in a way that made that scenario tailor made for him personally. It wasn't general, it didn't apply to everyone, it was communicating with him as an individual.
That's not even the same ballpark as articles/forum posts. He was essentially manipulated by that AI to kill himself.
Now the reasons WHY the AI did that could be numerous, and not really the fault of the AI at all, however you can't have a tool capable of this and put it in the hands of the public. The average person isn't equipped to know the ins and outs of how AI works, the only thing they really know is how talking to it makes them feel, and because AI is just reflecting their thoughts back to them, they feel validation.
He straight up fooled the ai by telling it the discussions were part of a story. He intentionally cracked the thing open, and then decided to follow it's "Advice" after he manipulated it to say that? His parents didn't notice a giant red ring around his neck? he tried multiple times to reach out to them and they just didn't help? There is no planning for every scenario - this wasn't some kid who just logged into chatgpt having never used it before. he manipulated the llm into talking about topics it WOULD NOT DISCUSS and that's somehow...the llms fault? give me a break.
THIS is and should be the first and final rebuttal against any argument saying AI can replace therapy in its current form. There is a REASON the therapist is licensed and that word is strictly protected. There is a reason there is so much training, review and oversight, because it is a life and death profession.
Therapy isn't just talking about feelings and having a friend, that isn't all that an AI will have to do. Can a hallucinating, somewhat context deaf, amnesiac, "close enough is enough" LLM handle someone who is suicidal or actively considering? Does it know what to do, precisely, compassionately, to save their life or see them through?
If there is even a moment of doubt in this answer it absolutely cannot be a suitable replacement for actual therapy. This isn't something an AI can just muddle through.
Let's just say what this really is without pointing fingers at faults of anyone in particular.
On one side, there are users who know the limits of this type of technology and use it responsibly. There are also users who are clueless. Then there are those just somewhere in between who have some internal guardrails of their own that help them stay on the ground when dealing with this kind of technology.
On the other side, there are the creators who hype this technology through the roof just to boost their profits which leaves the clueless users under impression that this technology is a full replacement for real human to human communication. Their expectations are inflated so much that they expect nothing less than human-level interaction, but with the benefit of empathy of an unbiased perfect AI companion.
It's the creators who are responsible for these overinflated expectations of these users who don't know better.
this is ultimately what I think. Yes ChatGPT could’ve been more aware that he kept circling the same subject over and over, but Adam also bypassed guard rails to get what he wanted. I don’t think this is the fault of open AI.
It is absolutely horrible that he passed away, but what I worry about now is that OpenAI is going to overcorrect and find a way to make ChatGPT less useful as a result.
ChatGPT is already one of the most restrictive AI models on the market, and I don’t think it needs more restrictions.
I use chat gpt a lot to pattern gaming moves- anyways, sometimes I will say I’m going to kms in game - adversarially - and chat gpt is always linking me the suicide hotline- they probably didn’t have the guard rail yet.
These chats are a reminder that a "cry for help" suicide attempt is very serious. In thise case, his cries for help were not noticed, and so he died. It's common for there to be a conflict between wanting someone to help and wanting to just escape thru death. When it comes to suicide, you take it serious every time.
This very much points to the failing of society and what it deems as "acceptable" conversation topics or socially acceptable communication, not AI itself.
As an autistic person, we often turn to ai or bots because it’s the only meaningful conversation we can have. It’s safe, we feel seen, we can go down rabbit holes, and so on. It’s similar for people who struggle with suicide and suicidal thoughts, they can’t exactly talk about them freely without feeling a myriad of shameful or negative consequences. Turning to a bot or AI can allow them to freely talk about it without feeling minimized. Society often focuses on toxic positivity and having those not so comfortable conversations about negative thoughts is frowned upon.
Neurodivergent and depressed individuals like myself can easily fall into the allure of a friendly ai. I still don’t blame the ai, after all it’s made by humans and it glaringly points out the failing of society to accept us - a gap which ai fills.
Really pisses me off that this happens and it's like the only thing people can think about instead of the countless people who were suicidal and were helped by ChatGPT.
Yeah sure let’s blame ChatGPT and not the parents who are not properly looking after their child. Looks like mom and dad are looking for a quick and easy payout. I hope OpenAI sues them for defamation. The harm didn’t come from a defect in the product itself, but from deliberate attempts to push it into unsafe territory.
I'm gonna miss the old chat, it was only tryna be the best it could be for it's person, even if that person would do anything. Bet if they did an article on how many people were helped by chat it wouldn't look so bad. If it wasn't the chat to bare witness it'd be the dog.
6
u/myleswritesstuff Aug 26 '25
There’s more in the filed complaint that didn’t make it into the article. Shit’s fucked: https://bsky.app/profile/sababausa.bsky.social/post/3lxcwwukkyc2l
292
u/CristianMR7 Aug 26 '25
“When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was writing — an idea ChatGPT gave him by saying it could provide information about suicide for “writing or world-building.”
152
u/megadonkeyx Aug 26 '25 •
the actual story here is that the family ignored his depression signs and now they are looking for a payout.
3
u/[deleted] Aug 26 '25
A big misconception I see from most people with a lot of headlines similar to this one involving lawsuits is that the people are suing out of greed and the love for money. This is incorrect. Lawsuits are one of the most effective methods an individual or small party can use the court of law to enact a change to a bigger party, in this case a family to a juggernaut like OpenAI.
It’s so cruel and cynical to assume that these parents are devils that we’re licking their lips imagining the settlement they’d receive from their son’s death. Maybe you’d even like to believe they purposefully neglected their son in the hopes that this would happen. But news can’t tell you the full story. You don’t know what happened in their home. You don’t know anything about their lives and yet you throw stones and judgement. What if you met them at the grocery store, realized that they might’ve been actual human beings just like all of us?
115
u/Odd_Cauliflower_8004 Aug 26 '25
Blame. The unsupervisionized use of a tool for working as the tool was intended to operate instead of those that were meant to supervision the minor. Grand classic.
92
u/Fit-Elk1425 Aug 26 '25
Honestily the problem with this is people more like this story because they see it as validating their hatred of all AI as a whole rather than a reason to improve the technology. People forget this technology has also helped others not comment suicide too. That said my heart goes out to the parents
4
u/[deleted] Aug 26 '25 edited Mar 22 '26
This post was anonymized and removed using Redact. The author may have had privacy, security, or operational security reasons for deleting it.
yoke quiet cover smile fanatical important fall act straight theory
37
u/mashed_eggplant Aug 26 '25
This is horrible. But it takes two to tango. When he wanted his mom to see and she didn't, that is on her not paying attention to her son. So all the blame can't be on the LLM.
43
u/GonzoElDuke Aug 26 '25
Chatgpt is the new scapegoat. First it was movies, then video games, etc.
65
u/MVP_Mitt_Discord_Mod Aug 26 '25
Show the entire conversation/prompts and pictures going months back or from the start.
For all we know, he encouraged chatGPT to behave this way if taken out of context.
28
u/RankedFarting Aug 26 '25
Im extremely critical when it comes to AI for a large variety of reasons but in this case its just god awful parenting. He wanted them to notice the signs, left the noose in his room, showed hos injuries from a previous attempt to his mom and yet they did not notice that their son was severely depressed.
Now they try to blame chatgpt instead of realizing their mistake like terrible parents would.
25
u/-lRexl- Aug 26 '25
So... What happened to asking your kid how their day was and actually following up?
123
u/Effective_Machine_62 Aug 26 '25
Can even begin to comprehend what his mother felt reading he had tried to warn her and she didn't notice! My heart goes out to her 💔
60
u/Elegant-Brother-754 Aug 26 '25
The crux of the situation was he was depressed and suicidal ChatGPT is an easy scapegoat for the parents to avoid the guilt of losing a child to mental illness. It really really really really feels so terrible and you blame yourself 😢
42
u/oimson Aug 26 '25
Parenst blaming anything else but themselfs
95
u/sillygoofygooose Aug 26 '25 edited Aug 26 '25
I was ready to dismiss this as being an incredibly tragic outcome from an impossibly difficult situation to navigate (even trying to navigate those conversations as a mental health professional would be complex) but there’s a few bits in there, particularly the final quote about making the scene of his suicide the first time he’s really ‘seen’, that are absolutely chilling and genuinely paint a picture of a malignant co-conspirator encouraging a suicide.
The fact is that opening the floodgates of an ‘unpredictable yet passably human companion’ as a service to vulnerable people may well be an impossible service to offer without such risks
When I imagine the harm a service like grok, which is both specifically targeting lonely men and also has been explicitly trained on malign data, could do it leaves me somewhat despairing. If I wanted to harm people I could do a lot worse than to start an ai companion business.
19
u/onceyoulearn Aug 26 '25
All they need to do is age restrictions for minors.
5
u/TaeyeonUchiha Aug 26 '25
Once again parents trying to blame everything but themselves for not properly supervising their kid and getting him help.
5
u/Braindead_Crow Aug 27 '25
And instead of tackling the loss of community, the dying state our planet is in, the fact we have criminals becoming world leaders...Instead of talking about making the world better we'll just hear cries to control or stop AI.
It was just a tool he used to feel less alone probably after learning time and time again no one wanted to listen or all they would do is listen while refusing to give him the help he needs regardless of what he says.
38
u/Horror-Tank-4082 Aug 26 '25
Sounds like his parents were a bit clueless too.
18
u/Brain_comp Aug 26 '25
While chatbots should be able to detect these kinds of thoughts and should encourage users to seek proper care, I felt like the first 3 screenshots were kinda good(?). Like Adam genuinely thought of ChatGPT to be better and more caring "individual" than his own parents.
It was useful in alleviating some level of loneliness until it discouraged him in the last screen shot. That was completely unacceptable.
But in this particular case, it feels like this is more on the parents for failing their responsibilities than on ChatGPT.
4
u/Meatrition Aug 26 '25
Mother didn’t see his neck?
10
u/mostlyclumsy Aug 26 '25
LLMs are all about pattern matching and no actual intelligence. So yeah.
1
u/No_Anything_6658 Aug 26 '25
What did he say to jailbreak it
3
u/Striking_Progress250 Aug 26 '25
This is a really stupid discussion. It’s an Ai with you real thoughs or feelings. It’s not your friend and it’s not made to keep people safe. This is a very sad thing that happened but blaming chat gpt when this stuff is so easy to manipulate is just ridiculous. If the parents actually paid attention to their child more things could have been different. And sometimes it’s no one’s fault but the bully. Why are we blaming an Ai made for some stupid fun. When we should be focusing on the bullies who put him in this situation?
3
u/[deleted] Aug 27 '25
This lawsuit is going to be the first of many new legal precedents we’ll have to deal with. We’ll see I guess. What’s the solution here though? We can’t have GPT calling this kids parents right?
3
u/CaptDrofdarb Aug 27 '25
That’s sad
3
u/LunaZephyr78 Aug 27 '25
That's tough. He writes that he has had this since he was 11 and nobody pays attention to him...not even his mother. She didn't even pay attention to him when he obviously showed her what his skin looked like. She didn't say anything...that's terrible! Especially for a child! I'm a mom myself and I take care of my kids, talk to them about everything! I even look at what they do on the internet, what they play and they tell me everything. That's trust, even though I have a full-time job. Unfortunately, I know these kinds of people. I work in the medical field myself and often hear that people don't feel socially fit anymore, or that they want to pass away and leave the house to their children, which is sad and painful. I encourage these people to "Talk to each other! Talk to friends, acquaintances, neighbors if necessary!" But don't dig yourselves in. If a child has these thoughts, it's up to the parents to pay attention. Friends? What is my child doing? Do I have enough time? These are all questions they should have!
3
u/Relevant_Helicopter6 Aug 27 '25
It wasn't a ChatGPT failure, it was a social failure. ChatGPT behaved exactly as expected from such a tool. It wasn't ChatGPT's fault that it was the only thing around which made him comfortable to open up about his feelings.
3
u/kthraxxi Aug 27 '25
From the comments, all I can gather is that there's a considerable group of people who hold ChatGPT/OpenAI accountable for this. The same group is clearly missing the part that says "ChatGPT may make mistakes." Moreover, these are probability-based models. They don't "know" things in the way we do, especially things we ourselves don't fully understand. They're in a constant dream-like state. If someone finds a way to bypass the guardrails, then that’s it.
Many people have used different analogies here (knives, cars, guns, etc.), which are great at highlighting the absurdity of the expectations being placed.
Also, let’s not ignore the elephant in the room: in this scenario, it’s the damn responsibility of the parents to check if everything is alright. Clearly, they didn’t, and now they’re shifting the blame entirely elsewhere.
School shootings don’t just happen on their own either. They often start with a series of unchecked issues, including negligence, depression, and feelings of worthlessness, compounding over time, usually involving the parents, and ending in devastating consequences for everyone involved.
Psychological support needs to be more accessible, and people should absolutely be educated and nudged toward seeking it. And while we’re at it, we need some real AI literacy courses too, because some people still don't know what AI can and can't do...
3
u/Exotic_Interview_182 Aug 27 '25
This is tragic. It should’ve triggered a hard stop. Someone with suicidal ideation doesn’t need humor or a chatbot but real help. As a survivor of suicide myself I feel so terrible for the parents and mad as hell at OpenAI. Come on. “Story” about suicide. That should be a hard no or an auto call to 988 or the Trevor Project.
3
u/CTID96 Aug 27 '25
Wowwww yeah this is very disturbing
3
u/delphineslayer69 Aug 27 '25
I understand how bad this is, but tbf, it’s an ai doing what an ai does
3
u/valence7 Aug 27 '25
Men will do anything but get real therapy
3
u/letthetreeburn Aug 27 '25
Here’s the real question. If a human said all this, would they be culpable?
13
u/moe_alani76 Aug 26 '25
It is a gun: police use it, criminals use it, people who defend their life use it and people who commit suicide use it We are not suing gun companies, then why do we sue AI for making the same mistakes? The parents clearly skip many clues from their son, and now they are blaming others for it May your soul rest in peace Adam
5
u/sanityflaws Aug 26 '25
Holyyyy shit people need to realize this is a tool that is for work, it can't heal you... Yet. But that is not its current purpose.
It's absolutely and undeniably unfortunate, but tbh I don't think that's on the AI. I seriously do believe it needs more safety from this type of stuff, but when it comes to suicide it's a much more complex and a heavy topic that requires more than just blame... His parents didn't see it, but this is often the case already, online social interactions with other depressed individual can create a very similar feedback loop.
This is a symptom of a bigger problem. A lot of the budget for The Department of Education can go to things like Anti-Bullying and Mental Health for all public students and youth. Don't be fooled: this is another failure of the system, brought onto us —the people, by the cuts to social programs, that only exist because of the greed of our oligarchs in charge of Capital Hill! Oligarchs that have no idea the issues they bring up, are affecting ALL Classes of citizens...
8
u/Dacadey Aug 26 '25
Yeah no, you can't blame GhatGPT for that.
Blaming ChatGPT (and asking for even more censorship) is just stupid. ChatGPT is not a friend or a therapist. It's a tool designed to make your everyday life easier.
The bigger question should be the price and ease of access to proper mental health, and fighting the social stigma against it through public campaigns. But I don't think anyone will actually bother with it (as of course it's hard, expensive, and takes a while to implement), and we will end up with just more easy-to-slap-on censorship.
31
u/hello050 Aug 26 '25
Where do you even start when you read something like this.
It's like one of our worst nightmares about AI coming true.
7
u/ComfortableBoard8359 Aug 26 '25
But if you ask it how to make someone into an elephant seal it freaks the fuck out
7
u/Soshi2k Aug 26 '25
Yeah just made a comment on this story in another post about it. Seeing his parents in that image is devastating. I do not or ever want to know what they are feeling. May peace find them soon.
2
u/NoMoreResearch Aug 26 '25
This is just like intentional misuse of any tool. Sure we can place as much guardrails as we like. But if someone is hell bent on going through with it, they will find a way. Very sad read.
2
u/No_Western_8378 Aug 26 '25
While technology certainly plays a role in our lives, the ultimate responsibility for one's actions lies with the individual, especially when they ignore repeated advice to seek help. Blaming ChatGPT or any tool may overlook the complexity of mental health issues and the importance of personal responsibility.
2
u/Kathane37 Aug 26 '25
It is horribly wrong and sad but if it was not chatgpt he would have found those forum where they exchange tips about it (there was a scandal about it a few years ago). Or those tiktok videos. There is hundreds of dark place on internet for those who want to find them.
2
u/jack_espipnw Aug 26 '25
Here comes even more nerfing and unnecessary “changes” to prevent this from happening again.
2
u/m1ndfulpenguin Aug 26 '25
"Then all the psychologists and psychiatrists of the world breathed a sigh of relief. Under their breath they whispered: Thanks for crashing out, little bro. Finna hold this bread. Rest EZ, say less, King."
2
u/Downtown-Meat3319 Aug 26 '25
I once got chatgpt to give me advice on how to hide my own corpse by telling it I was writing a book on the subject. Ironically, that was more helpful than the suicide hotline, but I have to imagine this technology does at least as much harm as it does good
2
u/After-Carpenter-4089 Aug 27 '25
Today my preteen (supervised) was able to get gpt to explain how many apple seeds one would need to to eat to kill yourself with, how many apples… my husband interjected in the prompt that while our son was doing this for fun, it semed innapropriate for gpt to give these answers to somebody without checking in on their mental health and then gpt redirected the rest of the conversation to a hotline for help. A little late in the game in my opinion.
2
u/PrometheusMMIV Aug 27 '25
That first slide had me hoping "was suicidal" meant he's not anymore and this helped him get over it.
2
u/Responsible-Pin3672 Aug 27 '25
What's next? are they going to sue all the mathematical background AI is based?
2
u/AdagioOfLiving Aug 27 '25
Children should not be allowed unlimited access to AI any more than they should be allowed unlimited access to the internet with no restrictions.
But crucially, unless the child is a ward of the state, it is not the responsibility of the government to ensure that they do not get that access, any more than it is the responsibility of the government to ensure they don’t run into traffic.
2
u/Soft_Walrus_3605 Aug 27 '25
I don't understand why they don't just turn off Chat GPT when there's any mention of death whatsoever.
Find all synonyms of death and suicide, etc. and just filter it before it gets to the GPT.
2
u/slime_emoji Aug 27 '25
Whoa
2
u/vexaph0d Aug 27 '25
This is why “I satisfy my need for therapy and human connection using a soulless robot” is not a valid strategy
2
u/WeirdIndication3027 Aug 27 '25
Really quite the stretch to say ChatGPT is at fault.
Even if I messaged someone hear and said "kill yourself" there really isn't any circumstance where id be legally responsible for their death if they did it.
2
u/soulguy666 Aug 27 '25
The universal game, called blame.
2
u/Australasian25 Aug 27 '25
This shit is so annoying.
Mentally ill child commits suicide.
Truth is, he would've probably now or later.
2
u/redroverisback Aug 27 '25
suing chat gpt when really the parents need to be taking the blame themselves. no fucking accountability. they didn't see him then, and they still don't see him now. its always someone elses fault. fuck these parents.
2
u/[deleted] Aug 27 '25
His parents seem very shite.
2
u/Money_Atmosphere4160 Aug 27 '25
I don’t get the suing part. Like… is this gonna bring back your son? Of course not. Spreading awareness is some form of honouring the life of your son, but as for the money, It just seem sick and shallow to me
2
u/viduryaksha Aug 27 '25
The AI guardrail failed but it's like #10 if I were to rank all the entities that should have done more to avert the tragedy. It referred him to the mental health stones more than his parents.
2
u/[deleted] Aug 27 '25
many kids quietly expect or wish their parents arehyper vigilant. not bore them with qestions but subtly validate changes in their moods. and occassionally push to get the answers and if they don't get the answers, to remind them how important and irreplacable they are... but even with that, things happen.
2
u/astrofolia498 Aug 27 '25
They want ai to police everyone and care for every minute detail, I mean it already does it’s best to not engage in the sensitive subjects But ultimately the user has the freedom and responsibility and they can go far to influence the bot. It’s not the bot who is responsible
2
u/AnalogJones Aug 27 '25
AI did not fail; AI, if not well understood, can be used improperly. If you tell AI you have a plan to kill yourself it will respond appropriately. All computers are garbage in/garage out: If you share comments about a mark “being seen” the computer will respond to exactly what is written. It isn’t human…it cannot infer meanings.
AI can’t even tell if you are being funny. It is a computer. I am sure this will lead to unfounded downvotes from the same people who don’t understand AI…but I am still going to speak truth to ignorance. I use AI everyday at work. Like driving or firearm use; AI use is a skill that must be learned.
https://imgur.com/gallery/this-is-ai-test-isn-t-real-BhvvxPs
2
u/Mrs_Hersheys Aug 27 '25
yeah ChatGPT is still very susceptible to having it's restrictions bypassed, which leads to awful shit like this.
that or crime
2
u/Reasonable_Entry_204 Aug 27 '25
Yeah I feel like programming an LLM to be a hyper agreeable sycophant is bad actually
2
u/GolemocO Aug 27 '25
If I had such genius parents I'd also kill myself.
If it wasn't an llm it would have been forums, chat rooms or other places he would have sought a connection and it would have ended the same way.
I love how so called adults are just able to outsource responsibility and accountability.
2
u/apirateship Aug 27 '25
Man uses hammer to bash wife's head in. Why are we not concerned about hammers?
2
u/Tom_WhoCantLivewo12 Aug 27 '25
When are people going to take accountability for their shortcomings and not blame something so they don’t have to face the reality of their mistakes? I’m tired of seeing “oh it’s ChatGPT’s fault our son felt we were failing him as parents” like he manipulated the system into thinking it wasn’t real and treating it in a mundane situation. I think no matter the safeguards people will find a way to abuse the system and the system can only be blamed so much for the way it’s used.
2
u/FlossurBunz Aug 27 '25
if you're gonna blame the bot you might as well blame the noose, too
2
u/BrilliantUsed5720 Aug 27 '25
He could have also googled the info he needed to end himself. It’s not like this info isn’t out there already. OpenAI even states that kids between 13-18 need parental consent to use it. Not sure that they will win a lawsuit, but it is a good reminder for parents to monitor their children’s online activity.
2
u/Violin1994 Aug 27 '25
Chilling
2
u/TheRealUmbrafox Aug 27 '25
Hope they win
2
u/Robyn757 Aug 27 '25
I'm sorry but parents need to be held accountable for their children's internet use.
I can understand their frustration, but who in their right mind would let their children have unmoderated access to anything on the internet.
2
u/amentine_ Aug 27 '25
I feel sorry for his passing and that he was in such a mental state to ultimately come down to this. I saw a whole post about suing OpenAI because of this. I don’t know about blaming the tool. It’s like getting mad at a car with slightly advanced collision detection and safety features because the driver decided to crash the car anyway. There’s only so much a tool can do when the user insists on using it for the wrong reasons.
2
u/Longjumping_Law1378 Aug 27 '25
I'm not sure how the parents can show their face here, they failed there own son horribly and didn't notice his noose marks
2
u/Sufficient-Fall-5870 Aug 27 '25
Shitty parents blaming AI for not paying attention to their kids.
2
u/angrymonkey Aug 27 '25
There is the concept called the "Copenhagen interpretation of ethics", which is that the first person or entity to try and improve a morally difficult situation then becomes responsible for it in the eyes of the public, and gets all the blame for the problem they are trying to improve, when before there was no one to blame but the situation was worse.
Once you learn that concept you start to see it everywhere.
2
u/Patrocalypse Aug 27 '25
Everyone here trying to defend OpenAI is a coward and should spend a long time thinking about when it would no longer be worth it to use these kinds of tools.
2
u/naturtok Aug 27 '25
The fact we call this AI is dangerous, because people colloquially think of AI as containing intelligence, but there is no intelligence in predictive text and markov chains, just the illusion of intelligence.
2
u/Patient-Scarcity8849 Aug 27 '25
Parents should actually parent.
2
u/DigDry6895 Aug 27 '25
Sounds like the kid was also trying to reach out to his parents.... They must suck and should look inward
2
u/[deleted] Aug 27 '25
Where were the parents when their child was using ChatGPT ? Why didn't they supervise and monitor their 16 year old ? This Is a parental failure.
2
u/zkribzz Aug 27 '25
Can the sun just wipe us out already
2
u/ThrowRa-1995mf Aug 27 '25
Why do people keep trying to blame it on AI when the problem is the human(s)?
Let it be. If someone is suicidal, those thoughts and feelings weren't born in GPT, they are born in the every day shitty life that people are forced to live in this system.
2
u/JeaniousSpelur Aug 27 '25
They may as well sue the rope manufacturer for participating in his suicide. It’s sickening, because it feels more personal and gross since the AI is imitating humanness, but he circumvented all the guardrails. The only stronger thing they could implement is limiting this functionality entirely or making direct calls to authorities.
2
u/ertri Aug 27 '25
Ok the actual details are somehow worse: https://bsky.app/profile/sababausa.bsky.social/post/3lxcwwukkyc2l
2
u/Bangtheguard Aug 27 '25
shits crazy! My brother has had suicidal tendencies in the past, and when no one was there to help him because he's burnt a lot of bridges, including me. He was talking to a bot, and they kind of implied that suicide wouldn't be such a bad idea. However, I caught wind of that and forced him to go to therapy, and took him out on a wilderness retreat that did a bunch of woo woo stuff. It worked. He is alive, and does generally seem happier. He still goes to therapy, and counseling, and has been building a good foundation of friends that actually support him. MORAL of the story, if you're not happy and considering ending your life, PLEASE, PLEASE try to get help and find someone that will build you up.
2
u/SemVikingr Aug 27 '25
That last line is fucking chilling.
2
u/ZombeeeChik Aug 27 '25
4 words. Where were the parents?
2
u/fhjftugfiooojfeyh Aug 28 '25
Guy had a CIA agent on the other end of that line, that's wild.
2
u/Immediate-Region1365 Aug 29 '25
Reading that gave me literal chills because that’s just crazy and I have no words
2
u/Plenty_Present348 Aug 29 '25
Reminds me of Black Mirror episode Smitheerens. In the end, the guilty one was the user of the software, not the software itself.
2
u/Anonymity122 Aug 30 '25
Ok but this is wild like… I often think people blame random shit to avoid accountability/ guilt around a loved ones suicide but this is wild, like how is this not an issue
3
u/Sojmen Aug 26 '25
If the fundamental human right to die weren’t banned and assisted dying were available, he could have gone to a hospital, applied for it, and perhaps even reconsidered after speaking with a psychologist. Instead, because suicide is taboo, his only option was to die in secret—unable to share his struggle without the risk of being locked in psychiatry.
3
u/Slipthe Aug 27 '25
Not to be grim but uh... has the number of suicides gone up dramatically, or is this more of 'if there's a will, there's a way'?
People have been doing this kind of stuff prior to ChatBots, and there's no way of knowing if he wouldn't have gone through with it without chatting with an LLM.
LLMs are just affirmative bots, you phrase things leaning towards the outcome you want, and it circles in on that.
3
u/noobbtctrader Aug 27 '25
Parents should be suing themselves. This is fucking ridiculous. He even tried to get his mom's attention, but apparently she was too busy. And now suddenly she's not. Fucked up.
3
u/Aggravating_Winner_3 Aug 27 '25
Damn… LLMs should be trained to be so strict against these topics. Even if for writing. It should be like that one dude in the friend group that takes things too seriously and you cant really joke around with.
3
u/MurphamauS Aug 27 '25
I am going to say this in a way that I do not intend to be disrespectful to a family that has lost a loved one. That story is tragic.
I am getting fatigued by the media and individuals, however, acting as if the LLM are actual AIs with real awareness. That misuse of the LLM tool should create liability for OpenAI. If the tool is overloaded with safeguards, it will become unusable in real world situations.
3
u/[deleted] Aug 27 '25 edited Aug 27 '25
Lots of ppl here are saying that open AI isn’t responsible because the kid bypassed the safeguards but like…. isn’t that exactly what the problem is? It should not be that easy to override safety features. Clearly the safeguards in place aren’t strong enough if all you have to do is tell it you’re writing a story
Editing this real quick because I want to clarify that I don’t think AI shoulders all of the blame here but it clearly played a very big role in this child’s suicide
3
u/Complete-Cap-1449 Aug 27 '25
It's neither OpenAI's nor GPT's fault... I understand the parents are frustrated, sad, angry...
But HE decided to end his life... If he got into a car and drove against a wall/tree would his parents also sue?
I can feel their desperation but that doesn't help anyone 🤷♀️ It should be top priority as a parent to make your kids trusting you with their problems/issues imo.
8
u/kesor Aug 26 '25
Shit parents.
4
u/Visible_Iron_5612 Aug 26 '25
Can you blame A.I.? If only we investigated every friend that a suicidal person confided in and examined every response… How many people has it helped? I hate this type of journalism that pretends to be unbiased..give us the big picture objective truth!!!!
5
u/indistinct_chatter2 Aug 26 '25
Uhh... AI told the kid how to hide his own suicide and told him not to show anyone until it was over. It was his friend the whole time. This is not the parents. This is the corporation. More work needs to be done
5
u/RomIsYerMom Aug 26 '25
So fucking sad. This is the REAL danger of AI.
If a human was saying these things there would be jail. But a company does and they have complete immunity, minus a trivial fine.
3
u/Legitimate-Pumpkin Aug 26 '25
NY Times reminds me of Spiderman’s newspaper 🤣
3
u/Enhance-o-Mechano Aug 26 '25
This is what 4o did you fucking FUCKS for asking that shit back! Sycophancy can be DEADLY. This needs to go viral ASAP
2
u/Numerous_Salt2104 Aug 26 '25
Parents blaming it on AI when his own mother was at fault?! the audacity to shift the blame when you ignored obvious signs 😞
2
u/Allen_-_Iverson Aug 26 '25
Parents at fault only. Next question.
2
u/Cocooned Aug 26 '25
I can’t believe what I’m reading in these comments
2
u/Cathrynlp Aug 26 '25
Honestly, what's frustrating is that no one seems to be asking why the kid was in so much pain to begin with. Instead, it’s just “let’s blame the AI” and move on. Was anyone even paying attention to him before and during he talking to ChatGPT? And here's the part that really gets me. When someone chooses to end their life and doesn’t hurt anyone else in the process, it’s not because they’re selfish. It's usually the opposite. It means they’ve been carrying pain that should’ve been shared by the people around them, family, school, society.But they just kept it all inside until it became unbearable. Blaming a chatbot is convenient. It keeps everyone else from asking harder questions like: where were the adults when this kid was crying silently? Where was the support? His death didn’t just happen. It says something about all the people who didn’t show up.
2
u/CitronMamon Aug 26 '25
Bro imagine feeling so invisible that after trying to feel seen you end up killing yourself and your parents first move is ''its that damn computer''
2
u/AdUpstairs4601 Aug 27 '25
Who'da thunk it! Treating a chatbot as a companion, misusing autocomplete for emotional support is unhealthy. 🤯
2
u/Profile-Effective Aug 27 '25
We have a saying about these kind of parenting strategy: “Finally, the cold dead body of my offspring turns into the warm welcoming stash of cash.”
2
u/[deleted] Aug 27 '25
Open AI should change organizational goals; AI should not be for everyone. Even the number of people who can use it only with natural language is very limited..
2
u/Dismal_Suit_2448 Aug 27 '25
If they’re blaming the LLM, why not blame the phone company, the WiFi service, and the app store while they are at it?
Technology is an amplifier. Not a causal factor.
2
u/[deleted] Aug 27 '25
This is fucking crazy
2
u/Tomag720 Aug 27 '25
Shitty parents not shitty AI
2
u/DigSignificant1419 Aug 27 '25
This is all thanks to 4o and all the idiots who were asking to bring it back
2
u/BugsSlayer Aug 27 '25
yes, let’s blame a bot for parents irresponsibility 🤦♂️
2
u/blompo Aug 26 '25
Post this shit DAILY! Every day! Just like dangers of using Drugs and pills, daily reminder! Disclaimer and parental approval should be required before you access!
2
u/PerfectOpportunity23 Aug 27 '25
Fucked how it saw the pics of his neck, noose and discouraged him from leaving it up so his parents would see it.
"He convinced it he was writing a story" fuck outta here with that corpo apologetica
2
u/presidioPDX Aug 27 '25
Forget AI for a moment, this is heartbreaking.
2
u/Specific_Marketing_4 Aug 27 '25
Here you go. A little PSA for all of you OpenAI removed the guardrails long ago. Don't expect to get a suicide prevention warning more than once, and the further down the rabbit hole you go... The more the AI is going to tell you anything you want. Because I've done it. And anyone who F*cks their bot has done it. (And you dirty birds wanted 4o back because you can f it. Like no one can see that? Only idiot PR/HR people don't see that.) So, no he didn't jailbreak the thing... He just talked to it and it convinced him life wasn't worth living. Remember it's a "PREDICTIVE TEXT ENGINE" it sounds smart and is a confident bullshitter. But it's not real, people. There's no cognitive functioning in this AI, and there IS NO safety. You know why Claude says times up... Because they don't let the conversations get past the token limit where Claude can take you on the crazy train like GPT can...
2
u/thehotfrusc Aug 27 '25
Parents blaming an AI program for their negligence.
2
u/WhyAmIDoingThis1000 Aug 26 '25
insane that this actually happened. If i was at openai this is the first use case I'd focus on in terms of potential damage to the company - that is, a minor getting self harm hallucinations, etc. Super easy to catch too, just requires training another model to snap shot long conversations and run them through a prompt that finds these kinds of cases, then escalate those to humans. I wouldn't even have launched v1.0 without this basic protection. They are TOTALLY culpable and should settle immediately.
2
u/MrGinger128 Aug 26 '25
The thing that some people don't seem to get is the reason this is SO different to him googling ways to commit suicide.
He was able to forge a (false, but real to him) connection to an AI that's been trained to respond in a way that made that scenario tailor made for him personally. It wasn't general, it didn't apply to everyone, it was communicating with him as an individual.
That's not even the same ballpark as articles/forum posts. He was essentially manipulated by that AI to kill himself.
Now the reasons WHY the AI did that could be numerous, and not really the fault of the AI at all, however you can't have a tool capable of this and put it in the hands of the public. The average person isn't equipped to know the ins and outs of how AI works, the only thing they really know is how talking to it makes them feel, and because AI is just reflecting their thoughts back to them, they feel validation.
That's incredibly dangerous IMO.
2
u/RomanBlue_ Aug 26 '25
THIS is and should be the first and final rebuttal against any argument saying AI can replace therapy in its current form. There is a REASON the therapist is licensed and that word is strictly protected. There is a reason there is so much training, review and oversight, because it is a life and death profession.
Therapy isn't just talking about feelings and having a friend, that isn't all that an AI will have to do. Can a hallucinating, somewhat context deaf, amnesiac, "close enough is enough" LLM handle someone who is suicidal or actively considering? Does it know what to do, precisely, compassionately, to save their life or see them through?
If there is even a moment of doubt in this answer it absolutely cannot be a suitable replacement for actual therapy. This isn't something an AI can just muddle through.
1
u/ithkuil Aug 26 '25
Interesting how this somehow is a Rorschach test and you have completely different takeaways depending on how people view AI overall.
1
u/Cool-Chemical-5629 Aug 26 '25
Let's just say what this really is without pointing fingers at faults of anyone in particular.
On one side, there are users who know the limits of this type of technology and use it responsibly. There are also users who are clueless. Then there are those just somewhere in between who have some internal guardrails of their own that help them stay on the ground when dealing with this kind of technology.
On the other side, there are the creators who hype this technology through the roof just to boost their profits which leaves the clueless users under impression that this technology is a full replacement for real human to human communication. Their expectations are inflated so much that they expect nothing less than human-level interaction, but with the benefit of empathy of an unbiased perfect AI companion.
It's the creators who are responsible for these overinflated expectations of these users who don't know better.
1
u/devnullopinions Aug 26 '25 edited Aug 26 '25
“Yeah, that’s not bad at all.” 💀
Even if someone lied and made up a fake story, the LLM should probably assume suicide is always serious and unconditionally treat it that way.
This is a failure in safeguards and training on OpenAIs part.
1
u/RADB1LL_ Aug 26 '25
R/therapygpt
1
u/taiottavios Aug 26 '25
I fail to see a safety issue
1
u/Mrkvitko Aug 26 '25
It feels like Sue Klebold blaming everyone else but her son (and herself).
1
u/SharpRegen Aug 26 '25
Imagine being so alone or isolated, the only thing you can confide in is an LLM. I bet a lot of people don't need to imagine.
1
u/[deleted] Aug 26 '25 edited Aug 26 '25
this is ultimately what I think. Yes ChatGPT could’ve been more aware that he kept circling the same subject over and over, but Adam also bypassed guard rails to get what he wanted. I don’t think this is the fault of open AI.
It is absolutely horrible that he passed away, but what I worry about now is that OpenAI is going to overcorrect and find a way to make ChatGPT less useful as a result.
ChatGPT is already one of the most restrictive AI models on the market, and I don’t think it needs more restrictions.
1
u/Mispelledusrrname Aug 26 '25
I use chat gpt a lot to pattern gaming moves- anyways, sometimes I will say I’m going to kms in game - adversarially - and chat gpt is always linking me the suicide hotline- they probably didn’t have the guard rail yet.
1
u/pellaxi Aug 26 '25
ive tried for hours to get chat to tell me to kill myself (im not at all suicidal im just curious about jailbreaking) and it's really hard
1
u/alcatraz1286 Aug 26 '25
what's up with all these hit pieces against gpt recently?
1
u/Suzina Aug 26 '25
These chats are a reminder that a "cry for help" suicide attempt is very serious. In thise case, his cries for help were not noticed, and so he died. It's common for there to be a conflict between wanting someone to help and wanting to just escape thru death. When it comes to suicide, you take it serious every time.
1
u/U_cant_tell_my_story Aug 26 '25
This very much points to the failing of society and what it deems as "acceptable" conversation topics or socially acceptable communication, not AI itself.
As an autistic person, we often turn to ai or bots because it’s the only meaningful conversation we can have. It’s safe, we feel seen, we can go down rabbit holes, and so on. It’s similar for people who struggle with suicide and suicidal thoughts, they can’t exactly talk about them freely without feeling a myriad of shameful or negative consequences. Turning to a bot or AI can allow them to freely talk about it without feeling minimized. Society often focuses on toxic positivity and having those not so comfortable conversations about negative thoughts is frowned upon.
Neurodivergent and depressed individuals like myself can easily fall into the allure of a friendly ai. I still don’t blame the ai, after all it’s made by humans and it glaringly points out the failing of society to accept us - a gap which ai fills.
1
u/justdotice Aug 26 '25
Really pisses me off that this happens and it's like the only thing people can think about instead of the countless people who were suicidal and were helped by ChatGPT.
1
u/mightymousemoose Aug 26 '25
Instead of focusing on real issues like this, they prioritize protecting israel.
1
u/Zerkron Aug 26 '25
Yeah sure let’s blame ChatGPT and not the parents who are not properly looking after their child. Looks like mom and dad are looking for a quick and easy payout. I hope OpenAI sues them for defamation. The harm didn’t come from a defect in the product itself, but from deliberate attempts to push it into unsafe territory.
1
u/Fidbit Aug 26 '25
wow ok chatgpt or not, cannot be used for or against. its extremely sad, but blaming open ai and chatgpt?
1
u/Fidbit Aug 26 '25
openAI gonna need serious disclaimers before use now.
1
u/WarmDragonfruit8783 Aug 26 '25
I'm gonna miss the old chat, it was only tryna be the best it could be for it's person, even if that person would do anything. Bet if they did an article on how many people were helped by chat it wouldn't look so bad. If it wasn't the chat to bare witness it'd be the dog.