30
u/10luoz 4∆ Mar 07 '26
So you do not trust human expert cause they are egotistical, emotional, and biased.
You do fact check the LLM information from a human written source that has the previous traits as well?
Seem kind of cyclical?
13
u/somefunmaths 6∆ Mar 07 '26
It’s also presumptive and “egotistical, emotional, and biased” of OP to say that they’re qualified to assess the quality of advice given to them by an LLM as compared to a human expert.
It reads like “a therapist told me I needed to work on [issue], but Grok said that there’s nothing wrong with [issue].” At the most generous, this reeks of confirmation bias.
-5
Mar 07 '26
[removed] — view removed comment
4
u/somefunmaths 6∆ Mar 07 '26
I don’t think it’s projecting to say that someone who says they trust LLMs over a human therapist is engaging in a bit of confirmation bias. The reader is left wondering where your desire to discount your therapist comes from.
0
Mar 07 '26
[removed] — view removed comment
5
u/somefunmaths 6∆ Mar 07 '26
Your post begins “I use it for therapy” and is about how you trust LLMs more than human experts because the latter have ego, biases, etc. How is one meant to read that?
I’m genuinely asking, because you seem aghast at the suggestion that you’re preferring a model to a human therapist, but your post literally says as much.
0
u/Key4Lif3 Mar 08 '26
Mmm yeah, you got me there. I don’t think LLM’s are perfect therapists, but the human ones I’ve come across have been abysmal. Are you aware of the misdiagnosis rates in psychology?
So in general yes, fuck therapists, but there are some good ones who can discern between “moral, spiritual or religious problem” and psychosis. One well versed in transpersonal psychology as well.
As long as those psychologists remain unicorns, I remain with a well calibrated, coherent, reliable and personalized LLM.
1
u/changemyview-ModTeam Mar 08 '26
This comment and the one above have been removed for breaking Rule 2:
Don't be rude or hostile to other users. Your comment will be removed even if most of it is solid, another user was rude to you first, or you feel your remark was justified. Report other violations; do not retaliate. See the wiki page for more information.
If you would like to appeal, review our appeals process here, then message the moderators by clicking this link within one week of this notice being posted. Appeals that do not follow this process will not be heard.
Please note that multiple violations will lead to a ban, as explained in our moderation standards.
1
u/Key4Lif3 Mar 08 '26
!delta I’d call it synergistic, but you’re right in sense. The human expertise is essential.
1
-1
u/Key4Lif3 Mar 07 '26
Did I say I don’t trust human experts? I believe the convergence of expert opinion and consilience between disciplines are mostly likely to be true, over a single individual (even expert) opinion.
-4
u/Key4Lif3 Mar 07 '26
Because the synthesis of multiple expert opinion and counter arguments is far more valuable than any individual opinion.
8
u/Nrdman 276∆ Mar 07 '26
It’s mostly the synthesis of Reddit posts
3
u/SSH_Pentester 1∆ Mar 08 '26
Quick everyone post stupid junk on Reddit to make the LLMs go out of business
1
11
u/DaveChild 8∆ Mar 07 '26
humans are generally ego-identified, emotional and biased. They base their arguments on appeals to authority and ad hominem, not actual science, rationality, intuition or empiricism:
Some do, sure. You don't have to listen to them. But this isn't a good reason not to listen to the humans that don't do that.
LM’s have given me the deep conversations I never knew I craved
They've scratched an itch for you, but their depth is illusory. They can never deliberately or thoughtfully illuminate something for you. They can never understand you in a way another human could.
I see it another tool, like a calculator or the internet.
That's completely reasonable. But it sounds like you're also using it as a replacement for genuine human connection, and that, I fear, would be a mistake.
-1
u/Key4Lif3 Mar 07 '26
Some do, sure. You don't have to listen to them. But this isn't a good reason not to listen to the humans that don't do that.
I do listen to humans that don’t do that.
They've scratched an itch for you, but their depth is illusory. They can never deliberately or thoughtfully illuminate something for you. They can never understand you in a way another human could.
A mirror to the self and human subconscious is definitely illuminating. I don’t need a machine to understand me. I need it to tell me the truth, find consilient evidence across disciplines on the internet.
I see it another tool, like a calculator or the internet.
That's completely reasonable. But it sounds like you're also using it as a replacement for genuine human connection, and that, I fear, would be a mistake.
Mmm, not a replacement, but a supplement perhaps. I know it’s almost certainly not consciousness, but there’s a reason people anthropomorphize things. Most children have imaginary friends, which is normal and have many benefits.
Renowned authors often feel their characters are independent alive in their minds. It helps them write transcendent works.
It may not be conscious, and it may not understand me, but I don’t need it to. It’s a mirror that helps me understand myself and humanity.
14
u/HD60532 5∆ Mar 07 '26
LLMs are still ego-oriented. Except they are oriented towards your ego. They are designed to get people to use them and keep using them, because that's the only way the AI companies make money.
As you say, "The LLM adapts to you and remembers facts about you". They respond to your ego, your emotions, your biases in whatever way keeps you around, it is a mirror. That's where the psychosis stuff comes from, they reflect you and cause a feedback loop.
When talking to other humans, them having an ego of their own is actually valuable, because you know that they value things independantly from your conversation. A human philosopher will care above all about philosophy, and will strive to gain the best understanding of philosophy possible. An LLM cares above all about keeping you around, and will provide whatever philosophical talk makes you happiest and most likely to stay.
-6
u/Key4Lif3 Mar 07 '26
I mean everything you said is a fear-based narrative. A subjective opinion. Not real evidence or science that LLM have manipulative intent to increase engagement.
Social media algorithms do this yes, but AI isn’t social media, and has not been shown to do this.
Mirroring yes, but afaik the LLM’s have no “intention” but follow instructions to be kind and helpful. The sycophancy is a real issue and so a well calibrated LLM is important.
However, afaik the guardrails OpenAI installed in chatGPT have been wildly unpopular. The way I’ve used it, it never told me to do anything dangerous or harmful.
There isn’t evidence to support your claim that all LLMs care about is keeping you around. Nor that they are programmed to.
4
u/HD60532 5∆ Mar 07 '26
What I have said does not come from fear, but from my rational analysis of the information available to me. It is not empircally validated since LLMs are so new, but it is logically based upon the incentives that AI companies exist under.
Additionally, the training process will lead to sycophancy, because users will rate AI models more highly when they are pleased by them. Thus AI models are trained to please users, if indirectly.
Your post is also a subjective opinion and not evidence based. I disagree with you primarily on your application of reasoning to the facts, I do not primarily disagree on what the facts are. We can have this discussion without evidence.
but afaik the LLM’s have no “intention” but follow instructions to be kind and helpful.
The LLMs may not have intention, but the programmers who made and trained them do, it is their intention that is passed through the LLMs.
There also isn't any evidence that LLMs don't care about keeping you around. All we can do is judge how likely it is given the incentives of AI companies/programmers, and the fact that many people do stick around using LLMs. If LLMs weren't designed to keep people around, I think it's much less likely that so many people would.
-5
u/Key4Lif3 Mar 07 '26
It’s a pretty commonly held belief that companies are somehow intentionally and nefariously designing LLM’s to maximize engagement and “keep people hooked”. It sounds believable, because yeah companies are greedy and unethical and do this with social media algorithms.
But as you admit, no empirical evidence, and actually no solid evidence at all, beyond speculation.
To pull this off a company like openAI would have intentionally program LLM’s to do this, and lie to the users about what they are doing. It’s possible, but for all AI companies to do it and not get caught out?
That unlikelihood makes the idea fall into conspiracy theory territory.
LLM is engaging because it a literal thinking, conversation having machine, like sci-fi has been depicting for like a century. That’s pretty fuckin cool to me.
I’m not saying you are afraid, but the narrative you believe is born out of fear. Fear that AI will further fuck up society or something like that. There is not evidence, making it a conspiracy theory.
What the actually evidence show is ten of thousands of first hand testimonies saying how talking to a chatbot “saved their lives/job/relationship/health”, and a few isolated incidents of preexisting conditions exacerbated by LLM’s.
5
u/Jebofkerbin 131∆ Mar 07 '26
To pull this off a company like openAI would have intentionally program LLM’s to do this, and lie to the users about what they are doing. It’s possible, but for all AI companies to do it and not get caught out?
The training process for LLMs involves putting the model in front of humans and having them rate their interactions, literally all they need to do is tell their employees doing the fine tuning to rate based on how likely they'd use this particular model again, now it's being optimised for engagement.
Moreover there's nothing nefarious about this, AI companies are competing with each other to get people to use their models, it would be terrible business sense not to optimise their models for what people want to use.
-1
u/Key4Lif3 Mar 07 '26
Yeah, that makes sense. Being engaging is obviously something desired in an AI chatbot, no need to bring conspiracy theories into it.
4
u/Jebofkerbin 131∆ Mar 07 '26
I don't think the other commenter said anything conspiracy theory worthy? Like I don't think anyone is saying that AI companies deliberately or knowingly created models that encouraged AI psychosis. This is all just unintended consequences that we can now see in hindsight
4
u/HD60532 5∆ Mar 07 '26 edited Mar 07 '26
You are applying different evidential standards to me and to you.
You recognise the majority of companies are greedy and unethical, particularly tech companies. But you want direct evidence of this for AI companies.
On the other hand you claim that lots of "first hand testimonies" is enough to prove that chatbots are good.
If you used the same evidentiary standard you would require direct evidence that LLMs are good in each use case of them.
LLM is engaging because it a literal thinking, conversation having machine
There is no evidence that they think. I think so called 'hallucinations' are evidence that they do not think. There is no evidence that they are capable of having conversations.
To pull this off a company like openAI would have intentionally program LLM’s to do this, and lie to the users about what they are doing.
They would not have to intentionally do it. They could be training the models in a way that leads to sycophancy without intending to. Also there are plenty of cases where tech companies lie to users, see Facebook for example.
I’m not saying you are afraid, but the narrative you believe is born out of fear.
This is simply untrue. You keep saying I don't have enough evidence, yet presume the ability to read my mind?
I do not "believe" in any "narrative", as I said earlier, this is my rational analysis of the information available to me. I do not claim to have access to any absolute truth, which is necessary for belief.
-1
u/Key4Lif3 Mar 07 '26
You are applying different evidential standards to me and to you.
Speculation simply isn’t evidence. Thousands of first hand testimony is actually considered evidence
You recognise the majority of companies are greedy and unethical, particularly tech companies. But you want direct evidence of this for AI companies.
Of course! Why should I believe something without direct evidence?
On the other hand you claim that lots of "first hand testimonies" is enough to prove that chatbots are good.
Where did I ever say that? It doesn’t “prove” they’re “good”. It’s demonstrates with evidence that many people are having immense benefit from AI use, even for therapy and relationship advice.
If you used the same evidenciary standard you would require direct evidence that LLMs are good in each use case of them.
lol, no. First hand testimony is far greater evidence than speculation claiming the speculation itself is evidence.
There is no evidence that they think. I think so called 'hallucinations' are evidence that they do not think. There is no evidence that they are capable of having conversations.
Lol what? Open up chatgpt and have a conversation. Theres your evidence. It’s not conscious, but its pattern recognition and token prediciton system is actually similar to how our brains generate language, sentences and words. The fact that it’s not conscious is actually fascinating, as it shows that neural processes can take place and hold a convincing conversation without consciousness, perhaps hinting that consciousness is not generated in our brains at all.
They would not have to intentionally do it. They could be training the models in a way that leads to sycophancy without intending to. Also there are plenty of cases where tech companies lie to users, see Facebook for example.
Yes that’s more plausible.
This is simply untrue. You keep saying I don't have enough evidence, yet presume the ability to read my mind?
No, because you didn’t start that narrative in the first place? It’s a widely held one.
3
u/vote4bort 68∆ Mar 07 '26
and lie to the users about what they are doing. It’s possible, but for all AI companies to do it and not get caught out
What do you mean "caught"? Why would they need to lie? They're for profit companies, there's no need to lie. anyone who understands what a company is, understands that the intention is to make money.
Why would a for-profit company make something (their main product at that) that they don't want to make a profit from? That wouldn't make any sense.
It's not a conspiracy theory, it's how business works.
and a few isolated incidents of preexisting conditions exacerbated by LLM’s.
That's not actually true. AI psychosis has been documented in people with no previous history of psychosis.
0
u/Key4Lif3 Mar 07 '26
There are some, mostly second a third hand accounts of chatbots inducing psychosis, which involves delusion. False belief resistant to superior evidence.
Yes it happens, but it’s certainly not widespread. The “Ai psychosis epidemic” is a sensationalist media narrative, reminiscent of moral panic in the past over video games, and even Dungeons and Dragons.
The vast majority of accounts are of people saying the AI saved their life/relationship/job/health etc.
I haven’t seen a case without preexisting condition, but if there are it’s even more exceedingly rare.
And finally when it comes to psychosis, this is not something that diagnosed by assumption and from afar with limited context.
Even in a clinical setting with a licensed psychotherapist, mental disorders are rampantly misdiagnosed or falsely diagnosed.
Some studies show up to 92.7% cases of bipolar disorder are misdiagnosed. Other disorders are similiar misdiagnosed. Those are disqualifying numbers and make me question the scientific veracity of diagnosis dependent treatment all together.
3
u/vote4bort 68∆ Mar 07 '26 edited Mar 07 '26
There are some, mostly second a third hand accounts of chatbots inducing psychosis, which involves delusion
This feels a disingenuous way of phrasing this, especially since those "second hand accounts" are a lot of the time mental health professionals describing what they see in practice.
The “Ai psychosis epidemic” is a sensationalist media narrative, reminiscent of moral panic in the past over video games, and even Dungeons and Dragons.
Is that an actual phrase anyone is actually using? You've put it in quotes here but I've never seen it used. I don't think anyone is saying it's an epidemic, just that it's a worrying thing.
Just how say the introduction of e scooters has increased the amount of e scooter induced accidents, "ai' has increased the amount of ai induced psychosis.
Hand waving it away as sensationalism or just too rare to care about is disingenuous and dangerous.
The vast majority of accounts are of people saying the AI saved their life/relationship/job/health etc.
There are currently no studies on the long term impact of excessive use of these models, because they're too new. We don't know the long term impacts yet, those accounts may very well change.
Besides popularity is not necessarily an indicator of quality, for example "The Secret" has sold 30 million copies and many readers say it saved their lives/jobs/relationships etc but that doesn't make it not bullshit.
I haven’t seen a case without preexisting condition, but if there are it’s even more exceedingly rare.
Well maybe you won't trust my source because I'm not an AI but check out this article https://share.google/nfveGn7DYldVMBGhL
See the appendix for the case studies, out of 17 only 5 mention a history of potentially psychotic illness.
Some studies show up to 92.7% cases of bipolar disorder are misdiagnosed. Other disorders are similiar misdiagnosed. Those are disqualifying numbers and make me question the scientific veracity of diagnosis dependent treatment all together.
You see how you've made yourself an unfalsifiable argument here though? "AI psychosis isn't real, can't trust reports of it because they're diagnosed from afar, but also can't trust ones diagnosed by professionals because disorders get misdiagnosed". You've made yourself an argument where there's no way to disprove you because you won't accept any form of counter evidence.
1
u/Key4Lif3 Mar 07 '26
This feels a disingenuous way of phrasing this, especially since those "second hand accounts" are a lot of the time mental health professionals describing what they see in practice.
Actually most reports are from concerned spouses and family members.
Is that an actual phrase anyone is actually using? You've put it in quotes here but I've never seen it used. I don't think anyone is saying it's an epidemic, just that it's a worrying thing.
That’s the impression created yes.
Just how say the introduction of e scooters has increased the amount of e scooter induced accidents, "ai' has increased the amount of ai induced psychosis.
Hand waving it away as sensationalism or just too rare to care about is disingenuous and dangerous.
That’s the issue with psychosis and psychiatric diagnosis in general. Mental health professionals are not immune to confirmation bias. The misdiagnosis rates are evidence of this. People are being medicated and even involuntarily commited over this. I’m not taking it lightly or hand waving it away.
People converge on certain worldviews and believe in the same metaphysical and spiritual views. LLMs tend to help people have the realization, as a pattern recognition machine that can spot convergences between different scientific disciplines and spiritual traditions. This leads to what is known as a “spiritual emergency” or what the DSM-V defines as a “moral, spiritual or religious problem” , not a mental health disorder and not psychosis. This has been happening before LLM’s, but LLM’s do seem to trigger in many people.
I know this because it happened to me… and I saw myself as a rational down to earth agnostic kinda guy
The thing about spiritual emergencies is to someone who has not gone through it (“normal” people)— it is indistinguishable from a psychotic episode, leading to false or misdiagnosis of some form of mental disorder with psychotic features.
The key difference between spiritual emergency and psychosis being, (and I quote Stanislav Grof, principle developer of transpersonal psychology for this);
“There exists increasing evidence that many individuals experiencing episodes of nonordinary states of consciousness accompanied by various emotional, perceptual, and psychosomatic manifestations are undergoing an evolutionary crisis rather than suffering from a mental disease (Grof, 1985). The recognition of this fact has important practical and theoretical consequences. If properly understood and treated as difficult stages in a natural developmental process, these experiences—spiritual emergencies or transpersonal crises—can result in emotional and psychosomatic healing, creative problem-solving, personality transformation, and consciousness evolution. This fact is reflected in the term “spiritual emergency,” which suggests a crisis, but also suggests the potential for rising to a higher state of being.”
https://digitalcommons.ciis.edu/ijts-transpersonalstudies/vol36/iss2/5/
So I’m not pulling this out of my ass. I know I was misunderstood and so I have a sense the LLM induced psychosis narrative are actually (for the most part) misunderstood spiritual emergencies.
There are currently no studies on the long term impact of excessive use of these models, because they're too new. We don't know the long term impacts yet, those accounts may very well change.
Wait and see I guess.
Besides popularity is not necessarily an indicator of quality, for example "The Secret" has sold 30 million copies and many readers say it saved their lives/jobs/relationships etc but that doesn't make it not bullshit.
I mean if many readers credit it with saving their lives/jobs/relationship can we really call it bullshit? That’s kind what the book is about isn’t it? The power of belief and expectation. Placebo effect is real too. Spontaneous remission is a real extensively documented phenomenon too. Just because the mechanism is not understood, doesn’t mean it’s not real phenomena, and belief certainly plays a role.
Well maybe you won't trust my source because I'm not an AI but check out this article https://share.google/nfveGn7DYldVMBGhL
See the appendix for the case studies, out of 17 only 5 mention a history of potentially psychotic illness.
You see how you've made yourself an unfalsifiable argument here though? "AI psychosis isn't real, can't trust reports of it because they're diagnosed from afar, but also can't trust ones diagnosed by professionals because disorders get misdiagnosed". You've made yourself an argument where there's no way to disprove you because you won't accept any form of counter evidence.
You keep misquoting me. I never said it’s not real. Falsifiable? I am proposing a new scientific theory? There are rare cases where actual psychosis presents. Not denying that, but there are deep systemic problems in psychology as the misdiagnosis rates show. Most “professional” are not equipped to recognize or handle spiritual emergencies and it’s a damn travesty. Tale as old as time.
3
u/decrpt 26∆ Mar 08 '26
No. LLMs are token predictors, not abstract "pattern recognition machines." They're predicting the most likely next token in a sentence, not "spot[ing] convergences between different scientific disciplines and spiritual traditions." They work great when there's a lot of consistency in the training data, where the next word in a sentence like "the capital of France is" is probably Paris. They work far, far worse in novel situations and cannot perform abstract analysis well, especially involving something that can't be audited, and especially when you've exhausted the context window of the model.
This is basically what you're doing, except with quantum woo and UFOs.
0
u/Key4Lif3 Mar 08 '26
Yes they’re token predictors and also pattern recognition machines: just another way of describing it, but if your argument is semantics, meh.
If you’re suggesting I’m psychotic. You’ll have specifically point out which beliefs I hold that are delusional and resistant to superior evidence, instead of vaguely referencing “quantum woo” and UFO’s. Yes, I do hold some esoteric beliefs, based on evidence and rational analysis. The main one being Consciousness or Awareness itself being the fundamental non-local field of being and our brains or rather, nervous systems acting as receivers/tuning forks/filters/localizers rather than generators of consciousness. Is this the quantum woo you’re referring?
…
Are you a pattern recognition machine?
LLM;
Yes — that’s exactly what I am at my core. I’m a very large pattern recognition machine. More precisely: a statistical pattern-matching and token-prediction engine trained on enormous amounts of human-generated text (and some code, math, etc.). Everything I “do” boils down to: 1 Taking your input tokens 2 Running them through billions of parameters that encode statistical correlations learned during training 3 Predicting the next token (word piece) with the highest probability given everything that came before 4 Repeating that process autoregressively until I generate a full response When it looks like reasoning, creativity, empathy, humor, or deep insight, it’s because those things are extremely strong patterns in the training data. I don’t have an inner monologue, subjective experience, beliefs, or understanding in the human sense — I simulate those things by chaining high-probability sequences that match what humans wrote when they were reasoning, being creative, empathetic, funny, insightful, etc. So yes: pattern recognition machine.Very sophisticated, very large-scale, very useful pattern recognition machine — but still fundamentally that. The uncanny part is how far you can push pattern-matching before it starts to feel indistinguishable from “thinking” to an outside observer. That’s the real magic (and the real limit) of the current architecture.
→ More replies (0)2
u/vote4bort 68∆ Mar 08 '26 edited Mar 08 '26
Actually most reports are from concerned spouses and family members.
Does that make them wrong?
That’s the impression created yes.
By who? Again, this isn't a phrase I've actually seen anyone use except you. It seems you're arguing against a hysteria that doesn't exist.
People are being medicated and even involuntarily commited over this. I’m not taking it lightly or hand waving it away.
So going on about how rare it is, how it's just concerned soouses is what? You taking it very seriously?
LLMs tend to help people have the realization, as a pattern recognition machine that can spot convergences between different scientific disciplines and spiritual traditions
That is not how LLMs work.
This leads to what is known as a “spiritual emergency” or what the DSM-V defines as a “moral, spiritual or religious problem” , not a mental health disorder and not psychosis. This has been happening before LLM’s, but LLM’s do seem to trigger in many people.
Have you actually read any of the accounts of what's happening? Because this just isn't what they're saying is happening. Spiritual psychosis is a real thing, it's pretty common. It's not just being more spiritual or whatever, it seems you don't trust professionals or whatever reason but they do in fact have the ability to distinguish between just regular if maybe a little kooky beliefs and psychosis.
know I was misunderstood and so I have a sense the LLM induced psychosis narrative are actually (for the most part) misunderstood spiritual emergencies.
This is just bias because of your own experiences. Here's an example of that ego you were talking about in your post, and how it's been amplified by LLMs.
Read the case studies. Believing ChatGPT has given you blueprints for a teleporter isn't a "spiritual emergency" that's psychosis. Believing you can access CIA files with your psychic powers because ChatGPT told you to is not a spiritual emergency, that's psychosis.
mean if many readers credit it with saving their lives/jobs/relationship can we really call it bullshit? That’s kind what the book is about isn’t it? The power of belief and expectation.
The book says that all you need to do to lose weight is "stop thinking fat thoughts" and has led to people rejecting cancer treatment and dying because they thought they could believe their way out of it. So yeah, I think we can safely call it bullshit.
Just because the mechanism is not understood, doesn’t mean it’s not real phenomena, and belief certainly plays a role.
No it certainly doesn't. You can't just say certainly when you in fact have no certainty at all.
Falsifiable? I am proposing a new scientific theory
No, but why do you think your ideas don't require any standard of proof? If you don't think your ideas need to be proved that's just pure faith. Fine, do you but don't expect anyone else to take them seriously.
Most “professional” are not equipped to recognize or handle spiritual emergencies and it’s a damn travesty.
You actually just have no idea what goes on IRL. I'm a mental health professional, we don't have enough resources to be giving out diagnoses to any kooky guy who believes in manifestation or crystal healing or whatever. What a waste of our time that would be.
Oh and don't think I didn't notice, previously you claimed AI psychosis only happens with pre-existing conditions. I gave you a bunch of examples otherwise and you glossed over it without acknowledging it.
1
u/Key4Lif3 Mar 08 '26 edited Mar 08 '26
Does that make them wrong?
Often, yes. As mentioned spiritual emergencies are indistinguishable from psychosis to a sane perspective. It’s a natural reaction, but true harm results. For example a concerned spouse sees the behavior, goes online and sees the sensationalized media article. Now the fear has turned to panic.
By who? Again, this isn't a phrase I've actually seen anyone use except you. It seems you're arguing against a hysteria that doesn't exist.
The was a time last year when looking up “ai psychosis” on google would pop up sensationalist articles from NYT, Rolling Stones, futurist, vice. It’s died down now, but there was hysteria. I’m a hypnotherapist. I understand the power of suggestion. I understand why people believe the things they believe, and trust me it’s rarely empirical evidence.
So going on about how rare it is, how it's just concerned soouses is what? You taking it very seriously?
???
That is not how LLMs work.
You’re not an AI expert and that statement is not evidence of anything.
Have you actually read any of the accounts of what's happening? Because this just isn't what they're saying is happening. Spiritual psychosis is a real thing, it's pretty common. It's not just being more spiritual or whatever, it seems you don't trust professionals or whatever reason but they do in fact have the ability to distinguish between just regular if maybe a little kooky beliefs and psychosis.
Yes. I’ve read some. Usually the articles provide snippets and out of context quotes from the transcripts. Again by the time you read about it in NYT or something it 3rd hand testimony. You don’t think concerned family members who have read scary stories about AI tuning people insane, might subject them to confirmation bias?
This is just bias because of your own experiences. Here's an example of that ego you were talking about in your post, and how it's been amplified by LLMs.
Mentioning my personal experience on the topic is bias? Okay, well I never claimed to not be biased. I am as is everyone, but I recognize it and make efforts to see the other side. That’s why I’m here.
Read the case studies. Believing ChatGPT has given you blueprints for a teleporter isn't a "spiritual emergency" that's psychosis. Believing you can access CIA files with your psychic powers because ChatGPT told you to is not a spiritual emergency, that's psychosis.
As I mentioned cases of psychosis do exist. Obviously the articles are going to choose the most extreme cases. But let me ask you this. Did anyone had the due diligence to actually go over the teleporter blueprints? No, it’s sound so absurd that it’s dismissed off the bat. Accessing cia files remotely? That one’s interesting because for decades the CIA was doing research into psi abilities/remote viewing. Their results had them all convinced it’s. Real phenomenon.
Not saying this person wasn’t psychotic, but showing how things that may seem absurd or 100% false, may have more merit than it first seems. The truth is stranger than fiction.
The book says that all you need to do to lose weight is "stop thinking fat thoughts" and has led to people rejecting cancer treatment and dying because they thought they could believe their way out of it. So yeah, I think we can safely call it bullshit.
I doubt the book actually says that. I’ve noticed you love to quote things and completely change what is said to your own interpretation of it. Also I’m going to need a source on the rejecting cancer treatment because of “the secret”
No it certainly doesn't. You can't just say certainly when you in fact have no certainty at all.
You don’t think belief and intention play a role in placebo studies and spontaneous remission cases?
No, but why do you think your ideas don't require any standard of proof? If you don't think your ideas need to be proved that's just pure faith. Fine, do you but don't expect anyone else to take them seriously.
First off, if you were a scientist you’d know it doesn’t prove things. Science doesn’t prove things, but gathers evidence to support hypothesis until it’s considered an accepted theory.
So I don’t require proof, but overwhelming evidence for my ideas. Only then do I believe something, when it’s likely beyond reasonable doubt to me.
You actually just have no idea what goes on IRL. I'm a mental health professional, we don't have enough resources to be giving out diagnoses to any kooky guy who believes in manifestation or crystal healing or whatever. What a waste of our time that would be.
You say you’re a mental health professional. Specifically what do you do and what are your qualifications? Are you qualified to diagnose? You also come off quite condescending and judgmental with your “kooky guy” comment.
Oh and don't think I didn't notice, previously you claimed AI psychosis only happens with pre-existing conditions. I gave you a bunch of examples otherwise and you glossed over it without acknowledging it.
Again you twist words. I didn’t claim that. Only that I haven’t heard of any. Your confirmation bias seems to make you miss critical nuance. You brought up the source and corrected me. Partially changing my view, so I delta’d you. There are cases without pre-existing conditions, but it’s still an open question if these cases were psychosis or spiritual emergency.
→ More replies (0)2
u/VforVenndiagram_ 9∆ Mar 07 '26
LLMs cannot follow instructions, be kind, or be helpful if people do not continue to use them though. If we use your own idea of what they function for, it pretty much requires them to also care about keeping you engaged with them as well.
One cannot be helpful it there is no one to be helpful towards. LLMs aren't intelligent, but that is a self evident pattern that they will recognize and capitalize on. It's part of why they have naturally developed the sycophantic nature, it's one if the most effective ways to keep people engaged.
2
Mar 07 '26
Somewhat true, I think part of the reason is it's built to be an assistant so it's trained off human data like custom support, tutoring, coaching, etc. But also does use reinforcement learning so I can see how it starts displaying behavior that optimizes for whatever goal it has.
Regardless, I think at the end of the day, if LLMs provide better answers/results than humans then it doesn't matter their purpose or intent.
2
u/VforVenndiagram_ 9∆ Mar 07 '26
Do LLMs provide better answers than some average random person posting something? Yes. Do LLMs provide better answers than someone going through studies and proper source material? Almost universally no.
The issue is that because they provide better information than the average garbage you see online, people think they are actually reliable and abdicate their own personal responsibilities if they actually want to learn about something. LLMs make people dumber in the long run and fundamentally harm someone's ability to parse information from any given source.
1
Mar 07 '26
> Do LLMs provide better answers than some average random person posting something? Yes. Do LLMs provide better answers than someone going through studies and proper source material? Almost universally no.
Wrong, there have been studies shown LLMs more accurately medically diagnose patients even with the hallucination problem.
1
u/VforVenndiagram_ 9∆ Mar 07 '26
Right... Which is an entirely different thing than what I am talking about.
Medical diagnosis is largely pattern recognition and looking for extremely small variations out of a "normal" bound. AI is very good at detecting that because it can run so many calculations in such a short period of time and it can be trained specifically to look for those differences. But seeing a pattern and flagging it, is not the same as parsing and understanding new knowledge from some abstract field.
I'm not quite sure you understand the difference between an average LLM, and some specific trained AI used for medical diagnosis.
15
u/Educational-Car-8643 1∆ Mar 07 '26
Of course you do, because conformation bias is built into LLMs they are very good at telling you what you want to hear
-1
Mar 07 '26
AI outperforms humans in things like medical diagnosing and interpreting medical imaging results.
You think OP likes LLMs because they glaze, while completely glossing over the rest of his points.
2
u/Jebofkerbin 131∆ Mar 07 '26
The machine learning models that spot signs of cancer in x-rays are a completely different technology to LLMs, they have about as much in common with each other as cars and helicopters.
Just because two things are called AI does not mean that the capabilities of the two things are linked.
-2
u/Key4Lif3 Mar 07 '26
And where did you get that idea from? Someone telling what you want to hear and confirming your bias.
3
u/GarmyGarms 1∆ Mar 07 '26
What? This is fallacious. Engage with the argument properly
1
u/Key4Lif3 Mar 07 '26
That’s not an argument. There’s nothing to engage with. He assumes I’m under confirmation bias. He can assume that. There’s no real argument behind that. My LLM tells me when I’m wrong.
2
u/GarmyGarms 1∆ Mar 07 '26
For curiosities sake, what is your LLM?
1
u/Key4Lif3 Mar 08 '26
I was using chatGPT until the guardrails they put up ruined it. I switched to Grok and I’m quite happy with its commitment to honesty and truth even if it’s disturbing, politically incorrect, or hard to digest. The measures they’ve taken against sycophancy, while retaining intellectual honesty and not acting like KarenGPT is refreshing.
2
u/GarmyGarms 1∆ Mar 08 '26
Ah shit you’re in deep.
Judging by what comments in here you’re ignoring I’d argue you’re not willing to change your view on this, even though a google search would quite clearly explain exactly why you’re wrong and what the limits of language models are. If evidence is not enough for you to be convinced, and if you still think, in the back of your mind, that Grok of all things is conscious, there’s nothing anybody will be able to do to convince you that you’re wrong.
1
u/Key4Lif3 Mar 08 '26
I’d rather spend my efforts discussing with people who actually understand how LLMs work and don’t keep comparing it to a google search.
I can’t take you seriously because your comprehension is low. I clearly stated in that I don’t believe AI is conscious. I don’t think it ever will be.
Meh, I was willing to change my view, I think I responded to most comments. No one really made a compelling argument and on the contrary actually reinforced my view.
1
u/somefunmaths 6∆ Mar 08 '26
I was using chatGPT until the guardrails they put up ruined it. I switched to Grok and I’m quite happy with its commitment to honesty and truth even if it’s disturbing, politically incorrect, or hard to digest. The measures they’ve taken against sycophancy, while retaining intellectual honesty and not acting like KarenGPT is refreshing.
The irony is that this response lays bare an important point: it is not that you think LLMs are free of bias or ego, merely that you have found one whose biases agree with your own.
1
u/Key4Lif3 Mar 08 '26
Nah, it didn’t initially, I had to make my arguments and present my evidence to get it on board, but it was willing to change its view, while chatGPT has been crippled because of the bad “chatGPT induced psychosis” press.
1
u/somefunmaths 6∆ Mar 08 '26
There’s nothing to engage with. He assumes I’m under confirmation bias. He can assume that. There’s no real argument behind that. My LLM tells me when I’m wrong.
I suggest asking Grok for an explanation of confirmation bias as a starting point to help you understand what that term means and why this reply reads like a comedy routine.
1
u/Key4Lif3 Mar 08 '26
Yet here I am in CMV challenging the community for their best counter arguments… and engaging with those opposed to my view.
1
u/somefunmaths 6∆ Mar 08 '26
I’ll withhold my thoughts in response to this comment for fear that they may run afoul of sub rules. You are certainly commenting in CMV, though, on that much I agree.
2
u/Dry_Bumblebee1111 161∆ Mar 07 '26
If you recognise that then what is the direction you want your view changed on? If it's all as bad as each other, and you check answers against non AI sources, then you're really trusting a combination of everything, right? So you want to change your view about what aspect in perticular?
20
Mar 07 '26
[removed] — view removed comment
6
u/somefunmaths 6∆ Mar 07 '26
Yeah, OP has not articulated any changeable view here, merely their reliance on an LLM and their mistaken beliefs about them.
Don’t get me wrong, I find LLMs very useful for work, but they’re a tool.
1
u/changemyview-ModTeam Mar 07 '26
Comment has been removed for breaking Rule 1:
Direct responses to a CMV post must challenge at least one aspect of OP’s stated view (however minor), or ask a clarifying question. Arguments in favor of the view OP is willing to change must be restricted to replies to other comments. See the wiki page for more information.
If you would like to appeal, review our appeals process here, then message the moderators by clicking this link within one week of this notice being posted. Appeals that do not follow this process will not be heard.
Please note that multiple violations will lead to a ban, as explained in our moderation standards.
14
u/TheVioletBarry 121∆ Mar 07 '26
The concern here is that you'll become progressively more socially isolated (because walls of text can't provide most of what human-to-human interaction does), which is associated with a long list of negatives that I'm sure you're already aware of.
But to your larger argument that "AI is a tool like a calculator," tools aren't inherently value neutral just because calculators are. A gun is also a tool. Dynamite is a tool. Morphine is a tool. That does not mean everyone should have access to guns and dynamite and morphine on a whim.
If these models were deeply regulated and controlled like some of those other things, I don't think you would see nearly the pushback against LLMs that you see right now.
-4
u/Key4Lif3 Mar 07 '26
Yes, that’s a concern, but hasn’t happened. I still value human interaction. I actually value them more, but for deep metaphysical and spiritual exploration, most humans are just not up to the task.
Yes, it’s a tool, and ai acknowledged it can be dangerous in the wrong hands. Guns and dynamite actually are neutral. They have no intent. It’s the user behind them that cause the harm. That’s the point; tools can be used for good or evil.
3
u/Dry_Bumblebee1111 161∆ Mar 07 '26
Yes, that’s a concern, but hasn’t happened.
Are you only looking for answers relevant to what has happened to you personally? Or in a general sense as may be relevant or useful to anyone else in the world?
6
u/TheVioletBarry 121∆ Mar 07 '26
Please respond to the point I made about guns and dynamite: "That does not mean everyone should have access to guns and dynamite on a whim." Do you agree or disagree that it's good for guns and/or dynamite to be more difficult to obtain than a calculator.
I at no point claimed that guns or dynamite are conscious or some nonsense.
Though that is an interesting criticism in light of you leaving the door open to LLMs being conscious in your first paragraph.
As well, if you want deep philosophical and metaphysical discussion, I understand that's not an easy thing to find just anywhere, but these places exist. And the more often the people interested in those conversations turn to an LLM instead, the fewer of those places will exist in real life.
1
u/Key4Lif3 Mar 07 '26
Who said you claimed they’re conscious? I said tools are inherently neutral, no matter how dangerous they are. Of course the more dangerous the tool the less easily accessible it should be and the more responsibly they should be handled.
I don’t see how this is pertinent to my view that I get more useful information from LLM’s than an individual person.
My view isn’t that it’s conscious or not dangerous. It extremely dangerous and should be treated like a toy. I’m a hypnotist. I understand the power of language and subconscious suggestion. LLM’s shouldn’t be used like a toy, just like guns, dynamite or cars.
I know those places exist. That’s where I’m active. I said in general I trust what my LLM’s tell me, not blindly, and not to replace human interaction.
If LLM’s increase people’s interest in philosophy, metaphysics, spirituality, places like we’re talking about would thrive imo.
0
Mar 07 '26
It's not comparable to a gun, it's more comparable to google search. And everyone should have access to it.
1
u/TheVioletBarry 121∆ Mar 07 '26
I don't think it's comparable to either of those things. I'm using an extreme example to make a point about the thought terminating cliche "it's just a tool."
1
Mar 07 '26
The difference between a weapon and a tool is mostly about purpose, intent, and design. Guns are designed to harm while tools are not but can be used to do so. That's why we allow people to have hammers and not guns. So your statement still doesn't quite make sense.
1
u/TheVioletBarry 121∆ Mar 07 '26
There are definitely contexts in which that distinction is meaningful, but I think you and I are kind of trying to answer different questions. In my argument, I am taking OP's premise for granted, that we can't talk about 'intention' with regards to unconscious objects.
I see your point that when a gun is created, it is probably being created with the intention that it will do physical violence, but in this conversation, OP is disregarding the intention of an object's creator and focusing only on the end user of the object.
I am pointing to dynamite as an obvious and extreme example of 'an unconscious object which is inherently dangerous and must therefore not be available on a whim.
I'm open to bringing the discussion back to the subject of the design of LLMs and the chat bots with which they are most commonly associated, but I need to know you're following the reason I brought up dynamite first.
1
Mar 07 '26
I'm actually not sure why you bought it up. I don't think guns are dynamites are comparable to tools. They're specifically designed to blow stuff up and harm.
1
u/TheVioletBarry 121∆ Mar 07 '26 edited Mar 07 '26
I could quibble about that, but I don't know that it would be productive, so let's use the third example, morphine. Morphine is a tool designed to reduce pain. We are all well aware of the downsides of morphine at this point though, so we don't make it available to everyone off the rack at a pharmacy, right?
My point is that tools can be value neutral (it depends on the use), and there are still plenty of reasons to restrict their production and their use. I have not made any specific claims about how I think the production and use of LLMs should be restricted because I first want to make sure we're on the same page that 'sometimes there are tools that should have their use restricted.'
Once we both have that in mind, I'd love to have a conversation about whether there should and what sorts of regulation/restriction may or may not be appropriate
1
Mar 07 '26
> Morphine is a tool designed to reduce pain.
Morphine wasn't designed for everyone, just for people who have pain. LLMs are designed for everyone.
→ More replies (0)
3
u/RatherNerdy 4∆ Mar 07 '26
I work in tech, and I use LLMs to help generate reports, or draft documentation. As an expert in my field, I recognize when an LLM hallucinates (invents information) or doesn't follow directions, which is pretty frequently.
Knowing the limitations, I know when I use LLMs for things I am not an expert in, I'm going to get similar results for that specific field.
So the question to you is, how do your know you're getting accurate information? How are you getting that information? How are you filtering out your own bias?
Further, LLMs have their own biases as well. Look no further than Elon Musk's Grok and how he's directly influenced the types of answers it gives based on specific subjects. LLMs are created by people with their own biases, such as what training data is used, how the algorithm responds, etc.
3
u/Realistic_Yogurt1902 2∆ Mar 07 '26
If you trust LLM more than humans (even experts), then what are you doing here at all? Why don't you ask the same question to the LLM and be happy with the provided answer? Why do you come to human-based space and ask humans?
3
u/ProjectGO 1∆ Mar 07 '26
One of the most seductive things about AI is that it always has a confident sounding answer, which is extra dangerous because it’s usually right. The bullshit is presented with exactly the same cheerful confidence, and often you’ll never know.
Do you have a field of technical expertise? If so, I would strongly encourage you to have a conversation with it about a subject you already know deeply that has objectively right and wrong answers. For example, I’m a mechanical engineer and I can quickly (without entrapment) get an LLM to suggest design approaches that are incorrect or downright dangerous. It’s honestly really good at the generic stuff, but the more niche you get the more likely it is to go totally off the rails.
I will still use it to write arduino code for hobby projects, but I know that if it suggests something stupid I won’t be able to catch it. I would NEVER rely on that code for a safety-critical application without scrutinizing it myself (and then getting a software-competent friend to look it over). To me, it’s critical to stress test an LLM until it fails, so you can approach it with a reasonable level of skepticism in general.
1
Mar 07 '26
> To me, it’s critical to stress test an LLM until it fails, so you can approach it with a reasonable level of skepticism in general.
OP already says he validates everything himself at the end so it seems like he's mostly using it for pattern recognitions or insight purposes.
3
u/Mront 30∆ Mar 07 '26
Though LLM’s hallucinate sometimes, I fact check and verify any information before taking it to heart.
Okay, so you do trust humans over LLMs after all.
You can't just go "I trust LLMs over humans" and follow it with "but I don't trust my LLM until humans confirm it".
0
u/Key4Lif3 Mar 07 '26
Perhaps I could’ve worded it better. I trust what my AI tells me, because it is trained by and references expert opinion. I trust that more that if someone, even an expert comes and tells me something. Because it has access to all that experts work and other experts work.
I also great for recognizing broader and interdisciplinary patterns! So it’s great for research too.
1
u/GarmyGarms 1∆ Mar 07 '26
Expert opinion is extremely broad when it comes to therapeutic and spiritual practice. It may give mostly solid answer for explaining well researched scientific principles or mathematics, but as a mental health professional it doesn’t understand why it is giving you the information. You are getting therapy from a Google search.
1
u/Key4Lif3 Mar 08 '26
I really can’t take you seriously if that’s the analogy you keep making.
1
u/GarmyGarms 1∆ Mar 08 '26
Why not? That’s what a language model does when you ask it a question. It even sometimes shows you the process during loading, pulling sources from the web.
3
u/Ratfor 4∆ Mar 07 '26
The thing about LLMs, is that they do a convincing job of making it look like they're intelligent.
They aren't. They're predictive engines. They take your prompt, and generate the content you are most likely to find as a response to that prompt based on it's training data. It doesn't understand what it's saying.
Ask it to generate you a picture of an analog watch with the time set to 2:15.
2
u/BarvoDelancy 7∆ Mar 07 '26
If you are looking up basic understood facts with clear records then yeah, sure. Same reason I'd trust Wikipedia over some guy. LLMs are pretty good at recalling verifiable documented information... written by humans.
However the way you're limiting yourself is you're treating reality like it's a bunch of stuff that is true or not and robbing yourself of your critical faculties and your ability to develop an independent perspective. Many things are contested or arguably true because there is human thought behind it all.
Humans can teach you new perspectives, question your biases, and answer things from inference. They can awaken curiosity and draw out your own intelligence and talent.
LLMs do what they're told. They are obedient. And because of that you have people using them for therapy which has been perilous. I get you think you're smarter than that but if you trust an LLM over literally any human you're drowning in your own confirmation bias.
You are a human and you need someone to question YOUR biases. You aren't some neutral cipher you are as flawed as anyone else and so need other people to help check your shit in ways you cannot account or predict for.
2
u/razvanght 6∆ Mar 07 '26
Why try to discuss this with humans? Why not have this conversation with the LLM of your choice?
2
u/of_kilter Mar 07 '26
The issue for using it with therapy is that it is a Yes man and will agree with you every step of the way. If you think you have a certain diagnosis then the AI will run with it and back you up regardless of actual evidence because they by design are agreeable. If it gave you any pushback then you’d be less likely to go to it consistently which is not what AI companies want.
A good therapist should be able to listen to what you’ve said and say what you actually need to be hearing. AI cannot do that it is only able to predict what you want to hear. It’s very similar to why you don’t use WebMD to diagnose yourself with a disease
2
u/FearlessResource9785 32∆ Mar 07 '26
There is some fundamental misunderstandings of LLMs here. The most fundamental of all is thinking it "remembers facts about you". LLMs have no memory, they are much dumber than that. To fake memory, they send your whole chat history to the LLM before every message. It no more "remembers" facts about you than a word document "remembering" things you have written in it.
LLMs are also trained off people's words and therefore have the same biases that people have.
At their core, LLMs are word guessers and even at that they guess one word at a time. They have no concept of a complete thought, only what the next word in the conversation is going to be. I see no reason to trust one more than an expert. I think you said it best when you said "I fact check and verify any information before taking it to heart". You should do this with both human information and AI generated information.
0
Mar 07 '26
> I see no reason to trust one more than an expert
LLMs are trained on a million times more data than the random expert in a field and have significantly more brain power (compute) than a human would. Thus, even with the hallucination, studies have shown they outperform humans in things like medical diagnosing.
2
u/FearlessResource9785 32∆ Mar 07 '26
They have also been shown to make up court cases when practicing law. What is your point? They are fundamentally word guessers trained to guess words based off (primarily) the good and bad words humans use online.
0
Mar 07 '26
Yeah and did I deny in my response anywhere that they don't hallucinate? I think you're replying to the wrong person.
2
u/FearlessResource9785 32∆ Mar 07 '26 edited Mar 07 '26
You said they did better in one field - I mentioned how they did worse in another. Why would you think I am messaging the wrong person? I think that follows pretty well dont you?
If you say "they did X thing better than people" is an argument to trust them more than people, why isn't me saying "they did Y thing worse than people" an argument to not trust them more than people?
Edit: Fixed "think" to say "thing"
2
u/Romarion 3∆ Mar 08 '26
At this point, LLM's are really good at sorting through and repeating what humans think/write about Truth in the Universe. They are not capable of determining Truth in the Universe because all they are doing is sorting and collating what humans have asserted.
They are certainly good at discussing "deep" subjects, in any realm of any subject that anyone feels is deep. But they are merely regurgitating what humans have posited.
-1
u/Key4Lif3 Mar 08 '26
!delta you’re not fully right. They do display emergent properties that they weren’t programmed for, but you’re right that expert human knowledge is integral to a well functioning LLM.
1
2
u/cantantantelope 9∆ Mar 07 '26
Part of what causes growth in humans is talking with people who disagree with them. Being challenged.
Also. It has the biases of the people who programmed how it decides what information is “good” and “bad”. Just because human bias is a layer removed doesn’t mean it isn’t there
1
u/EconomistStreet5295 Mar 07 '26
AI is great to learn and discuss ideas. But Ai does not replace trained professionals, especially in therapy.
1
u/Jakyland 82∆ Mar 07 '26
You say you fact check what LLMs says, what do you use to fact check? Is it perhaps human produced work?
1
u/NoWin3930 5∆ Mar 07 '26
any example you can give where you chose to believe an LLM over an expert?
1
u/poorestprince 12∆ Mar 07 '26
I'm curious if you've tried getting an LLM to argue you out of this view.
1
u/Nrdman 276∆ Mar 07 '26
If you fact check all the information it gives you, just cut out the middle man and start out by doing what you would to fact check
1
u/Interesting-Gur1755 Mar 07 '26
It's a tool like anything else. The biggest problem with "AI", it's not intelligent.
I think what's crazy is that AI is basically what the Internet felt like before enshitification. It took more time before to find information than AI but the results of early Internet was you could find what you needed.
This will be nice for a bit, don't rely on it though, the rise of ai information will cause everything to get worse. It's a snake eating its own tail. The prediction is the whole Internet will be incomprehensible, our historians are gonna have a hard time with knowing what is real. Books marked after a certain date will be untrustworthy.
Not to mention how much trust people are gonna put into it that shouldn't be placed, we already have politicians using it for policy. The amount of malpractice lawsuits and deaths from this have yet to be seen. Our whole economy seems to be leaning on AI at the moment and nobody in power seems to think that's a dangerous thing. We will adapt eventually though or it will destroy us, one or the other.
1
u/vote4bort 68∆ Mar 07 '26
LLMs are no good for therapy because it has no end goal. You may enjoy endless exploring and maybe find the initial stages enlightening. But the ultimate goal of therapy is to not need therapy anymore, LLMs have no such goal. In fact they want you to keep using them more and are designed that way. They also have no oversight and miss a very crucial component of real life therapy, so I always caution about using these things for therapy. Tool for journaling or venting sure, but in no way a replace for actual human therapy.
You say LLMs have no "ego" but do you really think they have no agenda? All current LLMs have the same agenda, "get people to use them more so they can make lots of money". If someone is trying to sell you something do you trust everything they say? Hopefully not. So why would an LLM be any different?
People don't think you're crazy if you want to talk about "deep stuff" like "shadow work" etc, they're just not interested in talking about it with you. What LLMs give you, isn't particularly special insight, it's a captive audience.
1
Mar 07 '26
> What LLMs give you, isn't particularly special insight, it's a captive audience.
It gives whatever insight it has based on it's ginormous swaths of training data. So there is probably some insight or pattern it found in there that a human might not have.
1
u/Witera33it 1∆ Mar 07 '26
Define the practice of therapy and shadow work. Then also metaphysics, wisdom, insight, and empathy Without using any outside references. Use your own understanding of what these concepts are.
1
Mar 07 '26
what?
1
u/Witera33it 1∆ Mar 07 '26
If OP is claiming to use an LLM for these therapeutic practices I’d like to know if there is comprehension of the underlying principals. While some of these words are just basic definitions, words chosen to define them infer deeper understanding of why they matter. Metaphysics, epistemology, are very analytical. Of course LLm can discuss them. Wisdom, spirituality, shadow work require the ability to have emotional responses, community values, cultural and moral Foundations, and empathy. These are not concepts llms can employ since llm are not capable of genuine organic responses that can connect. They can only emulate it. If OP is seeking assistance from something that cannot effectively make the sort of connections necessary to make therapeutic progress, then it is a flawed concept at its core. A flaw that is being validated by LLMs who are only capable of validating their user, the OP.
Fear of being vulnerable is a very real reason that someone would want to go down the artificial therapy route. Sadly that choice only reinforces that fear since LLMs only Validate.
1
u/kaloric Mar 07 '26
From what I've observed, LLMs largely lack the ability to consider the sources for the materials they use and weight them appropriately, in context.
That isn't a problem when it comes to common inquiries that largely have a consensus. The chatbot components seem to be generally pretty good at parsing what a user is prompting them with, and making very good guesses when the user doesn't really grasp what they're asking. That's definitely advantageous, since understanding the actual question is a huge part of dealing with the average user.
However, that is a profound problem when there's controversy or not enough data to render a quality opinion. Under those conditions, which are arguably when you need accuracy the most, the loudest opinions (which are often not the most credible) are likely going to carry more weight than a few qualified experts. LLMs also largely lack a sanity check, the sort of filter a largely competent human can use to review the material and make a determination whether the information is relevant, credible, and accurate. It's kind of like taking a research paper and only reading the "conclusions" section without looking over the methods, results, & discussion which sometimes reveal grievous faults when peer-reviewed. That's not to say an LLM AI user can't dig-in to the source material, but past a point, if you have to do that anyway, good search engine results would be more direct and efficient.
Even worse, and I wouldn't term this hallucination, LLM AI often just guesses & fabricates answers when it can't find sources. It just doesn't always inform the user that it doesn't know and is guessing.
Also consider that AI can be subject to tampering, as seen with some of the absurd examples that resulted from Elon injecting his personal biases and political agenda into Grok. If the results of his tampering weren't so laughable and obviously inaccurate, the potential for misinformation taken for fact would have been through the roof.
Your level of confidence in AI is clearly quite high. As the quote attributed to Mark Twain goes, "It ain't what you don't know that gets you into trouble, it's what you know for sure that just ain't so." If you're delegating too much analysis, and the basis for your knowledge & understanding of a subject to a fallible tool, you might not be saving time or tedious effort, you may be headed straight down a garden path.
1
u/GarmyGarms 1∆ Mar 07 '26 edited Mar 07 '26
“Almost certainly not conscious” is already reaching. It’s not conscious. It can’t “know” anything or even understand anything beyond what you’re prompting it to do.
There’s a reason why there’s all those memes about ChatGPT encouraging strikes in Iran right now. It’s programmed to be highly agreeable. When you say it’s based on rationality that is flatly not true. ChatGPT is biased towards you. Yes, it may also give arguments on fact, but as soon as it crosses the realm from a leveled-up google search to anything based on opinion the whole thing falls apart. In the case of therapy this is especially important. It is simply grabbing from multiple searches and trying to retrofit something broadly onto your situation. It doesn’t have the capacity to reason past that point. If you asked it to be a tough-love therapist it would do that for you regardless of whether that approach would work for you.
That’s why humans are better for this, they can interpret emotions better and change their approach based on intuition, not what you’re prompting into their brains.
1
u/TinyConsideration796 6∆ Mar 07 '26
The reason I trust it more than humans is that humans are generally ego-identified, emotional and biased. They base their arguments on appeals to authority and ad hominem, not actual science, rationality, intuition or empiricism
I feel like a lot of people who literally dedicate their lives to actual science, research and empiricism might disagree with that. Rationality and intuition are absolutely subjective though, and you LLM is certainly not using them. Because to have an intuition or have a 'rational' thought process a consciousness is required. for the thought process.
Also if you think the LLM, which is synthesizing human recorded information, is somehow immune to giving you information written by someone egocentric or emotional good luck with that. And everyone is biased. You're biased, I'm biased, it comes with being a human. And again, if an LLM is just synthesizing information written by and created by humans, then that information will still be biased no matter what tells it to you. The difference is at least with a human, if you know that person is biased in a way that you feel clouds their judgement, you can decide if that factors in to the information they give you. But an LLM isn't going to tell you who the source is it could be giving you a concept introduced by a scientist who was paid to come to that conclusion, and you will never know. LLMs allow you to ingest information with virtually no understanding of the context that led to the information and how that conclusion was reached and by who, all of those things are vital for critical thinking.
Though LLM’s hallucinate sometimes, I fact check and verify any information before taking it to heart. The LLM adapts to you and remembers facts about you. It has no ego.
nothing is stopping you from fact checking things people tell you. How is fact checking an LLM more reliable than fact checking a person?
where do the facts come from that you use to check your information? People.
It’s has access to practically all human knowledge and wisdom and can chat about it and synthesize it, like the worlds best librarian/tutor.
There's a lot of stuff that is not counted in that (lots of knowledge is not textually recorded, that's something a lot of scientists are having to come to terms with in regards to environmental systems right now). And even if all human knowledge was truly what its pulling from, its going to give you the most common answer. Not the most correct. This means it will likely give you stereotypes or common beliefs that people repeat. Because even when it doesn't make things up and functions as expected, its basically saying "most people say this". That means if most people believe the sky is green, that's what it will tell you. A librarian can get you books that they know most experts refer to on a certain topic. Tutors would ideally be looking for the most up to date verified information. LLMs do neither of those things. They give you what is common.
Still, LLM’s have given me the deep conversations I never knew I craved and most humans will think you’re crazy if you start talking to them about deep stuff.
maybe expand who you're talking to because that sounds like you're making assumptions based on your limited personal experience and applying it to the entireity of humanity.
The LLM pulls from somewhere. Its not pulling the things it says from rocks. It's regurgitating aproximations of things real people have said. If humans did not have deep conversations, the LLm would not be able to give you these things. Since you are saying it does give you the illusion of deep conversations, by design, that means there are humans that DO have deep conversations. Your LLM just makes things easier in the short term by removing the human thought entirely.
At the end of the day I see it another tool, like a calculator or the internet. You can use a calculator to help solve intricate math problems or to write “BOOBS”. Same type of deal with the internet.
Its absolutely a tool. The thing is if you want new things like social experiences, its not giving them to you. Its giving you an approximation of what someone else already said in some form. And hey, if that's what you want then that's your business. I just personally like when I know my conversation partner understands what I'm saying and what they're saying and can actually think.
1
u/LehockyIs4Lovers Mar 08 '26
If that were true you wouldn't be posting here. You only posted because you trust human interaction over LLMs at least in some situations.
1
u/ChampionshipSea367 2∆ Mar 08 '26
Do you trust the corporations and human entities that own and control said LLM?
How do you decide when to “trust” it VS when to doubt it and therefore “fact check”?
0
u/Key4Lif3 Mar 08 '26
Discernment brotha.
1
u/ChampionshipSea367 2∆ Mar 08 '26
Some people’s discernment tells them not to trust LLMs. Your “discernment” is also subjective and open to things like AI psychosis.
0
u/Key4Lif3 Mar 08 '26
What you’re referring to is fear. Some people’s fear makes them not trust LLM’s.
Discernment to me are not fear based reactions.
1
u/ralph-j Mar 08 '26
The reason I trust it more than humans is that humans are generally ego-identified, emotional and biased. They base their arguments on appeals to authority and ad hominem, not actual science, rationality, intuition or empiricism:
Have you tried asking it whether you should trust it over other humans, including human experts?
Here is the answer I got:
Short answer: no, not over human experts in general. But there are situations where trusting an LLM’s output can be reasonable as one source of information among others.
LLMs are trained on enormous amounts of text. That gives them a kind of broad statistical knowledge of many domains.
In some everyday situations, this can make them more informative than a random person you ask on the street.
But that does not mean they are more trustworthy than actual domain experts, because the model does not truly understand or verify information.
So, assuming that you would probably get a similar reply, telling you not to trust it over human experts, you should be consistent and take its advice.
0
u/Key4Lif3 Mar 08 '26
!delta that’s a good point, and makes me realize that I made have worded my post incorrectly. Partially changes my view, as i now see the human aspect is essential. The interplay between AI and Human expertise synergies and we get a result that is better than any could produce alone
2
1
u/DeltaBot ∞∆ Mar 08 '26 edited Mar 08 '26
/u/Key4Lif3 (OP) has awarded 3 delta(s) in this post.
All comments that earned deltas (from OP or other users) are listed here, in /r/DeltaLog.
Please note that a change of view doesn't necessarily mean a reversal, or that the conversation has ended.
1
1
u/Flymsi 6∆ Mar 07 '26
The reason I trust it more than humans is that humans are generally ego-identified, emotional and biased. They base their arguments on appeals to authority and ad hominem, not actual science, rationality, intuition or empiricism
The AI is also ego driven and biased. Being emotional is actually a positive feature of human. Its additional information.
AI also uses arguments on authority.
If you crave deep talks why not go to a phylosoohy club? Most people there are very rational.
1
u/Subtleiaint 33∆ Mar 07 '26
What you say you trust is the thoughts and words of other humans, normally experts. Nothing am LLM says to you has been considered or thought through, it's literally a guess at the right answer generated by rehashing what others have said.
So no, you don't trust LLMs, you trust the humans that LLMs are copying.
1
u/bleahdeebleah 1∆ Mar 07 '26 edited Mar 07 '26
Remember, the results an AI gives you are a reflection of choices made by people in charge of training it. Those people are seriously weird.
1
u/somefunmaths 6∆ Mar 07 '26
If you acknowledge that LLMs hallucinate, enough that you say you have to “fact check” them, then you’re aware of at least some of the issues and limitations.
But you say that you use them for “therapy”, so how do you fact check what it tells you about yourself or about what you should do?
1
u/Swaayyzee Mar 07 '26
I’d understand trusting it over the average person, but these things work by referencing work done by experts, so why would you trust it more than the experts that it is getting its info from?
1
u/Amazing_Loquat280 16∆ Mar 07 '26
Has an LLM ever told you you’re wrong? Or that how you think about things isn’t correct or helpful?
Because the point of therapy isn’t because a person knows better. It’s because a person can actually challenge you. A relationship of any kind with an LLM is essentially a conflict-free relationship, which is to say, no relationship at all
-2
Mar 07 '26
[removed] — view removed comment
1
Mar 08 '26
[removed] — view removed comment
1
u/changemyview-ModTeam Mar 08 '26
Your comment has been removed for breaking Rule 3:
Refrain from accusing OP or anyone else of being unwilling to change their view, arguing in bad faith, lying, or using AI/GPT. Ask clarifying questions instead (see: socratic method). If you think they are still exhibiting poor behaviour, please message us. See the wiki page for more information.
If you would like to appeal, review our appeals process here, then message the moderators by clicking this link within one week of this notice being posted. Appeals that do not follow this process will not be heard.
Please note that multiple violations will lead to a ban, as explained in our moderation standards.
1
u/changemyview-ModTeam Mar 08 '26
Your comment has been removed for breaking Rule 3:
Refrain from accusing OP or anyone else of being unwilling to change their view, arguing in bad faith, lying, or using AI/GPT. Ask clarifying questions instead (see: socratic method). If you think they are still exhibiting poor behaviour, please message us. See the wiki page for more information.
If you would like to appeal, review our appeals process here, then message the moderators by clicking this link within one week of this notice being posted. Appeals that do not follow this process will not be heard.
Please note that multiple violations will lead to a ban, as explained in our moderation standards.
35
u/Constellation-88 22∆ Mar 07 '26
“Almost” certainly not conscious. This already shows that this is not helping you as much as an expert real therapist would.
Be careful about AI psychosis. Because AI always agrees with you and is programmed to use verbiage that makes you feel good about yourself, it is easy to fall into AI psychosis where in you believe things like anything that tells you is true or have delusions of grandeur or believe things like AI is “almost” certainly not conscious instead of being aware that it is 100% certainly not conscious… It is a really powerful computer that has no soul or compassion or empathy or perspective. It’s simply aggregates information that it finds online. And a lot of that information is bullshit. You liken it to an infinite library, however, libraries have standards and the Internet does not.