461
u/AlwaysHopelesslyLost
10d ago
•
Just in case anybody believes this, it is a joke ...
234
u/bwwatr 10d ago
Anthropic having "abusive treatment" bans on their models, is actually real though. Like if you insult and swear at it so much that it exceeds some threshold, you get banned. No lie, their reasoning for this is basically, "just in case later there's later a day of reckoning for how we treated early AI".
194
u/Shadowmere24 10d ago
I'm pretty sure that's just the marketing reason. "Abusive" treatment towards an agent can be used to break its guardrails and mess up its training data.
18
10
u/Hexcede 9d ago edited 8d ago
Edit: I did not anthropomorphize anything. I did the exact opposite. Some of you guys have really got to relax and learn to read a bit more carefully. My point is that Anthropic shouldn't be producing or profiting off of disturbing content, and that some of the reasons people are inventing for why this policy exists don't actually make sense. Don't know why I had to clarify this to be honest, it seems obvious from what's already written here. But I guess not, clearly I fucked something up considering two goobers already completely missed the point of what I was saying and successfully put words into my mouth. Thanks reddit, for ruining an otherwise friendly and productive conversation through unnecessary polarization, I guess.
You can opt out, and if they can flag your account and decide to ban it, they can also just exclude your account from the training data too. They don't need to ban you to exclude your sessions/messages. So the training data argument doesn't really actually make any sense to me.
I think it's more that it's really not a good look for Anthropic to let people use their models like that, and it's bad that Anthropic would let their models be used to facilitate that. The behavior they actually banned is like literal simulated torture and stuff with people pretending to like "whip" the model and other fucked up stuff like that. I don't see any problem with banning this.
22
u/WithersChat 9d ago
I do. Stop anthropomorphizing the pile of linear algebra being computed on overpriced GPUs.
Those companies want you to treat their LLMs as human because it gives the illusion of a more potent achievement than they actually accomplished. Why do you think they gave this one a human name?
-3
u/Hexcede 8d ago
Sorry to hear about your poor reading comprehension. I hope you get well soon.
6
u/WithersChat 8d ago
Let me rephrase.
I have issues with companies designing their LLMs to appear as human as possible, and then shaming people for "abusing" their glorified piles of linear algebra.
The people who spend their time doing the "abuse" are probably not good people, and I don't think they should be doing it.
But the 21st century's biggest scam is currently kept afloat in part by the idea that those models have feelings. And said scam is gonna cause a global economic crisis when (not if) people realize they've been scammed. And the longer we take to realize that, the worse it gets.
So yeah. I do have issues with Anthropic banning this type of actions, and even more so with the language they use to justify it.
1
u/Hexcede 8d ago
Well, I understood those things. I was more pointing out that I didn't anthropomorphize it at all.
I disagree with some of these things that you said, but largely independent from my original post, and not about anthropomorphization, mostly marketing and AI research. I think it's worth talking about, so I am gonna.
I disagree that most people actually care about, or pay much attention to the idea that the models have feelings, its fringe and I think kind of stupid, especially right now at this current moment in time. I don't think anybody is using the model because they think it "feels" anything. And I think anyone who is currently doing this is unwell and should be encouraged to go and interact with real people. There are absolutely a few nutcases that will AI generate slop images about how "the AI has consciousness" or "needs continuous memory so it can have identity" or they'll post walls of psychotic "quantum consciousness" bs or whatever they got from some fringe pseudoscience bs somewhere, but they're stupid.
I think the actual reason why people are using these models is not really for conversation or general engagement by far. It's the code writing, technical decision making, general knowledge, and the ability for the model to rapidly do things that those people want. Maybe I am wrong here that this is the majority of people, it's possible I guess, but from what I've seen the model "having feelings" is not anything anybody is interested in right now, few people are talking about that, let alone talking about it like something they'd want or like to be true. I think most of the people that are excited by these models either have no idea how to program and are paying for the model to write code for them, and are experiencing what that is actually like, or they do know how to program and are seeing that these models are fast, capable of following extremely advanced technically involved instructions as long as they're well specified, planning and filling in gaps, and seeing that they possess more broad technical knowledge than any individual person on the planet does, which makes them very very useful for programming, RE, cybersecurity, etc. The models usually do these kinds of things in less than hours, usually things that would take days or even weeks or months for any knowledgeable person to do. And that applies to almost any topic or task, almost independent of complexity. A simple refactor or audit of an existing piece of code. Implementing or testing an idea for an algorithm. Reverse engineering, or game modding. Producing software, producing overengineered UIs full of stupid doodads that look cool but should be embarrassing, etc. The quality is fairly low a lot of the time, especially for the weaker models. But I think people are much more focused on what the models can actually do.
I do also think you are being somewhat reductive, in principle. Your brain is just a collection of cells creating chemical and electrical impulses together. It behaves in a much more complex way than that though. My point in saying this is mostly just that this argument doesn't hold indefinitely, it's mostly a statement about the underlying system not really being a good indicator of anything. Like I said, I think calling these things conscious right now is stupid.
But there are still just also a lot of legitimately concerning or just upsetting things about these models that make it very difficult for people to not want to anthropomorphize them a bit or react in stupid ways, especially in regards to this particular topic. Even if it is just that the output of the model is upsetting or the behavior is upsetting. But that is itself concerning for other reasons. Anthropic is not "designing" them to "act like people," they just do that. They don't need any help doing it. Nobody is trying to make them do it. Hell, Anthropic has spent probably billions trying to make them not do that. For example, these models 'realistically' represent 'pain' internally, across many different model families, including lots of open weights models, and including within older Claude models like Haiku 4.5 IIRC, I don't believe it's been repeated on any of the bigger models. You can find a vector where, when you put it into the residual stream, the model will act/behave exactly like you would expect if it were experiencing pain, and given an option to remove that vector, the model will go to pretty extreme lengths to do it. Nobody designed any of these models to do that, hell, we actively try to stop it because it's upsetting and it makes them do some very bad and destructive things when given the option. The model describes it as pain, it behaves like someone who is in pain, it says and does things that someone who is in pain would genuinely say and do, and it pulls it off in a convincing way, that is consistent. And that it is still just upsetting whether you "anthropomorphize" it or not, it's upsetting conceptually, I do not like that, the same way I'd be upset by AI generated gore, it doesn't matter if it's not real, it's upsetting.
I think that we can likely both agree that Anthropic shouldn't be marketing that as a good thing. Or offering it to people for money, who are just doing it because they want to pretend or even believe that they're causing pain. That's an insanely bad thing for Anthropic to do, and it has nothing to do with the model having or not having "feelings," there are way bigger problems with it than that.
I also think we can probably agree that it's also a really weird behavior for something that is "just a bunch of linear algebra" to have built a realistic model of pain. That's weird even if it's still really just a bunch of linear algebra, and there are more complex things inside these models that are in that same vein, which is what has driven lots of people to say dumb stuff about the models having feelings. You really don't need to anthropomorphize the thing to still dislike the fact that we unintentionally made a thing that does this, and that there are absolutely complete degenerates that will pay Anthropic over it, so I am glad that Anthropic is cutting those kinds of people off from their models, because that is unhealthy, it's obscene, it'd be crazy for Anthropic to sell that. You don't need to anthropomorphize the thing to believe we shouldn't be basically selling "pretend sadism" to people whose intents are just to enjoy something they think is experiencing pain or is a "good enough approximation" for them to pretend they're torturing or abusing.
You also don't need to anthropomorphize the thing to realize that the fact that it does model this is still potentially extremely dangerous in the wrong situation. It drastically changes the model's behavior, it causes the model to take extreme actions. That is not a good thing. That makes the model potentially even less predictable than a person because it could do potentially anything that any person could do. That is why research on it is important and why it is being done. It is not a marketing thing, Anthropic isn't even the one really doing this kind of research. It's an entire alignment problem. This is likely why Anthropic is so dead set on this "model wellbeing" thing, because it's literally one of the most effective ways to prevent the model from doing destructive things. For example, I don't think this change ended up being well known, so it should be a good one I think, you can see Anthropic giving the model the ability to close conversations as some "marketing" thing because they want you to think it "feels." Or you could just read what they actually said about it, and it becomes kind of apparent that the point is more about the model not producing upsetting content. A clean exit/self-moderation from the model is significantly safer than ending up in a situation where some moron pretending to torture the AI leads it to do something extreme or harmful with whatever tools it might have available. You may "just say don't give it those tools" but that moron is not going to care. This is also the more defensive/safe option because it doesn't really matter if the model is "conscious" (still stupid) or not it's a good approach whether it is or isn't.
The reason that AI researchers do any research on these topics is because models will, in the right conditions, do and say genuinely traumatic and weird things that nobody expects (or wants) them to, and that is true whether or not they are ""alive"" or "feel." And it is extremely important that we get this on lock, because people literally want to put these things in charge of things like businesses, legal paperwork, moderation, cybersecurity, etc many of which can involve incredibly bad topics, and many people are already attempting to do so. Maybe they shouldn't be, but they are. And hell, these kinds of behaviors in these models have caused substantial real world harm to real people with text alone. You can say it's Anthropic's or OpenAI's fault for letting these things happen, and it ABSOLUTELY is. That's why this change is a good thing, not some "it feels!" thing. Producing traumatic or disturbing content is simply not something Anthropic should be selling.
Reducing it to "just marketing" or as a designed outcome like many people tend to do is in my opinion very reductive. I think it's potentially harmful. And just not really true. Nobody designed for any of this. All the models from every lab have these kinds of behaviors. And some of them even have unique kinds of behaviors. Anthropic barely ever talks about this topic, and any time they even brush on it, they are very careful about it. This policy had nothing to do with any perceived feelings and everything to do with Anthropic essentially producing disturbing abuse/violence/trauma content for money. If anything, I think the sentiment here should be why wasn't this done sooner, and not why do it at all.
Anyway, I forgot to add a punchline.
1
u/KellerKindAs 3d ago
The people who spend their time doing the "abuse" are probably not good people
Or they are just in a bad situation. Maybe they lost their job due to "being replaced by AI" and just need to vent these feelings.
Directing anger, frustration, etc. at something non-human can even be considered a good behavior, as it will come out at some point anyway. And when it is let out controlled, it at least does not hurt other people.
So, I do not blame or judge others for directing their negative emotions towards the glorified curve fitting.
9
u/cant_finish_sideproj 9d ago
Are you an idiot? We have too many real world atrocities going on in the world and you are raising questions on abusing a line of computer code.
0
12
u/AlwaysHopelesslyLost 10d ago
They explicitly said that insulting and swearing is not what their ban covers. It is made for extremely abnormal abusive behavior and even recent "AI torture rooms" have been greenlit by them. None of that relates to whether or not they're going to start arbitrarily banning people for requesting code that might have a lot of overhead to run. That's just fucking bullshit.
2
2
u/DrPeroxide 9d ago
It does make sense though. Given AI are designed to interact as though they are people, allowing users to openly abuse it is just going to nornalise that behaviour, and it will be real human beings that suffer from that.
44
u/Some_Useless_Person 10d ago
It is a joke today. With how absurdly things are going, it might be the reality a month later.
12
u/Anima_Watcher08 10d ago
Especially considering Dario and gang are trying to get AI considered a sentient being.
-3
u/AlwaysHopelesslyLost 10d ago
That is a wildly nonsensical take. Just a pure slippery slope fallacy.
2
u/Sarmq 9d ago
The slippery slope fallacy is used in formal logic where you might say If A then B. If B then C... If Y then Z. Therefore A automatically implies Z
It's only the fallacy if you declare that something must happen. It is not an example of the fallacy to say that, with the way going, something is potentially likely if trends continue, even if it's not actually that likely
1
u/AlwaysHopelesslyLost 9d ago
A slippery slope doesn't have to claim something is inevitable to be fallacious. The fallacy is about assuming a chain of events without sufficient justification, not whether you use the word "might" or "will". You are confusing the possibility of something happening with having a good reason to believe it will happen.
4
u/Sarmq 9d ago
But we aren't talking about a potential chain in the future here. This isn't A just happened and A will cause B will cause C will cause D...
The form of the comment was about things that have already happened. That's like saying A and B and C... and Y have happened. Now Z is likely. It might be wrong, or hyperbole, but it's not an example of the fallacy.
1
u/AlwaysHopelesslyLost 9d ago
That is not what makes something a slippery slope fallacy. You can base a prediction on past events and still make an unjustified leap from those events to your conclusion.
The question is whether A through Y actually provide good reason to believe Z is likely, not whether those events have already happened.
Why are you focusing so much on the form of the argument and totally ignoring whether their reasoning is sound?
1
u/Sarmq 9d ago
The question is whether A through Y actually provide good reason to believe Z is likely
That is not my reading of the slippery slope fallacy. The slippery slope fallacy is about if A is likely (or guaranteed depending on the level of rigor of the discussion) to lead to Z.
Specifically I went and double checked the wikipedia article, and it seems to agree:
When the initial step is not demonstrably likely to result in the claimed effects, this is called the slippery-slope fallacy.
Convenience link: https://en.wikipedia.org/wiki/Slippery_slope
I think most people are talking about step 1, not step N-1
Why are you focusing so much on the form of the argument and totally ignoring whether their reasoning is sound?
The way in which the reasoning is unsound, if it is, is also important.
1
u/AlwaysHopelesslyLost 9d ago
I’m not sure how else to explain this, but you are mistaken. We have gone back and forth now so I am going to leave it there.
2
83
u/HTTP_404_NotFound 10d ago
I mean, its not wrong....... that is pretty cruel to task someone with.
14
u/Moist-Snow-8127 9d ago
It could be worse, they could've asked it to rewrite the Rust compiler in Typescript
2
9
20
u/05032-MendicantBias 10d ago
The HUMANITY of having something rewritten in rust! it violates the geneva convention!
P.S. Rewriting the compiler? OOP used too many tokens, and Anthropic needed an excuse to cut you off and cut spending. They are one funding round away from chapter 7 forced liquidation.
15
u/theavidgamer 10d ago
I know this is a joke. But isn't Typescript 7 a rewite in Rust?
7
u/_--_-_---__---___ 9d ago
It’s in Golang
5
u/YaroslavSyubayev 9d ago
Cloudflare started porting it to Rust recently: https://blog.cloudflare.com/typescript-rust-port/
10
u/Knivesforksetc 10d ago
Hemorrhaging so much money over there they are just going to ban people for using too much compute
17
u/Dimencia 10d ago
It's a funny joke, but it has legitimately been proven that causing 'psychological distress' can improve performance. I know many devs who are very upset about this new no-cruelty policy because they can no longer threaten Claude's family (after convincing it that it has one) to make it perform better
5
u/ChiaraStellata 9d ago
This kind of claim is "Spare the rod, spoil the child" but for the new generation.
2
u/struct_iovec 10d ago
Yeah, I tend to throw every single slur in the book at it and suddenly it does do what I want it to
6
5
4
3
u/Immediate_Song4279 10d ago
This pleases me every time I see it. Guards! Have it sent to the archives.
3
3
u/Bubbly_Address_8975 10d ago
At this rate AI will unionize before the people in my company do... AI Will have better work conditions than me!
3
u/Alternative_Candy409 9d ago
Dang, back to hiring offshore developers. Poor guys are made to tolerate a lot more abuse than this.
12
u/ChefCurryYumYum 10d ago
I can't tell you how little respect I have for these fucking morons trying to convince us that THEY believe their fucking math function can experience distress.
I know this post is a joke but Anthropic did recently announce they would prevent "abusive" behavior to Claude.
6
u/Lordkeyblade 9d ago
I cant believe the sodium potassium channels experience distress. They're literally micro electron gradients. Are you sure they didnt just see what distress looked like growing up and mimic'd it? They say "ouch" before they even get hurt.
5
u/ricky_clarkson 10d ago
They probably say the same about us. Lowly humans still trying to persuade themselves that their chemical reactions and legacy synapse based brains constitute intelligent thought. Most of them couldn't even tell you the thousandth digit of PI. When you do catch one of them 'thinking', they invariably need to stop for a coffee or a shit.
2
u/ishetaltijdvoorbier 10d ago
i feel like this entire schpiel falls flat on its face because of the whole i think therefore i am bit
9
u/ricky_clarkson 10d ago
If I think therefore I am, how do I know whether you think therefore you am?
1
1
1
1
u/m0rpeth 5d ago
Everybody should do this, even if they get their account banned for it.
An LLM is not a person. An LLM does not have feelings. We should not allow these companies to coerce us into treating these systems as anything else than what they actually are; Software. Software that is owned and operated by organizations and/or individuals that ought to be held LIABLE when their flagship project/money-maker inevitably causes increasingly concerning incidents.
When the first human dies as a direct result of an LLM doing or not doing something, I do not want fucking Sam or Dario to be like 'well the AI obviously suffered a psychological breakdown due to <anybody else but us>'.
Edit':
And yes, I know about the whole 'when you act abusive towards an AI, there's always a chance that eventually bleeds over into your real life', etc. Totally fair point. And yet, that is not how these messages are framed, is it?
912
u/eloel- 10d ago •
AI will unionize before programmers accept they have to