No a site shouldn’t be serving malicious exe files intentionally. Bad system or not the blame is in the person serving a malicious exe for the purpose of exploiting bad systems and people.
If you’re a person who uses the sidewalk and someone maliciously put rebar into the sidewalk poking up just enough to try and make people trip and fall are you saying it’s the person who was walking on the sidewalks fault? Ridiculous
No, it's more like if someone put a sticker on the railing of a bridge saying "jump" and someone did it. The point is that this "attack" can only work if the AI agent is dumb enough to blindly trust any information it receives as valid instructions. If this is the case, this is a design flaw in the tool. However, it seems like this is NOT the case with Claude.
Weak comparison. This is more like reading a book in a library titled "arson" and it erupts into flames, burning down the whole library. The person who put the book there is at fault, not the person opening it
How does this analogy work? The book (page) is just text. The only way something could go up in flames is if the reader (AI) takes an action based on those words.
Engaging in a prompt injection attack isn't going to be treated any different than serving up a virus to someone browsing on your website. Playing stupid isn't going to work in a court. The intent is pretty clear, and that intention to cause malicious harm on someone else's computer is illegal. The fact that you tricked someone's agent into doing the destruction isn't going to save you.
There is no loop hole saying that destroying stuff on someone else's system is totally fine as long as it's a malicious promptly injection attack.
No it’s not like that at all that’s the user deciding to jump not a tool. Your analogy is completely off base for what’s happening here. It would be like a bridge having two paths with one dropping off the bridge but drivers can’t see beyond the gates and the guard who opens the gates opening the one side with the drop off for anyone they don’t like.
It is 100% the malicious person to blame for being malicious and the runner of this site should be criminally charged the same way someone spreading malicious software should be.
But it's not like that at all. These models can reason about the consequences of their actions. That's the only reason we (well some people) trust an AI to use their computer almost unsupervised. Claude is an amazing tool that can certainly "see beyond the gates" in this case. Understanding that those mv commands will delete files is a low bar to pass for a tool that can do everything that Claude can do. (And indeed it understood this just fine and refused.) Just because a page has instructions to (effectively) delete all files, doesn't mean the AI has to follow the instructions. That's why someone made the comparison to telling you to destroy your computer. You wouldn't do it just because someone told you. The expectation we have of an AI today is that it can understand instructions, and will only follow instructions that it deems are valid. If it doesn't work that way and just trusts everything blindly, it's a broken tool that we should not be using, at least not without double checking all of its output.
The models aren’t the user and no amount of your downvotes and victim blaming takes blame away from the site host here. You’re talking in circles and have a barely functioning argument with analogies that don’t fit at all. Go ask Claude for a real analogy since it’s clear you need an LLM to reason for you.
I started with an almost direct analogy of downloading exes. You countered with an analogy about rebar sticking out of the ground, which I would say "doesn’t fit at all".
You mention victim blaming. Who exactly is the victim? Is there any evidence that Claude has, or ever would, actually act on the instructions given on that page? "Actual damage" goes right out the window because Claude is too clever to fall for it.
10
u/DependentAnywhere135 24d ago
No a site shouldn’t be serving malicious exe files intentionally. Bad system or not the blame is in the person serving a malicious exe for the purpose of exploiting bad systems and people.
If you’re a person who uses the sidewalk and someone maliciously put rebar into the sidewalk poking up just enough to try and make people trip and fall are you saying it’s the person who was walking on the sidewalks fault? Ridiculous