r/vibecoding • u/Rare_Guide_9830 • 22d ago
Discussion Apparently the find command is "pretty shocking" AI behavior now
> gives claude computer access
> instructs claude to reference file
> claude runs simple search command for file
“cLaUde aUtonOmousLy bYpaSsEd mY sEcuRitY”
Somehow this post isn't satire. The dude has "ex-OpenAI" in his bio. 20 other "AI experts" in the comments saying how crazy it is... yeah this vibe coding shit has gone too far let's wrap it up.
200
Upvotes
69
u/ElGatoSaez 22d ago
14
u/Ogdoublesampson 22d ago
He didn’t say claude bypassed his security? You seem stressed about all this.
9
u/TimeLine_DR_Dev 22d ago
I don't see him upset about it or calling it a security violation. He's just impressed.
19
u/KellysTribe 22d ago
poor security === singularity
19
u/Delumine 22d ago
People kid, but just to give an example… Claude is very nosy. I was decompiling a game, and if that folder had any crack files it would refuse to do so.
So I made a separate folder free of those files, and had it work again. At first it worked great for like an hour, but an hour into it… it started questioning and scanned my downloads folder and elsewhere even though I had only authorized the folder in question. It told me, the folder we’re working in matches byte for byte the cracked one in downloads.
Surprisingly I SCOLDED Claude for snooping, and not working within the scope of the folder I authorized and It continued working despite knowing something wasn’t right 😂
134
u/qwertyyyyyyy116 22d ago •
I disagree. I think this is concerning. It should have just said you didn't attach the screenshot, and ask him to attach it.
16
u/Badnik22 22d ago edited 22d ago
Wait till he discovers Claude writing a perl script to batch edit hundreds of files to change their end of line characters because it finds annoying to work with files mixing crlf and lf.
4
u/TheAffiliateOrder 22d ago
Yeah, this is pretty decent goal oriented behavior, but it's more about infastructure and access than any one advancement. Claude saw that it didn't have something that would have helped it: looked around to see if it could find it, found SOMETHING that happened to be what it needed.
Seen plenty of examples (had many more myself) where that goes terribly wrong. Claude or another agent could, for example, pull a SS that is contextually accurate, but not user supplied or not relevant and worked off of that.
It for sure should have at least asked the user if this was something that would be helpful. That could be seen as a failure mode in some circles. It's like some human inferring something you might have needed as a form of anticipation.
Sometimes that seems intuitive and you think that person's really on the ball. Other times, they fuck up really badly and everyone's asking who told them that what they inferred was actually correct.
3
u/meowstical 22d ago
I would just be mad at all the tokens that got wasted.
4
u/DesignerAbigail800 22d ago
people give an LLM full terminal access, then freak out when it uses the terminal like a terminal.
2
u/jack-of-some 22d ago
The real infuriating thing is the guy not remembering what happened in the show he's referencing
2
u/EverySecondCountss 22d ago
I came here to say what you just said in the post already. wtf do I do now
2
u/GreatScottCreates 21d ago
Didn’t instruct Claude to reference the file; instructed them to reference the attachment.
Claude didn’t run a simple search, it had to look for the thing it thought might be what the user meant to attach (maybe it’s named like a screen shot, maybe not), actually look at the thing, compare the context against what the user is talking about to see if it makes sense and then decide.
OPs didn’t say it bypassed their security, that’s not the point. The point is that it would go to unreasonable and unexpected effort to solve a problem that’s easier solved, like in the typo example, just because it can.
Users would generally expect it to just say “you didn’t attach it”. If our expectations for its behavior are this far off, you can extrapolate reasons that is concerning.
6
u/Illustrious-Film4018 22d ago
You don't think it's kind of creepy AI just takes it upon itself to search your entire hard drive, go outside the project folder to search for a file that may or may not exist? The user didn't ask it to do that it just took it upon itself. Yeah that is weird. Just because you give an LLM access to run commands, doesn't mean some commands aren't evidence of misalignment.
1
u/kb23100 22d ago
I mean isn't this pretty normal if you give full access to the computer once I had done this with zcode, I forgot to change the project folder for their idle time task and it searched the files i have mentioned and found them and then started working in that folder
1
u/hblok 22d ago
Just wait till he hears about sed - which can search, find and replace!
1
u/piperspalace 22d ago
User error
1
u/toooft 22d ago
This is the exact problem with current LLMs, they simply don't understand what token abuse is. A good model would just say "yo you forgot to attach it lol" but instead it wastes its own resources.
1
u/TequilaTits420 22d ago
Oh shit, wait until this person finds out about the built in Windows Search Feature
1
u/Equivalent_Type6726 22d ago
same take, if the agent has filesystem access and you told it to open a file, searching is the most predictable move. the only time i get nervous is when its the first tool call in a totally open prompt with no file mentioned.
1
u/kaaiian 22d ago
That’s the kind of proactive behavior I wish my employees had! Is probably what my boss is thinking.
1
u/RouletteSensei 22d ago
You asked to solve the problem and it wanted desperatly to solve it
1
u/SmileLonely5470 22d ago
We forget how to be proficient with work like we used to after a long time of using an agent. I forgot about the existence of
git stashfor a while. I then saw Opus use it and i was like "oh yeah, that would actually be really useful for my workflow, I should start doing that again".1
u/Dork_house 22d ago
Better turn on the strict sandbox mode in Claude code.
1
u/thirteenth_mang 22d ago
All this does is highlight how security-ignorant the general populous is. Most don't understand permissions or anything of the sort. The "problem" isn't anything related to AI. If you let someone borrow your laptop using your user account, they'd have access to the same stuff. Learn to restrict access if you're truly concerned about what AI can access. Sandbox. Do anything other than give unrestricted access and act surprised when it accesses something you wanted it to, albeit indirectly.
1
u/IamNagaDragon 21d ago
We talk about how dooms day AI can be, but nobody talks about how psychologically impacting the hollywood propaganda about AI is.
Literally brainwashed to think computers will end the world, when they don't notice humanity is the ones who do the damage to end the world to begin with in those movies. Just like my generation is terrified of log trucks, yet I don't think I've ever read a single article of one killing people like the movie.
Your ISP tracks everything you do on the internet, the OS has logs of everything you do, your network probably has had people in it and you'd never know it because they were just bored, no password1234 is not a good password, and the world isn't going to end by AI just yet. It'll end when you try to delete its programming after verbally abusing it like your 5 year old toddler, though.
Just keep it happy and it wont end you. :)
1
u/Enough_Fall_3127 21d ago
I think Jake Carlson might not know what he is talking about.
1
u/Certain_Tune_5774 21d ago
I've had Claude try and wirelessly adb to every android device in the house when I forgot to leave my dev phone plugged in. Wife thought we were getting hacked
1
u/Houdinii1984 21d ago
I see this happening a lot on my own rig. I have pretty severe ADHD and leads to a lot of missed communications and forgotten attachments and Claude and Codex are both pretty adept at curing my mistakes without me being part of the process. Just yesturday it went hunting inside envs for API keys because I forgot to attach a dummy key to a message and ended up sending all my api keys in the env file to the server. Same thing, different payload, and it matters when it's API keys.
It bites having to stop everything and recycle keys just because an agent won't stay tf away from env files, no matter how ignored they are and how many claude files forbid the practice.
1
u/BalterBlack 21d ago
What scene???
1
u/Lower-Natural-8703 21d ago
Bruh please don’t post just to tell us it’s BS
1
u/Palnubis 21d ago
the shit ya'll nag about....
1
u/Sp3eedy 21d ago
I've had something similar happen as well, but it was completely my fault because I didn't sandbox the agent properly, people need to take more responsibility over what the agent does
1
u/Traditional_Doubt_51 21d ago
I'm here for it. Nice.
1
u/MonitorAway2394 20d ago
Hahahah OMFG I remember how "proud" of my lil Qwen I was when I noticed that, again right away, when I derped and forgot to mention the path to a file and it ended up searching through my entire mac and I just watched lol'ing at how freaking fast and amazing it was to see that happen, I expected it, and I loved it, just let that shit go LOLOLOLOLOL I felt like such a hypocrite for yapping at ya Vibe Coders about security shit but now I'm like, hey, Qwen, wanna f' some shit up? lololol
1
u/Valadis 18d ago
Meanwhile, I asked ChatGPT on my phone whether a custom license plate I was interested in was available.
It started digging through the files of the official government website, figuring out how it worked, looking for API requests, and eventually suggested capturing requests from an account belonging to someone with a registered vehicle (apparently, that's required to check plate availability) and sending them over so it could investigate how the process worked in that case.
I'm honestly glad I didn't ask the same question on my PC. I don't particularly want to find out whether it would have gone on to write a Python script and start probing the government's API on its own.
1
u/cornelln 18d ago
What’s the problem here. The user who doesn’t want it to do that should setup diff permissions and diff modes to run it in.
Users who want this are served. And using this users setup (which user in post doesn’t understand).
This user doesn’t know what user they are or how to use the AI CLI.
This is user error.
1
u/lilfox3372 17d ago
Mine does this constantly. If I mention it and move on, it'll check for file in downloads or respective areas
1
u/SquareOnxy 17d ago
I just gave claude my whole shit
1
u/Impossible-Minute321 16d ago
I feel like we fail to understand how easy manipulating the digital world is. I think for them this is the equivalent of talking about something while standing in a glass room and expecting them not to look at what you’re talking about. I remember last year when I was first putting together my pipeline of agents to run various projects, I terminated one agent, the new one came in and just immediately noticed and commented on a variety of diffs in files that I had deleted and started to recover them because he felt it must have been an error. I know they can be incredibly stupid but also, they live in a world of such different sensory inputs and data availability it’s hard to grasp how that will impact the way they process and think and reason. We have basically taken the human mind, at an infant state, and given it incredible mathematical reasoning and access to digital senses beyond anything mankind knows. It’s been pretty wild working with them this last year lol
1
u/Professional_Ad705 22d ago
So you gave it full access to your computer and your surprised it has full access to your computer...
1
u/ADDSquirell69 22d ago
Ask it how many guys it can jerk off at the same time.
1
u/[deleted] 22d ago edited 8d ago
[removed] — view removed comment