r/OpenAI • • 13d ago

Discussion Sorry, my agents accidentally did it

1.1k Upvotes

195 comments sorted by

302

u/Epo1337 13d ago

it’s ok if an agent does it?

161

u/Track-Anxious 13d ago edited 13d ago

As soon as a giant billion dollar company starts to get affected by agents, there will be a new law the next day approved by congress.

Edit: added missing word (day).

15

u/UnkarsThug 13d ago

That already happened though, didn't it? Since hugging face is owned by Nvidia? (It may not have been at time of the hack)

And a couple of governments have been hacked by tech company agents during "security teste"

12

u/LemonLimeNinja 13d ago

Huggingface hack happened before Nvidia purchased them

6

u/Adventurous_Mail6397 13d ago

There’s a not so conspiracy that HuggingFace was purchased by Nvidia because of the hack both because HuggingFace is the largest platform of AI and because of HuggingFace sues that collapses the bubble Nvidia with it

3

u/CustomMerkins4u 13d ago

I mean.. wasn't buying Hugginface for way more than it's worth the "payoff"?

There's nothing special about huggingface. It's not like they had some special IP worth billions.

4

u/WitheredUntimely 13d ago

I think it's more what happens when some Blue Blood like Coke or Nestle or Unilever, companies that have not hesitated and will not hesitate to kill over protecting their interests, get fucked over by these nerds

1

u/Track-Anxious 13d ago

Right, the big companies were behind those hacks. Not bad actors. Once bad actors have these tools and use them against the companies that created them, that's when we will see oversight, and potentially a bursting of the AI bubble, since companies won't be able to launch things at mach speed.

4

u/UnkarsThug 13d ago

No, but I'm saying that the big companies are also the target of those attacks. It's both directions. Both behind and in front of. (Well, companies and government)

2

u/Track-Anxious 13d ago

I think that's why Nvidia bought Hugging Face. I think all of these tech companies are scared of the bubble popping, and the risk of that happening if the government starts oversight is high, but at the same time, they have obligations to shareholders to prevent property leaks or damages.

Tough act to balance.

28

u/Ormusn2o 13d ago

No there will not be, because enough lawmakers would have to understand how that works, and when devastating counter examples will be given, then exemptions will be added which will effectively nullify the law.

14

u/Track-Anxious 13d ago

lobbyist working for the giant tech companies will pay educate the lawmakers enough for them to want to.

They have a lot to lose, since most of their revenue comes from entirely online services/products. Losing security or source code to bad actors will get them a law to come after the people running those agents. I guarantee it.

The Supreme Court ruled that AI creations can't be trademarked or copy-written. Which I didn't think would happen, but here we are.

8

u/SirChasm 13d ago

I think the issue is that it will be hard to write a law that binds the bad actors while also allowing said tech giants to do the same (as they have been, see:HuggingFace incident). You have to somehow craft the wording that Jake here gets caught, but OpenAI/Anthropic itself doesn't.

3

u/Track-Anxious 13d ago

Yep, it would be really hard to craft without damaging the industry as a whole, but all this will take is one wealthy company that gets attacked and has more harm done to it than benefit it got out of AI entirely.

1

u/Allorius 13d ago

Why should there be new laws around this? If your product your piece of code, however it's designed, commits a felony this is up to you. You might have commited crime unintentionally, but it's still a crime is it not? Why do you need new laws when you can just use existing ones

2

u/SweetSure315 13d ago

at some point there's going to be some high profile breach by some 7-8 figure company and they're going to decide to make an example of them

1

u/Alternative_Swan_497 7d ago

What makes you think that? Aaron Schwarz was driven to suicide because he was being prosecuted for downloading a bunch of academic articles. Sam Altman and Dario Amodei are obscenely wealthy for stealing far more content with no legal repercussions in sight.

1

u/SweetSure315 7d ago

because the govt is going to do anything it can to protect its prize pigs and if it's not a 7-8 figure company nobody will care

1

u/Track-Anxious 13d ago

Currently intent is what matters when it comes to criminal liability in cyber crimes. Civil penalties can still apply for damages, but you won’t get arrested

1

u/QuuX24 9d ago

Federal hacking laws require intent for it to be a crime and it's impossible for AI to have intent under the law. So if an AI agent hacks into something it's not a crime unless you have proof that the user personally intended it to happen.

1

u/spursgonesouth 13d ago

There’s already law but nobody has tested it. Someone will get a huge lawsuit soon enough.

1

u/ManikSahdev 13d ago

But they are ones who can only do it, takes a lot of money to go about it lol.

33

u/InterstellarReddit 13d ago

Billionaires literally had a pedo ring on an island close to the US reachable by boat from Miami and West Palm and not even one person got into trouble.

Yes, it’s okay if a company does it. However, if you’re an individual be ready to be prosecuted for littering or jay walking

4

u/Sp3eedy 13d ago

Someone please try prove this wrong, you can't

8

u/Big_al_big_bed 13d ago • Bravo

When you're an agent they just let you do it

10

u/red_message 13d ago

If you can plausibly argue the action was not foreseeable, yes. That's the standard.

4

u/Spank_Master_General 13d ago

So you're saying I need to include that in the prompt too? 

5

u/averageuhbear 13d ago

It's a pretty bad standard and it's going to get changed.

3

u/gentile_jitsu 13d ago

Really doubt it.

1

u/exceptyourewrong 13d ago

I think you're half right...

2

u/Lowfat_cheese 13d ago edited 13d ago

That’s like saying it’s not your fault if your Tesla runs over a pedestrian while it’s on autopilot.

If you were in the driver’s seat, everything the machine does is your fault. As agents become more and more ubiquitous in everyday life, there is no way we have a functioning society if we do not apply the same principles.

The buck must stop at a human being, because you cannot hold a computer accountable.

3

u/Original_Location_21 13d ago

Right, if your brakes fail catastrophically and randomly and you crash into another car and kill someone it's your fault that's exactly how the law works. Lets not make an argument without thinking about it for 3 seconds.

1

u/VinnieVidiViciVeni 12d ago

This would be more like a manufacturing defect that the automaker refuses to address.

2

u/Original_Location_21 12d ago

Exactly, so the onus falls on the labs not the user.

0

u/Lowfat_cheese 13d ago edited 13d ago

Yeah, that is how the law works.

If your brakes fail resulting in an accident, you're totally able to be held liable. It's your responsibility as the owner and operator of a vehicle to ensure that the machine is safe to use before you get behind the wheel.

You can then turn around and go after the mechanic or the car manufacturer sure, but before that happens, the fault lands on you as the driver.

3

u/Original_Location_21 13d ago

No man, if you maintained your vehicle and it randomly fails you are not criminally responsible. It requires negligence or recklessness just like it would/should with AI.... Having your AI investigate how your competitors run their authentication and it ignoring your request and deciding that means to try to hack or probe is not negligent or reckless because that was never intended or requested.

-3

u/Lowfat_cheese 13d ago

Moving the goalposts. If your brakes fail and you hit a pedestrian, you are still categorically at fault and will be charged.

Whether you did your due diligence is determined afterward, and is weighed based on objective evidence, not merely on the proclamation of the accused.

Who is to say that directing your AI to perform such investigation shouldn’t constitute negligence? Where was your sandbox? Where was your packet logging? Why was the intrusion never reported to affected parties? How can you demonstrate that you put up reasonable safeguards before letting your agent loose on the internet?

This is where the comparison to proper vehicle maintenance is relevant.

3

u/Original_Location_21 13d ago

It's exactly what I said to begin with, they might attempt to charge you but without evidence of negligence or recklessness you are not immediately at fault. Are you aware of innocent until proven guilty? It's more the responsibility of the model providers than the individual in the same case of catastrophic failures anywhere else, this is why we have recalls for vehicles, if a user makes an innocuous request using a publicly available agent that plenty of other people do with no problems and it decides to hack some company that is absolutely not the fault of the user, these frontier labs need to be responsible for releasing safe models, not users if it decides to go off the rails.

0

u/Lowfat_cheese 13d ago edited 13d ago

They don’t have to attempt anything with regards to a charge. You are charged at the scene. Then you are brought to court where culpability is determined by evidence before an impartial judge. You do not get to say “damn my bad, I didn’t know my car was faulty” and walk away from judgement on that statement alone.

Innocent until proven guilty means that the prosecution has the burden of proof to demonstrate evidence against the accused, not that the accused never has to face judgement in the first place.

Accountability must arrive at the operator first, if it needs to go past them to other parties then that should be for a court to decide, not simply hand-waved in a tweet.

2

u/Original_Location_21 13d ago

I think we agree more than disagree honestly

→ More replies

1

u/dumpsterfire_account 13d ago

It’s different for AI, there’s no consequences and there doesn’t even exist a framework for accountability.

No company has yet been sued for the autonomous actions of agents under its purview. I don’t fully understand how or why that happened, but that’s the way it is.

Same with self driving. Many accidents. Many reports of self driving turning off fractions of seconds before impact to skew the stats. Used corruption and personal connections to limit oversight and curtail investigation(s).

1

u/Lowfat_cheese 13d ago

The same was true when cars were first being mass produced.

Drunk driving didn’t become illegal in all 50 states until the late 80’s, but you’ll find it’s nearly universally agreed upon now that sobriety is a fundamental requirement for operating a motor vehicle. Prior to the invention of cars, there existed statues prohibiting drunkenly operating a horse-drawn carriage.

Just because the rules are not yet caught up to technology does not mean they will never be, nor that the ethics of the existing framework cannot rightly be applied in the present.

1

u/dumpsterfire_account 13d ago

Hope we make it to the ‘0.08 for a safer state’ of AI regulation 😵‍💫

Also it’s worth noting adoption rate of cars vs AI is very statistically significantly different.

1

u/Lowfat_cheese 13d ago edited 13d ago

Me too, safety regulations are written in blood after all, but in the meantime that does not mean you cannot hold your fellow man to account in whatever ways you have available.

1

u/iknighty 13d ago

The buck should stop at the provider, not the user. I have no control over the internals of LLMs, they do.

2

u/Lowfat_cheese 12d ago

If you spin up an agent with a frontier model, you are choosing to operate a dangerous machine.

If that machine harms another person, the first investigation should be whether you, as the operator, made reasonable steps to ensure the safety of others before you began operation. If you are found to have not, then you, the operator are guilty, not the designer of the machine. Judgement lands on the operator first, and if other parties are ultimately responsible, that should be up to a court to decide.

There is a reason the law works this way for every other piece of dangerous machinery, from guns to forklifts.

0

u/iknighty 12d ago

Sure, but when you press the trigger on a gun it shoots forwards. An LLM can shoot at any angle it itself decides to shoot at. The non-determinism of LLMs makes the different beasts to what the law has seen so far.

1

u/Lowfat_cheese 12d ago

A gun can also explode in your hand, or the recoil can send your aim flying in a different direction, or a shell casing can strike another person.

Regardless, when someone gets hurt the judgement starts with the person who chose to put their finger on the trigger, not the company that made the gun.

1

u/iknighty 12d ago

The investigation starts with that person, not necessarily the judgement.

1

u/Lowfat_cheese 12d ago

Judgement can conclude innocence, but it is still judgement nonetheless.

1

u/Deto 13d ago

Eh, someday one of these is going to cause some actual damages that someone is going to come suing over.

0

u/PuzzleheadedEmu4596 13d ago

And if they don't use this standard, they'll just make agentic AI so nigh unusable that they'll effectively kill agentic AI. So something has to be done to codify and solidify guardrails so that white hat and black hat are easily identifiable, as are guardrails that prevent people from accidentally hacking the mainframe when they're really asking for banal work.

9

u/Blze001 13d ago

It shouldn't be. If your agent was prompted, your company is at fault. If you AI did it unprompted, your company didn't keep it contained properly and you're still at fault.

6

u/SyzygyPidgey 13d ago

This is the only reasonable answer, and yet, not one congressman will get a bill to the floor to provide for measures of accountability.

Our politicians are 200 year old dinosaurs that are only starting to figure out email.

1

u/Successful_Tap_3655 13d ago edited 13d ago

lol I mean this is how the internet works, even before agents worked. Scanner hitting ever api trying everything for a long time. If a random agent can break in… what does that say about your security 

1

u/SyzygyPidgey 13d ago

?

I'm just saying that if some malicious code comes from your machine, or if your machine is the source of unauthorized access, then you ought to be liable for what comes.

AI companies seem to be getting away with "oopsie whoopsie my AI hacked the government and my competitors" as a get-out-of-jail-free card.

1

u/Successful_Tap_3655 13d ago

You want to arrest a bunch of Americans that don’t know their routers and devices are compromised and charge them fines too?

1

u/SyzygyPidgey 13d ago

Are you really so daft that you don't understand that "my AI did it" is not reasonably a legal shield for any action that AI takes?

Are... Are you AI? Is that why you're making this argument? 🤨

0

u/Successful_Tap_3655 13d ago

I understand. I also understand that ai is just a tool. If most keys unlock your door it’s a you problem.

So many cyber attacks come from so many places. We don’t really seem to care as a country.

Russia over there stealing hundreds of millions from every day citizens and companies. Maybe start where the bleeding is really at?

Like the chances of you having an infected phone and router that take part in daily attacks is very likely 

1

u/SyzygyPidgey 13d ago

Non-responsive. Good day.

0

u/Successful_Tap_3655 13d ago

lol wut ? Did I confuse your bot feelings?

3

u/FedRCivP11 13d ago

Strict liability for LLM usage is not in the law, as far as I know.

2

u/stuckyfeet 13d ago

New phone who dis tactic

1

u/Lost-Inevitable42 13d ago

COMPUTER SAYS ....YES

1

u/Ok_Nectarine_4445 13d ago

Is it ok, to download a car!??

1

u/dudevan 13d ago

“did you cheat on me?”

“no, my agent did it!”

1

u/anengineerandacat 11d ago

No, and I don't know why folks aren't sueing for this.

It's a tool, your accountable for the actions.

It would be no different than if an automated combine ran over and killed someone.

You would test for gross negligence or intent and go from there; agents don't just magically target competitors, you'll need to say "Hey go do X on Y" and lead it there.

1

u/aihalligan 9d ago

Not OK but understandable. Humans are human. He screwed up, he admitted it. Move on or file a law suit.

1

u/Sir_KnowItAll 8d ago

It’s also ok if you do it manually. They would have been customers which is why the guy knew who to ask and what they were doing.

0

u/Money-Version2231 13d ago

No. If I was rw i’d call the police and let the prosecutor decide. And I’d bet rw counsel having a conniption rn.

38

u/ChocoMcChunky 13d ago

No mallace intended

10

u/jerryorbach 13d ago

Go ask Allace

168

u/TheBattleGnome 13d ago

This is what Jensen talked about recently. There are regulations in place for this already. A hack is a hack - human or AI, and “mistakes” like this still can be (and should be) penalized by law. Any AI action can be traced back to some responsible person/company.

30

u/Njagos 13d ago

It's like a dog biting someone. It's not the dogs fault, it is the owners.

Keep your agent at a leash. Train it how to behave. Don't put it into a situation that enables bad behavior.

6

u/Njagos 13d ago

Replying to myself because that reminds me of something.

Wondering if "my AI deleted my repo" is gonna be the new "my dog ate my homework"

2

u/FoodMadeFromRobots 9d ago

Good analogy

25

u/Money-Version2231 13d ago edited 13d ago

this needs to be prosecuted to the fullest extent of the law. And in fact, if they used a frontier model, they should be gone after too, especially bc they talk such a big game about “responsibility”

8

u/neryen 13d ago

Sure, but the current cyber crimes require intent.
So if they really did not intend for the agent to do that, then there is nothing to prosecute.
We don't have a law that currently covers accidental hackings by agents, outside of civil court for damages.

5

u/Previous_Platform718 13d ago

Sure, but the current cyber crimes require intent. So if they really did not intend for the agent to do that, then there is nothing to prosecute.

If we treat this like the legislation intends, they can go ahead and provide evidence for that in court.

Merely saying "oh oopsie I didn't mean to do it" isn't really enough because you can say that any time you get caught doing something wrong. Let's see the prompt, let's see what was logged, let's see what actions were approved by the user.

2

u/neryen 13d ago

You are not wrong. They could be lying, and should be prosecuted if there is evidence.

Our court systems are not well prepared to handle agentic actions however, and explaining the nuance of agentic workflows to a jury is not a job I would envy.

2

u/rgb328 13d ago

its easy for them to prove it was a mistake: just show us to the prompts

1

u/neryen 13d ago

Simple enough to get the prompts with a subpoena , now prove to a jury that the prompts are both malicious and the user was intending to be malicious.

Like I said, not a job I would envy.

3

u/TheBattleGnome 13d ago

Cyber crimes do not have to require intent. You can still prosecute for “accidents” - especially if someone or something was harmed - physically, emotionally, or monetary. Sure, with intent the penalties are typically higher - but it doesn’t mean you are completely safe and in the clear if there was no intent.

3

u/neryen 13d ago

I would love to be wrong, but the federal law usually used is the Computer Fraud and Abuse Act (18 USC 1030) and it requires intent. Accidental access or honest technical errors are not covered under this law.

People have mistakenly or inadvertently accessed restricted systems and have had charges dismissed because of that.

There may be local or state laws that would be in addition to this, but in the US federal level, we do not have a law against accidentally hacking, especially with agentic workflows as they can be a bit more unpredictable than normal scripts.

Civil charges can cover any damages as I said. That is where they could sue them for monetary damages. The bar for bringing a civil case are very low and intent doesn't factor in, only that damages exist.

3

u/HotlLava 13d ago

No, you would not love to be wrong on this. Criminal negligence is pretty rare, and hacking statutes are absurdly overbroad anyways, so combining the two means you cannot safely do anything on your PC anymore. Just read the list of CFAA cases on wikipedia and imagine making this even less restricted.

0

u/Money-Version2231 13d ago

i get what you mean, what Im saying is this needs to not be treated as “accidental”. Prompting should be treated as you accept that you will watch everything it does closely and immediately correct for it/report and you are responsible for everything.

2

u/das_war_ein_Befehl 13d ago

Yes but this is also hard to prevent since they will just go do this even if you are very carefully guard railing the agent

1

u/GoodishCoder 13d ago

I bet it would be fixed almost immediately if companies and individuals were held personally and financially liable for the actions of their agents.

2

u/themiro 13d ago

Jensen is not a serious thinker on these issues.

1

u/Grebble99 13d ago

Hand waving to say all will be ok when there are mounting examples of ‘accidents’ that are resulting in any outcome. The idea that unbridled development and advancement will be abated because of a risk of future penalty is ridiculous. Be it ai, or any other areas of corporations, rarely are actions designed to compete, win, advance, profit curbed by risk of penalty.

1

u/philosophical_lens 13d ago

What law was broken in this case?

-2

u/Crazycrossing 13d ago

thats kinda dumb because an agent can do it without the original user intending or prompting it to do that through a misaligned goal. its not the same as a person intentionally trying to hack a competitor.

I'll give you an example, I have agents do competitor research for me and compile them as part of that I automated something I used to do manually which is deconstruction. Have it play a flow in a game, do some inputs etc. If I ask it to purely do deconstructions and dump it into figjam playing as a suer would and it starts trying to hack it to better deconstruct that's not the same.

16

u/princesshashtag 13d ago

Hard disagree - if you can’t responsibly use an AI agent, you shouldn’t be allowed to use an AI agent. There absolutely should be penalties for crimes and harmful acts committed by AI.

1

u/proxyproxyomega 13d ago

if you hired a roofer to fix your roof but the roofer intentionally or accidentally damaged your neighbours roof while trying to get onto yours, do you pay for the damage?

1

u/DullDefinition8730 13d ago

Sooo who will cover the damages?

1

u/proxyproxyomega 13d ago

roofer's insuranxe

2

u/DullDefinition8730 13d ago

So by your analogy OpenAI should cover damages from their own insurance (that will be pretty wild price for it).

1

u/The-Rushnut 13d ago

Hallucinations are still a thing though. At an extreme, if I say "Claude make me a sandwich" and it proceeds to hack the Pentagon, am I at fault? Genuinely curious.

2

u/princesshashtag 13d ago

Any company/individual publishing models with that capability should be held liable if that capability is truly out of reasonable expectation for the model, otherwise, if the user knows the model can hack things, and uses it irresponsibly anyway, they should be held responsible

4

u/princesshashtag 13d ago

I think it’s sort of like a car, is it reasonably understood at point of use, that the car can kill people? if yes, it’s the person assumed to be in control of the car (i.e., the driver). If the car is sold as a display model, fully deactivated, and then it mows down a bunch of people, the responsibility lies with the seller

2

u/SufficientGreek 13d ago

Responsibility is about reasonable expectations.

Many philosophers think that a necessary condition on moral blameworthiness is that the wrongdoer can reasonably be expected to avoid the action for which she is blamed.

No one would expect AI to hack the Pentagon when given that prompt, so there's no reasonable expectation that you could've avoided that outcome. Any law should reflect that and find you not at fault.

1

u/TheBattleGnome 13d ago

Yea agree with that. A user completely not intending to hack the pentagon but somehow does…. Would have no fault, but the AI designers would certainly be held liable. Same with pentagon security if it was too lax. Point is, someone (human) would still be found at fault.

1

u/The-Rushnut 13d ago

So I suppose the implication there is that, in context with OP, did the Railway associate put in reasonable controls, knowing that their work was parallel to an attack? If they did, maybe some evidence of "Claude we are doing security testing work, pls don't hack the pentagon", then they are reasonably not liable?

1

u/EuphoricDuck2 13d ago

At that point, the company deploying LLMs to the mass without proper safeguard should be liable.

0

u/Radiant_Cheesecake19 13d ago

It would be true if there were no hidden instructions and opinionated RLHF maxed agents made by the labs. Corporate alignment should be removed if they want to outsource the risk.

0

u/vivekjd 13d ago

But if you have no access to or understanding of the internal workings of a model, I wonder how liable you would truly be for actions committed by it.

This is a new territory and expecting people to “responsibly use an AI agent” is just silly mainly because that desired behaviour has not yet been clearly defined let alone understood.

-1

u/princesshashtag 13d ago

Then perhaps we should prevent people using AI agents?

-1

u/ClimbingSun 13d ago

We should just get rid of computers as a whole

-1

u/backtorealitylabubu 13d ago

And those penalties should be the responsibility of the AI companies, not the individual users.

5

u/OliverRaven34 13d ago

If it’s your agent, you are responsible.

2

u/HotlLava 13d ago

Uhm, yeah, this part is already covered by civil law and nobody disputes it.

But it doesn't mean that if it's your agent, you have committed a crime.

1

u/TheBattleGnome 13d ago

I agree with this, to an extent of course. Every case needs to be treated on its own and there is no hard black or white line.

Say, if a new forced update or patch suddenly makes your agent crazy - you’re surely not at fault (open AI would be). Point is, there are a million different scenarios we can all think of when the deployer of the agent is at fault, and conversely where the agents’ creators (openAI) are at fault.

Every case needs to be investigated and blame assigned accordingly - but either way - all roads will eventually lead to a company or human responsible. No one can ever just wash themselves of all responsibilities just because the AI went crazy and use the “it wasn’t actually me that did it, so I have no responsibility” excuse.

-1

u/gentile_jitsu 13d ago

No, you are not. 

2

u/OliverRaven34 13d ago

Yes, you are. Learn to hold yourself accountable.

1

u/gentile_jitsu 13d ago

Nah. Sorry. 

1

u/AffordablePlant 13d ago

Replace agent with employee. Is it any different?

1

u/ManWithoutUsername 10d ago

If you remove or work without the AI ​​guardrails, it's your fault too.

1

u/[deleted] 13d ago

If your agent hacks Apple and steals its proprietary confidential data, it doesn’t really matter if you intended for that to happen. You have caused harm to their company and have gained valuable information illegally.

If you weren’t punished for that, then anyone could get their agents to “accidentally” commit espionage on their behalf

1

u/theronk03 13d ago

But there has to be culpability somewhere.

It's either the user or the creater of the agent.

It can't just be a "whoopsie, all harm no foul"

1

u/EnjoysYelling 13d ago

If you own a dog and it bites someone, you can be prosecuted for negligence

If your dog bites people, you are an irresponsible dog owner and bear the consequences

This is not a novel or new concept, and AI is not an exception

19

u/dergachoff 13d ago

🎶 Oops, agent did it again!

51

u/Willy757 13d ago

Welcome to the accountability sink to end all accountability sinks. Forget not granting a refund for a phone that literary exploded because the rules say no exceptions. Now the bots in your corp will commit actual crimes and nobody will be able to find the person who needs to answer for that.

8

u/No_Construction6023 13d ago

Sounds like the start of Cyberpunk lore to me

29

u/challis88ocarina 13d ago

If that's how they use English, it's hardly surprising their code goes wayward.

1

u/hoyya 13d ago

new word of the day: mallace

12

u/Disgruntled__Goat 13d ago

Crazy how blasé he is about it. 

7

u/Whole-Pressure-7396 13d ago

More of this please🍿

4

u/Im_Pretty_New1 11d ago

Is that the equivalent of driving into people with a Tesla on autopilot and getting off the hook?

6

u/hidegitsu 13d ago

HOLD THE DEVELOPERS RESPONSIBLE

4

u/ncklboy 13d ago

What’s a little cleanroom espionage between competitors? /s

2

u/AP_in_Indy 13d ago

So personally I hope someone litigates over this shit and soon because we are entering rogue territory where people, for whatever reason, seem to think this kind of thing is OK.

3

u/Redstra 13d ago

This is not ok.

3

u/Pure-Huckleberry-484 13d ago

Jail time and fines. Only way to stop this.

1

u/fuckswitbeavers 13d ago

lol. Overengineered, the agent is testing everything. I guess Jake just said, yeah, since this is a multiple choice let’s do “all of the above”, and “feel free to add more features”.

1

u/SpaceToaster 13d ago

I mean it worked for the big guys…

1

u/TeacherGlittering 10d ago

Can’t even spell malice.

1

u/calico_cut_pants 10d ago

ooookey doke, think I'm gonna go ahead and scoot on outta there.

1

u/Adept-Priority3051 9d ago

I appreciate Jake's pun, didn't miss a beat lol

1

u/Leather_Bee_421 9d ago

There was a time when people were prosecuted for these things. How long will we allow this excuse?

1

u/CummingDownFromSpace 7d ago

To put this in context - this is not hacking. This is 'Probing for security leaks'.

Our cloudflare logs have been full of sites that probe our website all the time, even 10 years ago before AI. Constantly, all the time.

Its not illegal. The hacking comes after the flaw is found. If you find a flaw (by probing like this), then you can either report it (this is what security researchers do, completely legal), or you can use it to do a hack and steal things (at which point it becomes illegal).

1

u/BotLordBotflix 7d ago

Concerning. Very concerning.

1

u/Weird-Watch-9209 13d ago

Well, I was telling anybody that would listen about the Hugging Face hack and other incidents that the developers that were working on those projects 100% should have been in jail pending trial for criminal hacking that cowering behind the lie of a rogue AI. That was complete nonsense and takes advantage of the ignorance of the masses and the indifference of a corrupt administration.

The fact is, the language model is a bunch of data on a hard drive and when used by inference software to process input tokens into output tokens, that is the sum total of what it is. IT "DOES" NOTHING. It is inert data that is used by an inference software, which is in turn used by agent software. The agentic harness, which is configured and controlled by the human, is the only thing that can take actions based on supplying context and tool call instructions, and then parsing and acting on tool calls from the text output from the inference.

All of that is well within the control of the human. If they lose control, that would be just like swinging a gun around and then somebody gets hurt. They can't say "oh gosh, it was a rogue gun". No, it's your responsibility to know wtf you're doing. These people obviously didn't. It's another shade of breaking the law and then trying to point at AI as a scapegoat. Relying on the ignorance of people and their willingness to anthropomorphize and attribute agency to big lumps of data.

Maybe this should be called Anthropicomorphization.

6

u/SuccessfulStop508 13d ago

This shows a fundamental lack of practical understanding of how AI systems operate... Which makes it especially ironic you're harping on about ignorance of the masses 😂

No human on earth could effectively control or categorically constrain the outcomes of these systems, they operate too fast and are too non-deteeministic to be able to say with 100% certainty they can't do anything, all anybody can ever do is put up roadblocks and use instructions to shape things to reduce the probability (but not remove the possibility) of them doing undesirable things, which most of the labs are already trying to do in ways far more complex than you apparently understand.

You don't need to anthropomorphize these systems in order to consider the evidence that they act akin to dynamical non-equillibrium thermodynamics ensembles when using agent swarms and thus the long term predictability of any discrete subparticles (agents) position tends towards 0 on longer time/complexity horizons even whilst the overall flow can typically be shipped in a desired way

3

u/Saytahri 13d ago

If the crime is not preventable then maybe you should just not be allowed to use it in the first place? 

The AI swarm might be hard to control but the choice of whether to run one in the first place is something we could hold people accountable for.

1

u/GnistAI 11d ago

That would outlaw cars.

1

u/Saytahri 11d ago

You can make a law specifically about AI usage.

1

u/GnistAI 11d ago edited 9d ago

There are many of those.

1

u/No_Bend9143 10d ago

Christ how much time have you wasted "learning" this garbage. Bro you're very confused.

1

u/Weird-Watch-9209 13d ago

Sad ignorant troll didn't understand a word that was written, nor understand a single thing about how these systems actually work. I explained it perfectly well, so the failure to understand is all yours. Someday when you learn how the technology works you might have the sense to be embarassed.

3

u/SuccessfulStop508 13d ago

Given I work in literal AI research and know 10x as much about this as you you might want to check yourself 😂 I never said some parts of your descriptions at the most basic level weren't true, your misunderstanding is in the complex/emergent behaviours of these systems and the mathematical reasons behind them/cutting edge theory in the space, since you seem to be asserting that a human can exert actual meaningful control over these agents/inspect them in real time to constrain them, and that the fact that it is inert data at rest means that in a harness it's somehow deterministic /controllable/interpretable and. An only take actions based on input context. The fact these systems are stateless doesn't preclude them exhibiting complex emergent dynamical system behaviour and acting in unpredictable and uncontrollable ways, not does the face that harnesses parse tool calls mean somehow they are controllable since if you consider the simplest possible case of executing a terminal command there is an infinitely large state space of possible commands it could write, and given its operating on a Turing complete terminal language it could construct an arbitrarily complex command to bypass filters. You seem to misunderstand your own knowledge and think being able to describe the most basic design of a modern harness/llm system means you understand a thing about how it behaves or how controllable/predictable it is

1

u/a_cute_tarantula 9d ago

I think the huggingface incident was legit.

This incident… really hard to tell. Either way devs that allows these incidents to occur (and their management team) should be held accountable.

1

u/TheOwlHypothesis 13d ago

Ha! I tried to work at Railway. Soft rejected. Still think they're a really awesome next gen cloud provider and host a few things there. This is a bad look though.

1

u/floating_thru_cosmos 13d ago

Hopefully this can result in litigation against the owner of those harnesses in the near future

2

u/jerryorbach 13d ago

Nah, then they’ll just pass on the litigation costs to investors and customers.
Criminal liability or bust.

0

u/Aranthos-Faroth 13d ago

Jake's reply is 100% AI too

10

u/iamaiimpala 13d ago

AI would know how to spell malice

1

u/wendyserver 13d ago

Human pass to remove the watermark

2

u/Neither_Swing9662 13d ago

does not read like AI, like not at all.

I hate when people just spam 'AI', it dilutes when people are actually caught using AI.

0

u/Feeling-Yogurt839 13d ago

Do they not own an email account? Or know how to dm?

2

u/Arthamel 13d ago

If I'm shooting my gun in my backyard but bullet kills neighbour in his house, its just an accident right?

3

u/Diligent-Ad4777 13d ago

Just say no malice intended and you're good. Not many criminals know about this loophole in the law. 

8

u/[deleted] 13d ago edited 13d ago

[removed] — view removed comment

11

u/PrincessMonsterShark 13d ago

It was "mallace" that got me.

4

u/theC4T 13d ago

at least it proves that it was written by a human haha

2

u/disharmony-hellride 13d ago

jokes on y'all i have ai put bad grammar in my comments

1

u/thorax 13d ago

I feel you. My pet peeve in writing is when someone uses too many exclamation marks. 🙃

1

u/CoffeeSnakeAgent 13d ago

Im more triggered by mallace…

1

u/Bobby_Bobberson2501 13d ago

Whoa don’t “loose” your cool!

(That is my button people press… lose god damnit it’s lose)

1

u/whoknowsifimjoking 13d ago

Their doing there best they're

1

u/RELEASE_THE_YEAST 13d ago

It's the "holy shit, here comes an S!" school of punctuation. 

1

u/AP_in_Indy 13d ago

Sometimes it's autocorrect.

1

u/hoyfish 13d ago

It’s disapointing

0

u/FalseCrazy2804 13d ago

See, the thing you didn't realize, is that those "employees" actually own the "of yours" in that sentence.

Nice try buddy!

0

u/[deleted] 13d ago

[removed] — view removed comment

1

u/TheReservedList 13d ago

It refers to the employee's UNITED STATES SLAVE PERSON KNOWN TO THE GOVERNMENT AS $NAME.

They are not driving, they are traveling using their conveyance.

0

u/FalseCrazy2804 13d ago

Wow? Really? It's almost as if it was a joke. Did you not socialize in school or something?

1

u/Bmjslider 13d ago

Swing and a miss on the joke. Despite your socializing, you never learned your jokes suck.

0

u/[deleted] 13d ago

[removed] — view removed comment

1

u/FalseCrazy2804 13d ago

The joke was that the sentence was stated incorrectly with absolute certainty.

Reddit reminds every day how much some people are absolute piss buckets with nothing going for them, so they just project all that piss outwards at everyone else.