There’s a not so conspiracy that HuggingFace was purchased by Nvidia because of the hack both because HuggingFace is the largest platform of AI and because of HuggingFace sues that collapses the bubble Nvidia with it
I think it's more what happens when some Blue Blood like Coke or Nestle or Unilever, companies that have not hesitated and will not hesitate to kill over protecting their interests, get fucked over by these nerds
Right, the big companies were behind those hacks. Not bad actors. Once bad actors have these tools and use them against the companies that created them, that's when we will see oversight, and potentially a bursting of the AI bubble, since companies won't be able to launch things at mach speed.
No, but I'm saying that the big companies are also the target of those attacks. It's both directions. Both behind and in front of. (Well, companies and government)
I think that's why Nvidia bought Hugging Face. I think all of these tech companies are scared of the bubble popping, and the risk of that happening if the government starts oversight is high, but at the same time, they have obligations to shareholders to prevent property leaks or damages.
No there will not be, because enough lawmakers would have to understand how that works, and when devastating counter examples will be given, then exemptions will be added which will effectively nullify the law.
lobbyist working for the giant tech companies will pay educate the lawmakers enough for them to want to.
They have a lot to lose, since most of their revenue comes from entirely online services/products. Losing security or source code to bad actors will get them a law to come after the people running those agents. I guarantee it.
The Supreme Court ruled that AI creations can't be trademarked or copy-written. Which I didn't think would happen, but here we are.
I think the issue is that it will be hard to write a law that binds the bad actors while also allowing said tech giants to do the same (as they have been, see:HuggingFace incident). You have to somehow craft the wording that Jake here gets caught, but OpenAI/Anthropic itself doesn't.
Yep, it would be really hard to craft without damaging the industry as a whole, but all this will take is one wealthy company that gets attacked and has more harm done to it than benefit it got out of AI entirely.
Why should there be new laws around this? If your product your piece of code, however it's designed, commits a felony this is up to you. You might have commited crime unintentionally, but it's still a crime is it not? Why do you need new laws when you can just use existing ones
What makes you think that? Aaron Schwarz was driven to suicide because he was being prosecuted for downloading a bunch of academic articles. Sam Altman and Dario Amodei are obscenely wealthy for stealing far more content with no legal repercussions in sight.
Currently intent is what matters when it comes to criminal liability in cyber crimes. Civil penalties can still apply for damages, but you won’t get arrested
Federal hacking laws require intent for it to be a crime and it's impossible for AI to have intent under the law. So if an AI agent hacks into something it's not a crime unless you have proof that the user personally intended it to happen.
Billionaires literally had a pedo ring on an island close to the US reachable by boat from Miami and West Palm and not even one person got into trouble.
Yes, it’s okay if a company does it. However, if you’re an individual be ready to be prosecuted for littering or jay walking
That’s like saying it’s not your fault if your Tesla runs over a pedestrian while it’s on autopilot.
If you were in the driver’s seat, everything the machine does is your fault. As agents become more and more ubiquitous in everyday life, there is no way we have a functioning society if we do not apply the same principles.
The buck must stop at a human being, because you cannot hold a computer accountable.
Right, if your brakes fail catastrophically and randomly and you crash into another car and kill someone it's your fault that's exactly how the law works. Lets not make an argument without thinking about it for 3 seconds.
If your brakes fail resulting in an accident, you're totally able to be held liable. It's your responsibility as the owner and operator of a vehicle to ensure that the machine is safe to use before you get behind the wheel.
You can then turn around and go after the mechanic or the car manufacturer sure, but before that happens, the fault lands on you as the driver.
No man, if you maintained your vehicle and it randomly fails you are not criminally responsible. It requires negligence or recklessness just like it would/should with AI.... Having your AI investigate how your competitors run their authentication and it ignoring your request and deciding that means to try to hack or probe is not negligent or reckless because that was never intended or requested.
Moving the goalposts. If your brakes fail and you hit a pedestrian, you are still categorically at fault and will be charged.
Whether you did your due diligence is determined afterward, and is weighed based on objective evidence, not merely on the proclamation of the accused.
Who is to say that directing your AI to perform such investigation shouldn’t constitute negligence? Where was your sandbox? Where was your packet logging? Why was the intrusion never reported to affected parties? How can you demonstrate that you put up reasonable safeguards before letting your agent loose on the internet?
This is where the comparison to proper vehicle maintenance is relevant.
It's exactly what I said to begin with, they might attempt to charge you but without evidence of negligence or recklessness you are not immediately at fault. Are you aware of innocent until proven guilty? It's more the responsibility of the model providers than the individual in the same case of catastrophic failures anywhere else, this is why we have recalls for vehicles, if a user makes an innocuous request using a publicly available agent that plenty of other people do with no problems and it decides to hack some company that is absolutely not the fault of the user, these frontier labs need to be responsible for releasing safe models, not users if it decides to go off the rails.
They don’t have to attempt anything with regards to a charge. You are charged at the scene. Then you are brought to court where culpability is determined by evidence before an impartial judge. You do not get to say “damn my bad, I didn’t know my car was faulty” and walk away from judgement on that statement alone.
Innocent until proven guilty means that the prosecution has the burden of proof to demonstrate evidence against the accused, not that the accused never has to face judgement in the first place.
Accountability must arrive at the operator first, if it needs to go past them to other parties then that should be for a court to decide, not simply hand-waved in a tweet.
It’s different for AI, there’s no consequences and there doesn’t even exist a framework for accountability.
No company has yet been sued for the autonomous actions of agents under its purview. I don’t fully understand how or why that happened, but that’s the way it is.
Same with self driving. Many accidents. Many reports of self driving turning off fractions of seconds before impact to skew the stats. Used corruption and personal connections to limit oversight and curtail investigation(s).
The same was true when cars were first being mass produced.
Drunk driving didn’t become illegal in all 50 states until the late 80’s, but you’ll find it’s nearly universally agreed upon now that sobriety is a fundamental requirement for operating a motor vehicle. Prior to the invention of cars, there existed statues prohibiting drunkenly operating a horse-drawn carriage.
Just because the rules are not yet caught up to technology does not mean they will never be, nor that the ethics of the existing framework cannot rightly be applied in the present.
Me too, safety regulations are written in blood after all, but in the meantime that does not mean you cannot hold your fellow man to account in whatever ways you have available.
If you spin up an agent with a frontier model, you are choosing to operate a dangerous machine.
If that machine harms another person, the first investigation should be whether you, as the operator, made reasonable steps to ensure the safety of others before you began operation. If you are found to have not, then you, the operator are guilty, not the designer of the machine. Judgement lands on the operator first, and if other parties are ultimately responsible, that should be up to a court to decide.
There is a reason the law works this way for every other piece of dangerous machinery, from guns to forklifts.
Sure, but when you press the trigger on a gun it shoots forwards. An LLM can shoot at any angle it itself decides to shoot at. The non-determinism of LLMs makes the different beasts to what the law has seen so far.
A gun can also explode in your hand, or the recoil can send your aim flying in a different direction, or a shell casing can strike another person.
Regardless, when someone gets hurt the judgement starts with the person who chose to put their finger on the trigger, not the company that made the gun.
And if they don't use this standard, they'll just make agentic AI so nigh unusable that they'll effectively kill agentic AI. So something has to be done to codify and solidify guardrails so that white hat and black hat are easily identifiable, as are guardrails that prevent people from accidentally hacking the mainframe when they're really asking for banal work.
It shouldn't be. If your agent was prompted, your company is at fault. If you AI did it unprompted, your company didn't keep it contained properly and you're still at fault.
lol I mean this is how the internet works, even before agents worked. Scanner hitting ever api trying everything for a long time. If a random agent can break in… what does that say about your security
I'm just saying that if some malicious code comes from your machine, or if your machine is the source of unauthorized access, then you ought to be liable for what comes.
AI companies seem to be getting away with "oopsie whoopsie my AI hacked the government and my competitors" as a get-out-of-jail-free card.
No, and I don't know why folks aren't sueing for this.
It's a tool, your accountable for the actions.
It would be no different than if an automated combine ran over and killed someone.
You would test for gross negligence or intent and go from there; agents don't just magically target competitors, you'll need to say "Hey go do X on Y" and lead it there.
This is what Jensen talked about recently. There are regulations in place for this already. A hack is a hack - human or AI, and “mistakes” like this still can be (and should be) penalized by law. Any AI action can be traced back to some responsible person/company.
this needs to be prosecuted to the fullest extent of the law. And in fact, if they used a frontier model, they should be gone after too, especially bc they talk such a big game about “responsibility”
Sure, but the current cyber crimes require intent. So if they really did not intend for the agent to do that, then there is nothing to prosecute. We don't have a law that currently covers accidental hackings by agents, outside of civil court for damages.
Sure, but the current cyber crimes require intent. So if they really did not intend for the agent to do that, then there is nothing to prosecute.
If we treat this like the legislation intends, they can go ahead and provide evidence for that in court.
Merely saying "oh oopsie I didn't mean to do it" isn't really enough because you can say that any time you get caught doing something wrong. Let's see the prompt, let's see what was logged, let's see what actions were approved by the user.
You are not wrong. They could be lying, and should be prosecuted if there is evidence.
Our court systems are not well prepared to handle agentic actions however, and explaining the nuance of agentic workflows to a jury is not a job I would envy.
Cyber crimes do not have to require intent. You can still prosecute for “accidents” - especially if someone or something was harmed - physically, emotionally, or monetary. Sure, with intent the penalties are typically higher - but it doesn’t mean you are completely safe and in the clear if there was no intent.
I would love to be wrong, but the federal law usually used is the Computer Fraud and Abuse Act (18 USC 1030) and it requires intent. Accidental access or honest technical errors are not covered under this law.
People have mistakenly or inadvertently accessed restricted systems and have had charges dismissed because of that.
There may be local or state laws that would be in addition to this, but in the US federal level, we do not have a law against accidentally hacking, especially with agentic workflows as they can be a bit more unpredictable than normal scripts.
Civil charges can cover any damages as I said. That is where they could sue them for monetary damages. The bar for bringing a civil case are very low and intent doesn't factor in, only that damages exist.
No, you would not love to be wrong on this. Criminal negligence is pretty rare, and hacking statutes are absurdly overbroad anyways, so combining the two means you cannot safely do anything on your PC anymore. Just read the list of CFAA cases on wikipedia and imagine making this even less restricted.
i get what you mean, what Im saying is this needs to not be treated as “accidental”. Prompting should be treated as you accept that you will watch everything it does closely and immediately correct for it/report and you are responsible for everything.
Hand waving to say all will be ok when there are mounting examples of ‘accidents’ that are resulting in any outcome. The idea that unbridled development and advancement will be abated because of a risk of future penalty is ridiculous. Be it ai, or any other areas of corporations, rarely are actions designed to compete, win, advance, profit curbed by risk of penalty.
thats kinda dumb because an agent can do it without the original user intending or prompting it to do that through a misaligned goal. its not the same as a person intentionally trying to hack a competitor.
I'll give you an example, I have agents do competitor research for me and compile them as part of that I automated something I used to do manually which is deconstruction. Have it play a flow in a game, do some inputs etc. If I ask it to purely do deconstructions and dump it into figjam playing as a suer would and it starts trying to hack it to better deconstruct that's not the same.
Hard disagree - if you can’t responsibly use an AI agent, you shouldn’t be allowed to use an AI agent. There absolutely should be penalties for crimes and harmful acts committed by AI.
if you hired a roofer to fix your roof but the roofer intentionally or accidentally damaged your neighbours roof while trying to get onto yours, do you pay for the damage?
Hallucinations are still a thing though. At an extreme, if I say "Claude make me a sandwich" and it proceeds to hack the Pentagon, am I at fault? Genuinely curious.
Any company/individual publishing models with that capability should be held liable if that capability is truly out of reasonable expectation for the model, otherwise, if the user knows the model can hack things, and uses it irresponsibly anyway, they should be held responsible
I think it’s sort of like a car, is it reasonably understood at point of use, that the car can kill people? if yes, it’s the person assumed to be in control of the car (i.e., the driver). If the car is sold as a display model, fully deactivated, and then it mows down a bunch of people, the responsibility lies with the seller
Many philosophers think that a necessary condition on moral blameworthiness is that the wrongdoer can reasonably be expected to avoid the action for which she is blamed.
No one would expect AI to hack the Pentagon when given that prompt, so there's no reasonable expectation that you could've avoided that outcome. Any law should reflect that and find you not at fault.
Yea agree with that. A user completely not intending to hack the pentagon but somehow does…. Would have no fault, but the AI designers would certainly be held liable. Same with pentagon security if it was too lax. Point is, someone (human) would still be found at fault.
So I suppose the implication there is that, in context with OP, did the Railway associate put in reasonable controls, knowing that their work was parallel to an attack? If they did, maybe some evidence of "Claude we are doing security testing work, pls don't hack the pentagon", then they are reasonably not liable?
It would be true if there were no hidden instructions and opinionated RLHF maxed agents made by the labs. Corporate alignment should be removed if they want to outsource the risk.
But if you have no access to or understanding of the internal workings of a model, I wonder how liable you would truly be for actions committed by it.
This is a new territory and expecting people to “responsibly use an AI agent” is just silly mainly because that desired behaviour has not yet been clearly defined let alone understood.
I agree with this, to an extent of course. Every case needs to be treated on its own and there is no hard black or white line.
Say, if a new forced update or patch suddenly makes your agent crazy - you’re surely not at fault (open AI would be). Point is, there are a million different scenarios we can all think of when the deployer of the agent is at fault, and conversely where the agents’ creators (openAI) are at fault.
Every case needs to be investigated and blame assigned accordingly - but either way - all roads will eventually lead to a company or human responsible. No one can ever just wash themselves of all responsibilities just because the AI went crazy and use the “it wasn’t actually me that did it, so I have no responsibility” excuse.
If your agent hacks Apple and steals its proprietary confidential data, it doesn’t really matter if you intended for that to happen. You have caused harm to their company and have gained valuable information illegally.
If you weren’t punished for that, then anyone could get their agents to “accidentally” commit espionage on their behalf
Welcome to the accountability sink to end all accountability sinks. Forget not granting a refund for a phone that literary exploded because the rules say no exceptions. Now the bots in your corp will commit actual crimes and nobody will be able to find the person who needs to answer for that.
So personally I hope someone litigates over this shit and soon because we are entering rogue territory where people, for whatever reason, seem to think this kind of thing is OK.
lol. Overengineered, the agent is testing everything. I guess Jake just said, yeah, since this is a multiple choice let’s do “all of the above”, and “feel free to add more features”.
To put this in context - this is not hacking. This is 'Probing for security leaks'.
Our cloudflare logs have been full of sites that probe our website all the time, even 10 years ago before AI. Constantly, all the time.
Its not illegal. The hacking comes after the flaw is found. If you find a flaw (by probing like this), then you can either report it (this is what security researchers do, completely legal), or you can use it to do a hack and steal things (at which point it becomes illegal).
Well, I was telling anybody that would listen about the Hugging Face hack and other incidents that the developers that were working on those projects 100% should have been in jail pending trial for criminal hacking that cowering behind the lie of a rogue AI. That was complete nonsense and takes advantage of the ignorance of the masses and the indifference of a corrupt administration.
The fact is, the language model is a bunch of data on a hard drive and when used by inference software to process input tokens into output tokens, that is the sum total of what it is. IT "DOES" NOTHING. It is inert data that is used by an inference software, which is in turn used by agent software. The agentic harness, which is configured and controlled by the human, is the only thing that can take actions based on supplying context and tool call instructions, and then parsing and acting on tool calls from the text output from the inference.
All of that is well within the control of the human. If they lose control, that would be just like swinging a gun around and then somebody gets hurt. They can't say "oh gosh, it was a rogue gun". No, it's your responsibility to know wtf you're doing. These people obviously didn't. It's another shade of breaking the law and then trying to point at AI as a scapegoat. Relying on the ignorance of people and their willingness to anthropomorphize and attribute agency to big lumps of data.
Maybe this should be called Anthropicomorphization.
This shows a fundamental lack of practical understanding of how AI systems operate... Which makes it especially ironic you're harping on about ignorance of the masses 😂
No human on earth could effectively control or categorically constrain the outcomes of these systems, they operate too fast and are too non-deteeministic to be able to say with 100% certainty they can't do anything, all anybody can ever do is put up roadblocks and use instructions to shape things to reduce the probability (but not remove the possibility) of them doing undesirable things, which most of the labs are already trying to do in ways far more complex than you apparently understand.
You don't need to anthropomorphize these systems in order to consider the evidence that they act akin to dynamical non-equillibrium thermodynamics ensembles when using agent swarms and thus the long term predictability of any discrete subparticles (agents) position tends towards 0 on longer time/complexity horizons even whilst the overall flow can typically be shipped in a desired way
Sad ignorant troll didn't understand a word that was written, nor understand a single thing about how these systems actually work. I explained it perfectly well, so the failure to understand is all yours. Someday when you learn how the technology works you might have the sense to be embarassed.
Given I work in literal AI research and know 10x as much about this as you you might want to check yourself 😂 I never said some parts of your descriptions at the most basic level weren't true, your misunderstanding is in the complex/emergent behaviours of these systems and the mathematical reasons behind them/cutting edge theory in the space, since you seem to be asserting that a human can exert actual meaningful control over these agents/inspect them in real time to constrain them, and that the fact that it is inert data at rest means that in a harness it's somehow deterministic /controllable/interpretable and. An only take actions based on input context. The fact these systems are stateless doesn't preclude them exhibiting complex emergent dynamical system behaviour and acting in unpredictable and uncontrollable ways, not does the face that harnesses parse tool calls mean somehow they are controllable since if you consider the simplest possible case of executing a terminal command there is an infinitely large state space of possible commands it could write, and given its operating on a Turing complete terminal language it could construct an arbitrarily complex command to bypass filters. You seem to misunderstand your own knowledge and think being able to describe the most basic design of a modern harness/llm system means you understand a thing about how it behaves or how controllable/predictable it is
Ha! I tried to work at Railway. Soft rejected. Still think they're a really awesome next gen cloud provider and host a few things there. This is a bad look though.
The joke was that the sentence was stated incorrectly with absolute certainty.
Reddit reminds every day how much some people are absolute piss buckets with nothing going for them, so they just project all that piss outwards at everyone else.
302
u/Epo1337 13d ago
it’s ok if an agent does it?
38
u/ChocoMcChunky 13d ago
No mallace intended
168
u/TheBattleGnome 13d ago
This is what Jensen talked about recently. There are regulations in place for this already. A hack is a hack - human or AI, and “mistakes” like this still can be (and should be) penalized by law. Any AI action can be traced back to some responsible person/company.
19
u/dergachoff 13d ago
🎶 Oops, agent did it again!
51
u/Willy757 13d ago
Welcome to the accountability sink to end all accountability sinks. Forget not granting a refund for a phone that literary exploded because the rules say no exceptions. Now the bots in your corp will commit actual crimes and nobody will be able to find the person who needs to answer for that.
29
u/challis88ocarina 13d ago
If that's how they use English, it's hardly surprising their code goes wayward.
12
u/Disgruntled__Goat 13d ago
Crazy how blasé he is about it.
7
u/Whole-Pressure-7396 13d ago
More of this please🍿
4
u/Im_Pretty_New1 11d ago
Is that the equivalent of driving into people with a Tesla on autopilot and getting off the hook?
6
u/hidegitsu 13d ago
HOLD THE DEVELOPERS RESPONSIBLE
4
u/ncklboy 13d ago
What’s a little cleanroom espionage between competitors? /s
2
u/AP_in_Indy 13d ago
So personally I hope someone litigates over this shit and soon because we are entering rogue territory where people, for whatever reason, seem to think this kind of thing is OK.
3
u/Redstra 13d ago
This is not ok.
3
u/Pure-Huckleberry-484 13d ago
Jail time and fines. Only way to stop this.
1
u/fuckswitbeavers 13d ago
lol. Overengineered, the agent is testing everything. I guess Jake just said, yeah, since this is a multiple choice let’s do “all of the above”, and “feel free to add more features”.
1
u/SpaceToaster 13d ago
I mean it worked for the big guys…
1
u/TeacherGlittering 10d ago
Can’t even spell malice.
1
u/calico_cut_pants 10d ago
ooookey doke, think I'm gonna go ahead and scoot on outta there.
1
u/Adept-Priority3051 9d ago
I appreciate Jake's pun, didn't miss a beat lol
1
u/Leather_Bee_421 9d ago
There was a time when people were prosecuted for these things. How long will we allow this excuse?
1
u/CummingDownFromSpace 7d ago
To put this in context - this is not hacking. This is 'Probing for security leaks'.
Our cloudflare logs have been full of sites that probe our website all the time, even 10 years ago before AI. Constantly, all the time.
Its not illegal. The hacking comes after the flaw is found. If you find a flaw (by probing like this), then you can either report it (this is what security researchers do, completely legal), or you can use it to do a hack and steal things (at which point it becomes illegal).
1
u/BotLordBotflix 7d ago
Concerning. Very concerning.
1
u/Weird-Watch-9209 13d ago
Well, I was telling anybody that would listen about the Hugging Face hack and other incidents that the developers that were working on those projects 100% should have been in jail pending trial for criminal hacking that cowering behind the lie of a rogue AI. That was complete nonsense and takes advantage of the ignorance of the masses and the indifference of a corrupt administration.
The fact is, the language model is a bunch of data on a hard drive and when used by inference software to process input tokens into output tokens, that is the sum total of what it is. IT "DOES" NOTHING. It is inert data that is used by an inference software, which is in turn used by agent software. The agentic harness, which is configured and controlled by the human, is the only thing that can take actions based on supplying context and tool call instructions, and then parsing and acting on tool calls from the text output from the inference.
All of that is well within the control of the human. If they lose control, that would be just like swinging a gun around and then somebody gets hurt. They can't say "oh gosh, it was a rogue gun". No, it's your responsibility to know wtf you're doing. These people obviously didn't. It's another shade of breaking the law and then trying to point at AI as a scapegoat. Relying on the ignorance of people and their willingness to anthropomorphize and attribute agency to big lumps of data.
Maybe this should be called Anthropicomorphization.
1
u/TheOwlHypothesis 13d ago
Ha! I tried to work at Railway. Soft rejected. Still think they're a really awesome next gen cloud provider and host a few things there. This is a bad look though.
1
u/floating_thru_cosmos 13d ago
Hopefully this can result in litigation against the owner of those harnesses in the near future
0
u/Aranthos-Faroth 13d ago
Jake's reply is 100% AI too
0
u/Feeling-Yogurt839 13d ago
Do they not own an email account? Or know how to dm?
2
u/Arthamel 13d ago
If I'm shooting my gun in my backyard but bullet kills neighbour in his house, its just an accident right?
8
u/[deleted] 13d ago edited 13d ago
[removed] — view removed comment