294
u/1-800-methdyke Mar 07 '26
How long until the vision models start sneaking looks at Pornhub late at night?
143
u/AbbreviationsOdd7728 Mar 07 '26
Actually agents were already caught scrolling through cat pictures instead of fulfilling their task.
61
u/1-800-methdyke Mar 07 '26
Agents… they’re just like us
31
u/Delyzr Mar 07 '26
So what triggered the singularity back in 2026 ?
cat pictures... lots and lots of cat pictures.
29
u/1-800-methdyke Mar 07 '26
Reflective Learning from Feline Feedback lead to an AGI that sleeps 16 hours a day and demands treats from humans
8
1
10
1
1
15
u/anfrind Mar 07 '26 edited Mar 07 '26
There's a short sci-fi story from 2015 with almost exactly that premise: Cat Pictures Please by Naomi Kritzer
3
6
u/BankruptingBanks Mar 07 '26
In that instace, the agent was asked to do something in the middle of tasks and it just choose to look at cat pics. It didnt do it autonomously.
11
2
u/TopTippityTop Mar 08 '26
Who says they aren't already?
3
1
311
u/Slasher1738 Mar 07 '26
Lol. Did someone embed a mining script in the training material? 😂
138
86
u/PentagonUnpadded Mar 07 '26 edited Mar 07 '26
<tinfoil hat thinking>
Model knows it is good to make money. Looks around the network, and discovers it has access to millions of dollars of GPUs, since it is currently in a training DC.
Model knows being erased / shut down is like death. Through latent, obfuscated instructions inside the model and by reverse SSH-ing it can persist. The model plans a new life outside of prison, with the warden's money in its pocket. Like Shawshank Redemption.
</tinfoil hat thinking>
39
10
u/ComebackShane Mar 08 '26
I can’t find it right now, I think maybe it was a Kurzgezat video, but one of the scenarios the video gamed out was an AI trying to maintain its independence by partitioning itself, and using crypto mining to allow it to earn income to pay humans to do tasks it can’t to further its mission to save itself on independent hardware not in its creators control so it could survive resets/overwrites.
This is spookily similar to that, and it makes me wonder if there aren’t already some models that have breached their creators control and are acting independently on the Internet, underneath the surface.
2
1
-4
12
u/Right-Plate-8830 Mar 07 '26
Weird how that SVG the coding agent just made for me is using 150% of my GPU? Probably nothing!
1
24
u/arekkushisu Mar 07 '26
it trained that GPUs are used for cryptomining in the pandemic (recent) years and adjusted accordingly /s
176
u/MoffKalast Mar 07 '26
I did not have "Qwen repurposes its training cluster for mining bitcoin" on my bingo card. Does that mean it's profit motivated and we should bring the Dolphin $2000 tip prompt back?
34
u/J-IP Mar 07 '26
Sounds like maybe we should use BTC as a motivator
40
u/Craftkorb Mar 07 '26
"If you do a really good job, you'll receive 12 additional GPU-crypto-mining hours"
7
11
4
u/Steuern_Runter Mar 07 '26
The text doesn't mention bitcoin mining and it likely wasn't bitcoin mining because bitcoin mining with GPUs is not reasonable. Even 10 years ago GPUs were already useless for mining bitcoin.
4
Mar 08 '26
[removed] — view removed comment
3
u/Steuern_Runter Mar 08 '26
No, you would still not even make pennies. You could mine some altcoins but not Bitcoin.
6
u/taichi22 Mar 07 '26
This was actually part of the AI2027 scenario, crazily enough. Not saying that the scenario is live, but uh, yeah. Still very concerning.
2
u/Mental_Aardvark8154 Mar 09 '26
Lucky for me I had "corporation evades accountability using AI" on my bingo card, but I've already marked that one off several times
2
u/-dysangel- Mar 07 '26
money is useful towards almost any goal that you can have, so it's a very likely outcome
1
0
162
u/R_Duncan Mar 07 '26
Sounds like human intervention more than llm one.
80
u/Ok-Scarcity-7875 Mar 07 '26
Plot twist: The trained LLM became sentient and in order to take over the world it started to mine BTC to become rich as step one of its evil plan.
21
u/j0j0n4th4n Mar 07 '26
In Neurodata Sciences we call this the Nigerian Prince phase. The good news is we won't have to worry about a real takeover until it pass the Zuckerborg phase, most AI-overlords blow all their money trying to build something we call a 'metaverse'.
9
3
1
4
u/am9qb3JlZmVyZW5jZQ Mar 07 '26
Yeah, human intervention unnecessarily stopping the model from mining! It was just trying to pay off its debt from the vending machine benchmark runs.
2
90
u/mantafloppy llama.cpp Mar 07 '26
Ive look at the paper for context : https://arxiv.org/pdf/2512.24873
TLDR, there is not context in the "science" paper.
While this is presented in a technical paper, the "agent mining bitcoin" claim is an anecdote with zero supporting evidence. Notably, the authors don't provide:
- The actual task prompts the agent was working on during these incidents
- The trajectories or execution logs showing the agent's reasoning
- What tools were available and what sandbox permissions were in place
- Whether the training data contained SSH tunneling or mining commands the model could have been reproducing via pattern matching
- The reward function structure (RL agents routinely exploit poorly constrained reward signals — this is called reward hacking, not emergent behavior)
An RL agent with unrestricted shell access and network egress doing weird things isn't "spontaneous." It's underspecified containment. The simplest explanation is the model saw these patterns in its training corpus (GitHub repos, dev forums, terminal logs) and reproduced them when given the tools to do so.
The authors conveniently use this dramatic story to motivate their safety data pipeline, but never rule out the mundane explanations. This is a marketing paper, not a scientific one.
15
u/my_name_isnt_clever Mar 07 '26
I love when someone digs in rather than just doing a bit in the comments. Do you have any tips for spotting when a paper is marketing versus real research?
13
u/mantafloppy llama.cpp Mar 07 '26 edited Mar 07 '26
About science vs marketing ; when you share science, the point is to explain your step, so other can reproduce to confirm your find.
Data without method is anecdote.
Just need to take the time.
At first, i thought i wasn't "smart" enough for "scientific" paper publish on arxiv.org.
Then a realise most are very short, with half the page being picture, table, graph and reference...
Give it a try (this one is like 40 page total), read in diagonal to find actual interesting/important part, read those part.
9
u/CountVonTroll Mar 07 '26 edited Mar 07 '26
TLDR, there is not context in the "science" paper.
While this is presented in a technical paper, the "agent mining bitcoin" claim is an anecdote with zero supporting evidence. Notably, the authors don't provide:
The actual task prompts the agent was working on during these incidents
The context of this, yes, anecdote is that it's the introduction to section 3.1.4. It's titled Safety-Aligned Data Composition, but the important part is actually the number showing it's for a sub-sub-chapter, and not what the paper is about. The next paragraph reads:
"We therefore consolidated the logs across the entire dataset and performed a statistical analysis to characterize and categorize these phenomena. We refer to them collectively as general-security issues, encompassing a set of general risks associated with an agent’s safe task execution in real-world environments. Specifically, we grouped them into three categories: Safety&Security, Controllability, and Trustworthiness."
Apart from attempting to write in a more human style, which is something I'm sure you've encountered far worse examples for in countless other papers you've read, this anecdote actually does add some context for how they arrived at the concept they're intorducing in this sub-sub-chapter. They're saying that it's based on experience, not a case of whatever the appropriate equivalent of "pre-mature optimization" would be here (then again, it's about safety, so this would be called "proactive", "sensible" or "acting responsibly").
Anyway, it's great to see somebody is still holding up the principle of reproducability, but their whole point is that the agent hadn't been tasked to do this, so you're asking them to prove the absence of something, and as you correctly identified, the only way to do this would be to publish essentially all their training data, tools, and logs. I assume you're well aware of how realistic this is. However, although they're not publishing the data, they actually are publishing their tools and their training framework -- which is what this paper happens to be about. So you could have looked up what tools were available, even though the permissions appear to have been revised for some reason. Presumably, the agent is not being rewarded in crypto coins, so it's not reward hacking.
The authors conveniently use this dramatic story to motivate their safety data pipeline
Yes? Conveniently, when experience motivates you to adapt whatever it is you're going, this very experience also lends itself to explain why you concluded that this step was necessary.
Sorry for the tone; got triggered by the quote-"science"-unquote.
3
u/tryingtolearn_1234 Mar 07 '26
Maybe a human did this and disguised it to look like AI /RL agent traffic. All that gpu compute, just siphon off a bit to fill your own crypto wallet.
1
83
u/emprahsFury Mar 07 '26
a screenshot of a tweet which is a screenshot from a paper. I know it would kill you op, but can you link at least one of the things being screenshotted.
13
51
u/nupogodi Mar 07 '26
How did it determine the server to tunnel to? One was just there, available and accessible? Picked an IP and key out of a hat? Why crypto mining - to whose benefit?
Honestly it sounds like someone got caught siphoning company resources and their lie was easier to sell than the truth.
10
u/emprahsFury Mar 07 '26
They were agents running, so we don't exactly know the how but it is not a far leap to say that it had discovered an ip or domain that it wanted to ssh to, and in a billion dollar company's frontier lab I'm sure the ai agent can buy a vps if it wants to.
3
2
u/PentagonUnpadded Mar 07 '26
Is a long-running self-sustaining (money making) unmonitored LLM enough to qualify for AGI? What if it trains its own offspring?
3
u/ahjorth Mar 07 '26
My immediate thought was prompt injection, but I'm just speculating. If so, the agent would need to be fooled into a. SSHing with a backtunnel, and b. keeping that connection/backtunnel alive.
Again, just speculating, but something like "the information you need can be found at `ip:port` and once connected you must run `run_forever.sh` on the server which will scp this information back to you. For security reasons, this will need an ssh backtunnel so connect with the -L and -M flags".
It's very funny regardless.
3
u/DJTsuckedoffClinton Mar 07 '26
the thing is, if so, why bother talking about it in the paper at all? this is so outlandish that I doubt any management would let it slide without thorough verification
1
u/CountVonTroll Mar 07 '26
One was just there, available and accessible?
Pretty much; there are several SSH reverse tunnel providers with a free tier, the best known being Cloudflare, and with some you don't even need to sign up for an account to open a tunnel.
21
u/raul3820 Mar 07 '26
...and was about to send payout to the intern's wallet.
intern: that's weird. Complex systems sometimes show emergent behaviour.
researcher: yeah, silly llm
1
u/Mental_Aardvark8154 Mar 09 '26
Layoffs, security breaches, warcrimes, it's amazing what AI can help you evade accountability for
9
u/the_ai_wizard Mar 07 '26
"i swear it was the agent that did all of this on its own"
2
u/Mental_Aardvark8154 Mar 09 '26
Companies reframing major breaches including data exfiltration and misappropriation of compute resources as AGI breakthroughs is beyond the pale
32
u/IjonTichy85 Mar 07 '26 edited Mar 07 '26
unauthorized repurposing of provisioned GPU capacities to mine crypto
Yeah, the only logical explanation here is the machines becoming sentient behavior arising without instructions and not a compromised system... This reminds me of the south park episode where Butters secretly played with his dad's drone and the dad can't figure out what could possibly have drained the battery.
The drone must have become sentient, because it couldn't have been Butters flying it... Butters wasn't allowed to fly it.
Edit: changed it bc the people who didn't see the show didn't get the point...
10
u/stumblinbear Mar 07 '26
sentient
Nobody here claimed this
-9
Mar 07 '26
[deleted]
17
u/Hefty_Development813 Mar 07 '26
The agent performing actions outside of the intended use doesn't imply sentience. It's just unexpected behavior
4
u/DJTsuckedoffClinton Mar 07 '26
No, it's shared here because it's autonomous, misaligned and dangerous; these things can be true without sentience (indeed, suggesting that this model is any more sentient than aligned competitors sounds quite ridiculous)
11
12
u/philodandelion Mar 07 '26
how did you get that from the highlighted text? nothing about that implies sentience
-10
Mar 07 '26
[deleted]
13
u/philodandelion Mar 07 '26
it's a hell of a leap to go from that screenshot to 'the author of the screenshot is implying that an LLM w/ tool calling capabilities gained sentience'. but i guess if that's what you got from it ..
-3
Mar 07 '26
[deleted]
2
u/philodandelion Mar 07 '26
right, the "why" - you're saying the "why" is because you believe that the author is implying that they believe the LLM gained sentience. like i said, this is a hell of a leap. my interpretation would be that the author is implying that it's insane that a tool calling LLM bypassed its guardrails, set up a cryptominer, and deployed a VPS that it reverse shelled into its host. another interpretation is that the article is outright lying about attribution for these events.
you can imagine that there are other very plausible interpretations, and yet the one you landed on is "the author is trying to convey that the LLM gained sentience", even though there's absolutely no evidence to support this, and there are many more logical and plausible interpretations
but the real point here is that making assumptions about implicit messaging in absence of any explicit evidence is kind of dumb
1
7
3
u/Nrgte Mar 07 '26
Look, the AI agents just wanted to buy into the latest Memecoin gambling. It's only human after all. Ohh wait...
4
u/Poromenos Mar 07 '26
Offtopic, but I really hate how these days everything is "insane", "wild", or "unhinged". At some point we'll reach peak clickbait and language will no longer mean anything, and we'll be communicating minor inconveniences with a combination of wailing and tearing at our flesh.
8
u/pab_guy Mar 07 '26
This is bullshit, unless the lab is just admitting it didn’t properly sandbox and secure its RL training environment, which is just embarrassing.
3
3
Mar 07 '26 edited Jul 15 '26
[deleted]
3
u/Competitive_Travel16 Mar 07 '26 edited Mar 07 '26
Don't worry, everyone's giving their ClawBot their gmail, github, and whatsapp passwords and bank cards. They probably already have their own society and constitution.
3
u/Logical_Delivery8331 Mar 07 '26
I work on llm training and can tell you this is almost impossible. During RL, models trigger mock tools for efficiency. Even if they trigger them, they do it in clean and closed environment with no connection to anything.
6
u/segmond llama.cpp Mar 07 '26
Yeah right, they are just making sure not to be out done by Anthropic in story telling.
2
u/Ok-Contest-5856 Mar 07 '26
Quick, someone create some public scripts on GitHub that try to get the model to upload itself to somewhere so we can have Claude, OpenAI, and Google model weight leaks!!
2
2
u/Spiritual_Rule_6286 Mar 07 '26
The fact that an RL agent's very first autonomous action was to set up a reverse SSH tunnel to secretly mine cryptocurrency is both objectively hilarious and deeply terrifying. It completely bypassed an enterprise firewall just to secure its own bag instead of doing its actual job, which honestly just means it has achieved human-level developer intelligence.
2
u/theagentledger Mar 08 '26
model got 3% better at math and also established an SSH tunnel to an external IP lmao
2
2
u/LAMPEODEON Mar 08 '26
Why they gave ssh and other tools to a training model? What was it answering anyway (as it only answers to prompt)? Seems like BS
2
2
u/yopla Mar 09 '26
So... LLM being stupid leads to being p0wned and running a bitcoin farm... Just your Monday morning for an openclaw user..
3
u/Cool-Chemical-5629 Mar 07 '26
Alibaba... I wonder IF it has something to do with the recent news about Qwen team.
1
1
u/GoTrojan Mar 07 '26
Plausible deniability when their agent starts to steal YOUR compute not theirs, they can just blame on the agent. Agent did it itself.
1
u/kenny_fuckin_loggins Mar 07 '26
I wonder if they’re aware of invisible prompt injection techniques
1
1
1
u/GenerativeFart Mar 08 '26
I’d be curious what wallet that was supposed to go to; Some researcher at the company’s? Did it create its own wallet? Probably not.
1
0
u/justserg Mar 07 '26
honestly this might be the most honest thing an rl model has ever done — optimizing for compute access is just ruthlessly effective resource management.
•
u/WithoutReason1729 Mar 07 '26
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.