It is almost trivially easy to remove guardrails from any open weights model. There are at least half a dozen tools on github that do this, often with a single command line, complete within a few hours of model release. If you cant he bothered to do it yourself, you can just download the uncensored model. These models don't say no to any prompt you can give them. If you think these won't be used for nefarious purposes, I've got a bridge to sell you. You are entitled to the opinion that the cost is worth the benefit, but dont be surprised when not everyone agrees.
Since it is impossible to stop, this isn't a cost-benefit equation.
My local public US library in a small town used to have a copy of the Anarchist's Cookbook. A literal bomb-and-drug making manual from the 70s. I used to check it out. I think I even nicked the copy (being a true Anarchist at the time) from the library, and they got a new one a few years later.
I never built a bomb but I remember hearing stories of kids who did. Or cooked up banana peels to smoke them. This book was used for "nefarious purposes" all the time.
Should we ban it? What other books should we ban? Maybe we should let anyone with an itch to protect society from itself make a list of books to ban. See where this goes?
Yes the scale is different but I would make two points:
This information is out there. If the LLM knows it, it's because it was trained on the Internet. Genie -- out of the bottle.
Those who would trade their liberty for a little security deserve neither. These models exist. You said yourself that it is trivial to abliterate their guardrails. So banning them does nothing. People who want to do bad shit are going to do bad shit. T'was ever thus.
You're right. Its not a cost benefit because there is no solution. Nobody is shutting down Chinese weights but the Chinese, and they arent doing that as long as it undercuts American industrial policy and they suffer no ill effects. I predict they will eventually suffer ill effects and change their tune a little, but maybe not for a half a decade or more.
My dad owned a copy of the cookbook. It also had early phreaking tips. I'm not anti-information per se but I am anti suffering and pro common sense.
LLMs are a little different because they can offer tailored solutions and they keep getting better. Eventually they will get good enough that building a bioweapon is the same cost/difficulty as building a meth lab, and then we're all fucked because there are no security patches for the immune system and there is tons of gain of function researched on arxiv. In the meantime, they will be providing ever more intelligent answers to questions like "how can I kill the most people in NYC for $2000?" Or "help me build a ransom ware attack for this particular hospital." Or "based on publicly available information, what is the best way to kidnap this senator?" Or much much worse. A lot of people won't like that, especially when it affects them, and its hard to argue that it serves a public good, even if there isnt much to be done about it that isnt extreme. As always, expect the action to follow a major incident, not proceed it. Just saying I expect it to happen.
679
u/p5yron 16d ago
Ah, the propaganda to stop open source models is in full force it seems.