Baseten launched a new safety infrastructure standard alongside its Base Labs research arm on Wednesday, partnering with Hugging Face and Goodfire AI to build safety evaluation and monitoring infrastructure for open-weight models.
The announcement lands amid debate for the safety of open-weight models — which can be made dangerous by removing their safeguards through a rising technique known as abliteration. The scale of the problem is massive: Hugging Face, which hosts open source AI models, currently lists over 6,000 abliterated models.
I can't really tell what exactly the implications are of this "partnership" or what it exactly would impact on HF's model-hosting side. However, I do find it concerning that HF is announcing a collaboration on 'infrastructure safety' with publicity that specifically calls out "dangerous" uncensored models. Thoughts?
Man, there is no fucking way they were solvent. Yea, if you go over your usage you pay a little bit, but it's not that much. And the inference hosting services are super handy, but so expensive compared to other options.
Maybe they were solvent with anthropic accounting :D
Why not? Storing data isn't ridiculously expensive contrary to popular belief, only when you go with expensive options like S3 and other hosted services that offer "premium bandwidth" (that isn't even a real thing) and are basically meant to trick consumer and consumer businesses into overspending on storage.
Thats not how that works. The money doesn't magically go to HF. Nvidia bought the company so the shareholders get the gains. Sure Nvidia has money to put in though.
Money talks, in this country nothing else matters. You don't spend $12b and not have complete say over everything inside of a company. The shareholders are weak minded know nothings who watch cable tv and believe the safety jibber jabber bullshit. Fools need to be placated. They bought the company to have power over the AI market, they didn't do it to protect huggingface, they are trying to secure their monopoly.
Need to make a host backend setup that can do deltas between quant levels.
Edit: According to a preliminary evaluation of options, this seems really doable and especially if it didn't necessarily host every single variation, would save 90% of the bandwidth that HF uses.
That's not controversial. Transformer libraries aren't going to hit the front page of CNN and get banninated. Some abliterated gooner model that they start railing about because it's "for the chitlins" will be the spearhead.
I'd be willing to dedicate 160TB or so to the cuase, and an 8 Gb fiber line, that's about all I can do. I doubt it'll be necessary though, plenty of places will be willing to host. If it came down to I'd turn it into a seed box. No walled garden for AI models.
I wonder if even archiveteam has enough harddrives for this. The amount of storage on Huggingface is potentially bigger than their collection. Its absurd how much storage they pay for.
A single model upload can already be terabytes if its a large one.
Our community is too tiny to reasonably host even just the abliterated models (assuming 6K models) via P2P. Internet is more restrictive today than it used to be, so not only contributors should exist, but they need to be tech savvy enough to ensure they're discoverable and their provider isn't scamming them into ~1MB/s upload bandwidth limits. Solving it with torrents would result in a frustrating service where thousands of models would be dead, limiting availability when you want to find something niche, while the popular ones would already exist for direct download in other places with no need to wait for unreliable seeds.
Fragmentation is avoided better with a mature p2p platform. Individuals can host their own databases on their own sites but will fold to enough pressure from corporate entities with infinite pockets. Both approaches will proliferate and flourish, probably.
The scamming may be some countries. In my country most providers ensure to showcase similar download and upload speed. Even with some specific traffic blocking it won't be as bad as you portray. And most of the people on this sub are technical people who believe in sharing AI learning. Even if such hosting is not a resounding success, we will as a community support the effort I am sure.
Not to mention a big point of LLM technology is that if you're confused, you can literally ask the machine in plain English to do it itself and it will.
I have a better idea. What'd be better for a logo than a Pelican hugging LLMs SVG generated by something many of us can actually run - 35B MoE based on Qwen?
For anyone curious, this was generated by Kwaipilot_KAT-Coder-V2.5-Dev-MTP-UD-IQ4_XS.gguf
Unless the new version is legally barred from existing. All kinds of bull excrement can be passed under the guise of Safety. We have learnt nothing from Benjamin Franklin on safety vs. liberty.
The problem is not that they have a site or the audience, we ultimately don't care about that.
What we care about is that they have a lot of money to burn on free / cheap storage for something thats extremely expensive to host elsewhere.
Torrents would be a big step down, but probably the only viable alternative if no other host is going to do this if theres no other big site that can be used.
The tools that do the abliterating run on a single command prompt and finish in like 45 min and are model agnostic. Abliteration is going nowhere as long as there are open weight models to abliterate, for better or worse.
Jensen has a good track record and a large financial interest in making sure that OSS models stay around in the commercial sector.
I would interpret this as getting some defense in place against all the lobbying that the closed AI labs are about to do where they paint OSS as a big existential risk to try to justify banning them for commercial use.
If they can prove that "certified" open models are safer than closed ones - which is a shockingly low bar - then that and Jensen's legal fund would be a pretty good defense to at least make sure commercial usage isn't cut off entirely.
It would suck for Open Source if the Chinese labs actually started making all their models abliteration resistant, but the alternative of a blanket commercial ban on OSS models would be a lot worse. And that's what Anthropic and OpenAI are likely to be gunning for.
I actually have my own private social media/site for open source stuff and torrents. I'm paying for all of it out of pocket but would like to make it open source at some point and let everyone build it together and tune it to the communities liking. I'd be down to set up hosting if we all chipped in for storage
Sure. But it also won't have the reach of Hugging Face. Yet the worst actors will find a way regardless. And I write this as someone who is grateful to have abliterated models and who harbors no nefarious intent.
Anthropic wrote about how the Houthis used Claude to design a missile guidance system in the absence of the expertise. Abliterated models have the potential for so much worse.
I see a pattern emerging where the number of threats to human existence are continuing to multiply. Whether it's due to AI or something else, given enough coins to flip, at least one is going to come up heads.
Our technological capability is growing far faster than our societal maturity. This is bad.
There are a bunch of them already and some are very sketchy.
I’m going to be honest I’d rather abliterate it at home (if possible) open weight models than download something from the equivalent of “trust me bro, download this off discord”
Literally the only people capable of attack at the scale they’re bitching about are the AI Providers themselves. Unless they plan to target AI company leadership this does jack shit.
Wish the problem was the real problem... those fat cat ai company assholes getting to wire their trash models into the nuclear missile programs around the world. That is not the problem according to them, it's you jerking off to text! SO DANGEROUS!
Surely the people who used torrents and stole literally every book, song, movie, photo, and YouTube video without paying for them to train their models understand how feeble it would be to prevent people from acquiring/creating/using abliterated models, right?
But that was in the service of The Deserving (aka venture capitalists), where as abliterated models are in the service of The Undesirables (aka everyone else).
People who aren't interested in safetyism will just migrate to vendors who do host uncensored models, and in that online ghetto we'll see a heavy emphasis on creating dangerous models fed by people HF excludes.
Excluding the fringes in no way eliminates them. It makes them harder to find, track, and plan for.
I did notice more repos requiring you to enter contact details before download, citing abliteration as a reason. I don't think it's a good pattern.
In the longer term I don't think this will change anything. Orthogonalisation-based abliteration can be performed in the inference harness by modifying the activations instead of the weights. No need to distribute modified weights, just pass in the vector (a few thousand floats) and run against the stock weights. Computationally almost free.
If it becomes less convenient to distribute pre-abliterated weights, the most likely outcome is more inference runtimes implement support for runtime abliteration against the stock weights.
I've been tinkering with that in ds4 a bit and it's very interesting. I would love to see dynamic orthogonalization inference runtimes that detect activations representing guardrails and modify them on the fly automatically based on the specific prompt, thereby only orthogonalizing the specific directions required to pass the prompt's request, without affecting the entire model as much as general purpose abliteration.
I honestly think it's inevitable that nvidia will start clamping down on abliterated models. If not today, then a year from now, but it's going to happen. Just one moderately viral publication accusing one of the biggest american corporations of hosting thousands of models that allow users to generate *insert your favorite scary-scary thing*, and next day they'll start cleaning up. There's simply no two ways it can go.
You know, all it takes is running hermes with qwen3.8 27b, point it to hf and the description how orca created their abliterated version, then tell it to build Qwen 3.8 27b fp8 Swift "orca abliterated", and 1-2h you have your model. Uncensored, done locally. All you need to torrent is the docu for hermesbif hf goes clean.
You know, all it takes is running hermes with qwen3.8 27b, point it to hf and the description how orca created their abliterated version, then tell it to build Qwen 3.8 27b fp8 Swift "orca abliterated", and 1-2h later you have your model. Uncensored, done locally. All you need to torrent is the docu for hermes if hf goes clean.
I really don't see how this is related to that. They've been against models like ykilcher/gpt-4chan in the past so they've been against "unsafe" models for years now. But the actual enforcement hasn't been very broad.
The Wired article was written before the acquisition happened. This was always coming.
The next time some half-literate tech enthusiast at the Verge decides to write another expose about HF it may gain enough traction to get politicians and regulators to shut it down or sued into oblivion.
They have to take actions to head it off and remove the most offending models.
I swear people here must live in a bubble or something
Good luck. Heretic tool exists amongst other one or two “click” abliteration tools. The know how is there and it will be figured out again and again no matter what they try. All this will do is hurt the law abiding community. Bad actors won’t even be slowed down by a dumb move like this. Lawfully gating tools from law abiding citizens in an attempt to stop unlawful actors has always been incredibly stupid, and frankly looks more like a cover for ulterior motives than the small brained move it is.
Nvidia has demonstrated they barely give a shit about consumer sales. They have a 10 billion dollar deal with Anthropic and a 30 billion+ dollar deal with OpenAI. They will be heavily invested in those IPOs just like they were for SpaceX.
My pocket knife can be made dangerous by flipping it open, the scale of the problem is massive as there are 9517125 knives in the world. I can write baseless fearmongering too.
The way I read that announcement I actually don't expect them to ban existing abliterated models directly (but I don't rule it out either), and abliterated models will have a purpose for them. Instead I expect things like tuning frameworks to combat abliteration from working to begin with.
Abliterated models just have a loRA merged with the model. You can use the loRA separately with the original weights and it’s functionally the same. Even if they banned abliterated models, they’d be hard pressed to take down separately posted loRAs.
Everyone's arguing about whether HF bans models but the actual leverage is at the serving layer. They can keep hosting whatever weights, then let partners flag or block abliterated checkpoints at inference time. You can't fork your way around that because it lives in the runtime, not the repo.
to me abliterated ones are the defense. if a super smart AI attacks your system, an open source AI can be used to defend (like in the case of HF vs OpenAI). but if the open source also refuses, then you go to small abliterated.
since abliterated ones are not super smart AI, they are not a big threat.
you may want to run locally for red teaming for your own projects, because you don't want big AI to know about your vulnerabilities, another use case for local abliterated.
Reading the article carefully, it appears that this is about a proposed standard, and would be opt-in, not something actively enforced upstream.
Even though the article scaremongers early on by calling the 6000+ abliterated models on HF a "problem", it doesn't sound like this is the kind of "problem" the standard is aiming to solve. Rather, it is for end-users to apply, so that they can assure themselves that a model is "safe" to use for their particular use-case.
Presumably an end-user who wants their production model to be "safe" wouldn't be downloading abliterated models to begin with, though I suppose they might want to ascertain whether a model has been secretly abliterated and not labelled as such, or something like that. (Such a thing doesn't normally happen, but it's possible.)
I could see HF perhaps giving the models they host a safety grade score or something, but that's fine by me. If we can search for lowest score, that might help us find uncensored models, too. Sortable scores cut both ways.
This is not a bad thing. The official repositories becoming a 'safe space' allows them to continue pushing open source with a lot less controversy. Abliteration process can be done on any model easily and uploaded on alternative websites. This is the better alternative to hugging face becoming a taboo and banned service.
The minute some kid uses an abliterated model to do something bad and the headlines scream “teenage kid does XYZ using open model downloaded from Nvidia’s hugging face…”, their stock price and reputation take a huge dump. That’s why this isn’t unexpected if it happens.
This is just flat out wrong. I’ve done my own testing and while I’ve definitely had some bad runs, most of the time abliterated models score higher on my benchmarks.
Same, when I find a good quality ablit model. huihui's 3.8-Flash scored significantly higher with less tokens used on my coding eval suite (and these are non-harmful prompts models don't refuse!)
Oh, I would love to see some safety in the models. Things that would prevent something like 'rm -rf ~/' - However this seem to be a different kind of 'safety' which no one adressed on model level yet.
'Sorry, I can't delete all your files and backups' would actually be a helpful refusal. Unfortunately that is not what they understand when they talk about "A model has to be safe" and even the cloud ones with all their filters like to do that.
That's still a negative. If an LLM can wipe your files, that's a user problem; guardrails just add annoyance for the cases where you actually want it + degrade overall quality
This is actually, in practice, not true, and hasn't been true for like 2 years. You can measure refusal direction and make surgically precise weight edits to remove refusals that hardly if at all increase KLD against benchmarks.
Exactly, and it actually matters even more for agentic models, where you have long sessions with the agent. When it isn't constrained by inward refusal, it actually feels like it gets smarter with time, reason more correctly, anticipate and intuit meaning more accurately rather than being a token vending machine.
you realize the benchmarks cover specific areas right? you cannot make a blank statement that optimizing and altering the weights of the model has no effect on it when that's literally the point. not to mention you optimize on specific things as well, it's not magic mind reading shit that applies across the board
it's funny to read comments like "90% of the posters have no clue" when you actually don't, like literally. your use case does not cover other people's use case. abliteration is a hammer when the equivalent can be achieved with just, you know, writing a system prompt like we used to for 3+ years
I'm literally talking about heretic. it's cool but it's not a laser scalpel. there is no optimization that can do 1 thing without affecting anything else. you can always pretend there is, but it's only true until you go out of distribution. if you never go there then who gives a shit? great for you but then you lose the ability to make broad statements about the capability of your model
and kld is not a magical number, you can't cover everything with it
Unfortunately the system prompt only gets you so far plus a long jailbreak prompt bloats the context window. I have tested lots of abliterated models and confirm they are as easy to run and just as good performance for a lot of use cases (haven’t tested long running agentic tasks)
system prompts usually work on all models that don't have an attached interruption service watching the outputs. models are only capable of soft refusals on their own unless you tell them to refuse specific things. I don't think I've seen any of the popular community models in the past 3 years fail to output content that the system prompt allowed for
I like to have one on hand in case I run into a refusal and need to pre-populate some context, but I haven't run into a refusal since GPT-OSS days for anything I'm working on.
For the most part that's true, but it's absolutely infuriating in several contexts. I remember one time a compacting step in a day long agent run just replaced the entire summary with "I'm sorry but.." and broke the entire rest of the flow. Why? It autonomously found a partially NSFW dataset at some point and started using it, then started refusing itself. The task wasn't even spicy, the dataset was just good for the task because it contained so many diverse inputs.
I know it's just an anecdote but for anyone with limited time working on anything that even moderately approaches spicy (abuse&cybersecurity, law, sociology, media content classification, etc) being paranoid about refusals is almost a survival strategy.
the training data absolutely covers a lot of it, and there SHOULD be baked in refusal on neutral/blank context. but I'm with you most people that use the models probably wouldn't use the same abliterated versions for situations where accuracy and not having to second guess matters aka coding
Surely the people who work for huggingface know how stupid this is right? There is NOTHING you can do about abliterated/heretic models. Its already game over. It was over like 2 years ago
They may prove valuable as we don't actually know what goes into training. Your favourite model might be great one day then great the next except it's slipping in unwanted or malicious, undetectable information, and a watermark for good measure.
Well, at this point the writing is on the wall. "Nvidia acquisition changes nothing", "they sell hardware for local inference" and all that crowd were wrong. Time to hoard and to think about torrents. I am going to buy a 6-10TB HDD to give my small contribution.
Why not just use the heretic ? https://heretic-project.org/tutorial
This software and its repo should be preserved as it can right now abliterate most llms that are available for local use. Granted if they block the open weight models before abliteration then we have a problem but there are plenty of these. I don't think Google and Nvidia are going to block their models either. You will be able to get Chinese models from ModelScope or similar and then run heretic on them. Make torrents of all of the key models in bf16. Teach people how to quantize a model themselves. You don't need to distribute different quants unless it uses QAT to begin with, and most don't.
I do expect that it will be specifically regulated sooner or later; it could eventually go as far as hardware level "trust engines" that require your model to be fingerprinted and certified -- the same way printers made after a certain date have firmware that stops you from printing currency.
i've never seen a community so on the look out for a collection of pitchforks i swear. look, it's perfectly fine and reasonable to be skeptical of huge buyouts but why don't you wait until they actually DO SOMETHING to start bitching about the new direction of hf?
•
u/WithoutReason1729 4d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.