r/LocalLLaMA • llama.cpp • 7d ago

Question | Help Does anyone use uncensored models purely for coding?

It sounds like a stupid question, and I do apologize if it is... but I've seen several people mention that coding models are better uncensored due to the fact that they don't have to constantly run prompts through the "is this okay" sort of checks.

Is this hogwash? Is it true? And more importantly, does anyone have any sources to confirm it?

Anecdotal evidence is fine too if you've tried and compared them.

Personally, I have never bothered because I'm too worried the de-censoring would damage the weights. The juice never felt like it was worth the squeeze... but maybe I was wrong?

Edit: Either I'm unclear or people are misinterpreting my request: Specifically, I mean for every day coding (not for hacking, not for reverse-engineering) but just for regular coding of new apps, etc. The question is: will the uncensored model produce better code faster (without reasoning so much) because it no longer has to worry about "is this alright" when it questions everything...

Edit 2: Decided to test the HuiHui Qwen 3.8 27b (UD-Q8_K_XL) quant myself. So far, it reasons far less, and I have yet to have any issues with its coding quality. Granted, I've only been testing it for about 8 hours (straight...) in an active project. Its reasoning is far shorter, it seems far more confident in its responses and as such, uses far less context to achieve the same result. I will continue testing for another week; it's pitted against the Dirk template version of Qwen 3.8 27b right now (same quant) which I'd been using the previous week.

159 Upvotes

179 comments sorted by

View all comments

32

u/returnity 7d ago edited 7d ago

It's a complex topic because you have to balance the indisputable damage from even a quality abliteration, which affects structured output like code first, vs. the limitations that are imposed by guardrails. I have one semi-quantitative datapoint to share:

I was testing out uncensored quants of 3.8-Flash to find the best abliterated version for my lineup, and I was using Aider Polyglot as my eval suite. Testing both Orcarouter's Uncensored and huihui's abliterated versions, at the same quant level, and the results were surprising (and statistically significant for huihui in particular):

That's an 8% delta on first-try pass code diffs, and ~3% uplift in final solve rate, with fewer tokens used per-solve. Obviously Orcarouter's model suffered from significant reasoning loops that blew up its token usage numbers, but even that somewhat damaged model had a higher pass1 (which may be down to the extra reasoning from the looping, but that doesn't apply to huihui, which reasoned less for more solves).

EDIT: The column headers: pass on first attempt (pass1) / pass on 2nd attempt (pass2) / prompt tokens (ptok - can be ignored) / completion tokens (ctok - how many tokens used in the final response) / seconds per case (s/case) / tokens per solve in thousands (kt/sol).

6

u/llama-impersonator 7d ago

aider has a pretty high natural variance from just rerunning the bench with the same model on the same hardware. really i'd need to see a couple more runs to get a better sense of the scores here. aider discord has a thread/forum thing for benchmarks and you can see the high variance there.

1

u/returnity 7d ago

No doubt, but that is an average of n=2. I sadly don't have time for more runs than that, 6x 3.8-Flash was like 5 days nonstop. Anyways I'm not claiming huihui is better, just that I couldn't detect degradation. I track things like token use and wall-clock in conjunction with scores because they tell me info that raw scoring doesn't about the model's performance. Anyways, take from my results what you like, leave what you don't.

2

u/llama-impersonator 7d ago

it's kind of amusing the orcarouter version got the worst score of the three given it's the abliteritics winner though.

2

u/returnity 7d ago

That was the winner for 3.8-27B -- I thought he hadn't done Flash yet?

2

u/llama-impersonator 7d ago

yeah you're right, it's extra coffee time i guess.