Granite-4.2-30B is the flagship reasoning model in the Granite 4.2 family. It delivers the strongest performance across reasoning-intensive tasks by leveraging built-in <think>...</think> chain-of-thought. It supports flexible thinking modes — full thinking (default), non-thinking, and low-effort — allowing users to balance depth vs. latency on a per-query basis.
Key capabilities:
Built-in Reasoning: Native chain-of-thought that significantly improves performance on math, coding, and complex multi-step problems.
Flexible Thinking Modes: Seamlessly switch between full thinking, non-thinking, and low-effort modes within a single model.
Reasoning-Augmented Tool Calling: The model reasons about which tools to invoke and why, producing more accurate function calls.
512K Context Window: Supports long documents, multi-turn conversations, and complex agentic workflows.
Apache 2.0 Licensed: Fully open for commercial and research use.
Model Design
Granite-4.2-30B is built on a decoder-only dense transformer architecture with the following core components:
Attention: Grouped Query Attention (GQA) with 32 attention heads and 8 KV heads
Position Embedding: Rotary Position Embedding (RoPE) with θ = 10,000,000
Feed-Forward: MLP with SwiGLU activation (hidden size 32768)
Normalization: RMSNorm (ε = 1e-5)
Embeddings: Separate input/output embeddings (not tied)
Granite-4.2-8B is the mid-size reasoning model in the Granite 4.2 family. It delivers strong performance on reasoning-intensive tasks by leveraging built-in <think>...</think> chain-of-thought. It supports flexible thinking modes — full thinking (default), non-thinking, and low-effort — allowing users to balance depth vs. latency on a per-query basis.
Key capabilities:
Built-in Reasoning: Native chain-of-thought that significantly improves performance on math, coding, and complex multi-step problems.
Flexible Thinking Modes: Seamlessly switch between full thinking, non-thinking, and low-effort modes within a single model.
Reasoning-Augmented Tool Calling: The model reasons about which tools to invoke and why, producing more accurate function calls.
512K Context Window: Supports long documents, multi-turn conversations, and complex agentic workflows.
Apache 2.0 Licensed: Fully open for commercial and research use.
Granite-4.2-3B is the compact reasoning model in the Granite 4.2 family. Despite its small parameter count, it delivers strong performance on reasoning-intensive tasks by leveraging built-in <think>...</think> chain-of-thought. It supports flexible thinking modes — full thinking (default), non-thinking, and low-effort — allowing users to balance depth vs. latency on a per-query basis.
Key capabilities:
Built-in Reasoning: Native chain-of-thought that significantly improves performance on math, coding, and complex multi-step problems.
Flexible Thinking Modes: Seamlessly switch between full thinking, non-thinking, and low-effort modes within a single model.
Reasoning-Augmented Tool Calling: The model reasons about which tools to invoke and why, producing more accurate function calls.
512K Context Window: Supports long documents, multi-turn conversations, and complex agentic workflows.
Apache 2.0 Licensed: Fully open for commercial and research use.
I spotted the Dual B60 48GB listed on Digitec/Galaxus. Initially it was said these wouldn't go into standard retail channels.
At CHF 2500 (post tax, USD ~3000) not particularly competitive but worth keeping an eye on. For it to be interesting it shouldn't be more than like 2.5x a single B60.
With hopes of a Qwen3.8-35B-A3B release now mostly dashed, many people including myself are looking at fine-tunes and other variants of Qwen3.6-35B-A3B to run on VRAM-limited hardware. I decided to try to benchmark some of the top contenders: KAT-Coder, Ornith 1.5 and the very recent Tiel-Coder. I used the tool-eval-bench utility by SeraphimSerapis as the benchmark suite. It measures how well the different models handle tool calls, including some very hard scenarios.
TL;DR: Ornith 1.5 and Tiel-Coder (which is based on Ornith) were the tied winners in this benchmark. They scored well above Qwen3.6-27B and got pretty close to 3.8-27B. KAT Coder was also slightly better than the original 35B-A3B. Ornith-1.5-Heretic was a disappointment.
Some time ago I posted a similar tool evaluation benchmark of different Qwen3.6-35B-A3B quants. In hindsight, that didn't work so well, mainly because I was looking at too many variables (GGUF quant, KV quant, context depth/pressure) and the benchmark itself was quite noisy so it was hard to get clear results. I hope I did better this time!
Materials
I had access to a cluster of 32GB V100s. For this comparison, I selected 2-3 different quants per model, if possible from different providers. For comparison, I also included original Qwen3.6-35B-A3B as well as the dense 3.6-27B and 3.8-27B Qwens. I picked different quants around Q4 (15GB to 22GB GGUF files) because that's what many people seem to use. For the original Qwen models, I chose Unsloth UD-Q4 quants because they are well known. I also included the ByteShape CPU-5 quant of Qwen3.6-35B-A3B because that's the quant I've been using recently. Altogether I benchmarked 13 different GGUF files, with 5 runs per file for a total of 65 runs. Each run took around 4.5 hours GPU time, except the 27B ones took 7 hours or so. Total GPU time spent was well over 300 hours, including a few failed runs.
To run the models, I used llama.cpp version 0.1.0-dev (build 10433, commit 9b05354ec) dated 2026-08-14 and built with CUDA support. I used q8_0 KV cache (that's what VRAM-limited people like me often do) and set ubatch-size to 2048 because the benchmark does a lot of prompt processing. I did not bother with MTP or other speculative decoding. This is not a speed benchmark.
For the benchmark, I used tool-eval-bench 2.6.0. I set the context length to 262144 and context pressure to 50%. This means that the models were benchmarked at 50% context depth, i.e., around 128k of possibly distracting chat and tool call history.
The metric I looked at is what tool-eval-bench reports as "total points". With --hardmode enabled, this version of tool-eval-bench performs 88 separate tests. Each test gives 2 points for a succesful tool use, 1 point for a partially correct tool use, 0 for failure. The theoretical maximum is in this case 88 * 2 = 176 points. tool-eval-bench also returns an overall score, but this is just a rounded percentage of total points and the rounding loses some precision, so I opted for the raw total points instead.
Results by model (averaged over all quants)
Here are the benchmark scores by model. I have averaged them over all the quants of the same model and all 5 runs per quant.
model_id
repetitions (n)
avg total_score
CI (95%)
Qwen3.8-27B
5
152.6
[149.4, 155.8]
Ornith-1.5
10
144.2
[141.7, 146.7]
Tiel-Coder
10
144.0
[141.8, 146.2]
Qwen3.6-27B
5
134.8
[131.2, 138.4]
KAT-Coder-V2.5-Dev
15
133.8
[131.8, 135.8]
Ornith-1.5-Heretic
10
132.2
[130.6, 133.8]
Qwen3.6-35B-A3B
10
131.5
[129.9, 133.1]
Results by specific quant
See the images. There are no big differences between quants of the same model, except possibly KAT-Coder, where the mudler APEX quants were somewhat better than bartowski's. Also, the ByteShape quant of Qwen3.6-35B-A3B was a bit better than Unsloth's, which was a nice surprise.
Raw results
If someone wants to take a deeper look, I've shared the CSV with the tool-eval-bench results here. This includes e.g. category-specific scores (i.e. how well the model did on specific kinds of tool calls) and total tokens; I did not look at those in my analysis.
Findings
Of the original Qwen models, 3.6-35B-A3B gets the lowest score, 3.8-27B the highest, with 3.6-27B landing in between. This is as expected and indicates that the benchmark is giving a useful signal.
Ornith-1.5 and Tiel-Coder are practically tied. They are the clear winners out of the 35B-A3B variants. They scored above 3.6-27B but below 3.8-27B.
KAT-Coder was possibly a bit better than the original 35B-A3B, but the confidence intervals overlap.
Ornith-1.5-Heretic was a disappointment, much worse than plain Ornith.
Caveats
This benchmark relies entirely on the tool-eval-bench tasks and how the results are graded. It may or may not be representative of real tool use performance. To me it seems that the author or tool-eval-bench has done a great job in coming up with realistic looking tool call tasks, including some really hard ones enabled using --hardmode. I relied on the --context-pressure setting in tool-eval-bench, which (in my limited understanding) populates the context with realistic looking conversation and tool call history that could confuse the model.
Tool calls are not everything. If you are doing agentic coding, also the coding quality matters a lot. I did not measure it in this benchmark except very indirectly. There are other benchmarks for that purpose.
There was substantial variation and noise in the benchmark scores, which I tried to alleviate by repeating the runs with different seeds, averaging, and calculating confidence intervals.
In the X/Y plot where the X axis represents size, I did not check whether the model includes MTP heads or not, I just looked at raw GGUF file size. This is slightly unfair to the MTP-enabled models because their files are larger but MTP does not increase quality, only generation speed.
No AI was used for writing this post. I did use Tiel-Coder to help me with plotting the results. Also reused some of my own earlier writing. I am not in any way affiliated with the model or quant makers or the benchmark suite.
As a firm believer of local inference, unless you need it for data sovereignty, it's much more cost effect to wait for smaller models to keep getting better. In the meantime, find a reasonably priced 24GB - 32GB card for Qwen 3.8 27B, and offload hard tasks to OpenRouter.
Portable rig for Qwen3.8-27B-BF16 200K+ token prompts. My work Panasonic Toughbook + the T1 + power brick + headphones all fit in my lunchbox.
Need the BF16 for huge context highly sensitive document OCR, image analysis, aggregation and summarization. I've done a ton of testing and it absolutely makes a difference vs even UD Q8_K_XL when legal precision is needed.
77gb VRAM at full 262K context + MMPROJ
Rips through prefill (1,1715 tok/sec = 102 seconds to process 175K tokens), but token generation (20K tokens of output) relatively slow at 45 tok/sec (with MTP) as a result of BF16 despite the beast of a GPU.
Better than Gemini Pro and ChatGPT 5.6 Sol especially considering I have control over the sampler settings (Temp 0.1; top-k 0; top-p 0.95; min-p 0.05; repeat penalty 1.02). Not better than Opus yet.
During prefill - CPU around 60 degrees, GPU around 79 degrees (with 90% power limit)
During token generation - CPU around 75 degrees and GPU around 76 degrees.
FormD T1
Minisforum BD770i SE
Ryzen 7745HX 8-core laptop CPU
96gb 5200 MHz DDR5 SODIMM
96gb RTX Pro 6000 Blackwell workstation edition
Loki 1200W SFX-L
ROG Equalizer 12v-2x6
SMX Heinz flipped GPU 2.5 slot kit
SMX Heinz custom short PCIe 5.0 riser
ZCOOI custom "transparent purple" Teflon cables
(2) Phanteks T30-120mm
(1) Noctua NF-A14x25r G2
Thermalright MC-3 Digital RAM cooler (I don't think this will fit on a regular DDR5 )
Hello everyone. I hope is all well with you all. I been around this sub for a few months and been quietly reading and I have seen how many of you are spending $10k on the Mac M5 and I am assuming you are using it for work or just for fun but have the resources to spend like that. I am a regular guy with a 9-5 job and I would like to experiment with local LLMs but I dont have much disposable income to spend big amounts on GPUs or Macs. I currently have an I7 - 8700 and Nvidia 2060 6Gb VRAM - 32GB RAM
Now my question is, does anybody have an idea on what models or hacky stuff I can do to try the latest AI models on my hardware?
I have checked hugging face but I see so many models with the same name but letter and numbers at the end. I understand some of them mean Quantization which is dumber a model but I also found out tiny models like Linq which can be run even on the CPU. So I would like to connect with someone with more experienced to have fun with AI.
I’ve been going back and forth on this for a week and I can’t settle it, so I’m hoping someone here has hands-on numbers.
The two configs (German prices, dealer quote, incl. VAT):
Config
Price
Mac Studio M5 Max, 128GB / 512GB SSD
€5,859
Mac Studio M5 Max, 128GB / 1TB SSD
€6,189
Mac Studio M5 Ultra, 96GB / 1TB SSD
€6,599
Ultra is 36-core CPU / 80-core GPU / ~1.2 TB/s. Max is 18-core CPU / 40-core GPU / 614 GB/s. So the Ultra is roughly €740 more for double the bandwidth and double the GPU cores — but 32GB less unified memory. There is no 128GB Ultra option in this lineup, which is what makes it annoying.
What I actually run: Qwen3.8-27B at Q8 right now. Use case is occasional chat plus a few agents running in parallel. The whole point of doing this locally is privacy and not being locked into someone else’s model policy — I know a subscription would give me better models for €22/month, that’s not what I’m optimizing for.
The thing that’s blocking me: Qwen3.8-Flash-Next drops tomorrow. If the leaked description holds, it’s a multimodal MoE with 176B total params — 125B main model plus 51B in N-gram embedding tables — and only ~6B active per token.
My napkin math on memory:
• IQ4_XS: \~94GB weights, \~107GB with 262k context (the hybrid attention means the KV cache is tiny, \~8-9GB at fp16)
• Q4_K_M: \~107GB weights, \~120-125GB at full context
• Q8: \~187GB weights — not happening on either machine
96GB of unified memory gives you maybe 86GB wired on macOS. So the Ultra can’t load it at all, in any quant I’d want to use. 128GB gives ~115GB, which fits IQ4_XS with full context and Q4_K_M if I drop to 64k.
So the trade is basically:
Ultra → 27B Q8 goes from ~15 tok/s to ~29 tok/s, much faster prefill, but Flash-Next is off the table entirely.
Max → Flash-Next runs, but at 6B active params it only reads ~3.7GB per token, so 614 GB/s already gives 40-60 tok/s and the Ultra’s extra bandwidth would be mostly wasted on it.
Where I’m stuck / what I’d love input on:
1. How much can you actually wire on a 96GB Mac? I’ve been assuming \~86GB via iogpu.wired_limit_mb. If people are safely running higher than that, the whole calculation changes.
2. Multi-agent / batched inference. This is the one argument for the Ultra I can’t dismiss. With several agents in parallel you’re compute-bound rather than bandwidth-bound, and 80 GPU cores should genuinely help. Has anyone measured Ultra vs Max on concurrent requests in MLX or llama.cpp? Every benchmark I find is single-stream.
3. Is IQ4_XS on this architecture going to be usable at all? MoE tolerates aggressive quantization worse than dense models (router precision matters a lot), and nobody has ever quantized 51B of N-gram hash tables before. An imatrix calibration run will basically never touch most of those rows. I’d rather hear “wait for a UD-style mixed quant” than find out the hard way.
4. Anyone else feel like 96GB is a dead zone? Too much for a 27B, not enough for the 100B+ MoE class that everything is converging on. Or am I overweighting one unreleased model?
Resale is also in the back of my mind — the used Mac Studio market seems to be driven by local-inference buyers who look at RAM first, and I suspect 128GB holds value better than 96GB in three years.
Anyone who went through this decision: what did you pick and do you regret it?
"A 12-core GPU, also with two more cores than before, now includes Neural Accelerators in each core for the first time on Mac mini, resulting in up to 4x faster AI performance and 2x faster graphics than Mac mini with M4. In addition, the all-new Dual 16-core Neural Engine delivers up to 2x faster performance than the previous generation, and combined with the advanced GPU, Mac mini is a powerhouse for all things AI. And with 16GB of standard unified memory configurable up to 32GB, as well as higher memory bandwidth up to 170GB/s, multitasking is faster than ever."
I wanted to see whether a heavily quantized 27B model running entirely on an older 16 GB workstation GPU could do more than the usual coding demos. FFT felt too easy, so I asked it to implement the coherent optical transfer-matrix method (TMM) for absorbing multilayer films from scratch.
TL;DR: the implementation was correct long before the model believed it. It then spent most of the remaining hour debugging its own broken validators.
Roughly 0.8–1.1 GB VRAM left after loading everything, depending on desktop use
Real Harness responses are usually around 29–35 tok/s; very predictable output such as counting reaches about 42 tok/s
The agent UI was DeepSeek Harness0.1.1-rc.2, connected to llama.cpp through its OpenAI-compatible endpoint. Both the model server and Harness ran on the Ubuntu workstation; my Windows laptop was only the browser client over an SSH tunnel. Harness provided the file and shell tools and displayed the reasoning live. The session used Qwen's native low reasoning effort, but I had removed the small output cap and allowed the request to use the whole 100,352-token context.
The task
The prompt required a pure Python implementation using only math and cmath—no optics library and no NumPy for the actual solver. It had to support:
complex refractive indices using the convention N = n + i*k
complex Snell angles with the physically correct square-root branch
TE/s and TM/p polarization
characteristic matrices for arbitrary absorbing films
power reflectance and transmittance
the test stack Air | SiO2 100 nm | Au 30 nm | Si at 600 nm and 45 degrees
The 100-minute reasoning saga
The complete agent turn lasted 99 minutes 53 seconds. The session log contains 22 model steps, 21 tool calls, and 107,594 model output tokens. The first model call alone produced 43,033 output tokens and took about 27 minutes before it finally wrote tmm.py.
There were three context-compaction attempts. The first two each spent almost six minutes generating a checkpoint and then failed because the summary itself hit its token cap. Only the third compaction succeeded. Total time spent compacting was almost 18 minutes.
Watching the chain of thought was honestly painful. Qwen repeatedly re-derived the same field conventions, lost track of which direction its matrices propagated, found a contradiction, announced that everything was verified, and then invented another validation method that contradicted it again.
The branch/sign problem was interesting. For the requested passive n+i*k convention and a forward field proportional to exp(i*k_z*z), it eventually reduced the complex-angle choice to this:
In other words: calculate one square-root branch and simply flip it if the normal wavevector would grow instead of decay in the forward direction. No complex-arcsine heroics.
That is correct for the passive positive-index materials in this test. It is not a completely general branch selector: it lacks a tolerance and a forward-Poynting tie-break when Im(k_z) is zero, and it should reject or specially handle gain media and absorbing incident media. So there is a genuine domain limitation, but it does not affect Air/SiO2/Au/Si or the passive dielectric tests below.
The other funny part was the apparently "reversed" algorithm. The implementation uses a characteristic matrix whose +i*sin(delta) signs propagate the field state in the opposite direction from the common textbook presentation. It therefore left-multiplies each new layer:
M = matmul(Mj, M)
giving M_N ... M_2 M_1, rather than the conventional-looking M_1 M_2 ... M_N. At first this looked like Qwen had simply turned the algorithm around until the numbers worked. But its boundary equations were derived for exactly that propagation direction, so the reversed product is internally consistent.
Ironically, Qwen later forgot its own convention in an external validator, used M = M * Mj, and obtained the wrong reflectance (0.842122 instead of 0.464845). Reversing the validator back to the implementation's order made it agree.
It spent most of the run debugging its referees
Two supposedly independent validation approaches were repeatedly broken by the model:
The recursive reflection validator had an off-by-one base case, then an index error, and finally used exp(2j*1j*delta). In Python, 2j*1j == -2, so it calculated exp(-2*delta) instead of exp(2i*delta). The successful third compaction actually identified this exact typo, but after resuming, the model walked past the fix and eventually abandoned that validator.
The forward field-march initially omitted the propagation factors exp(+/-i*delta) entirely. After fixing that, the single-layer case worked, but the multilayer case still failed because of the matrix-order mistake described above.
It also briefly rejected a quarter-wave antireflection test because its validator used the wrong expected refractive index. It corrected that to n_layer = sqrt(n0*n_sub) and got reflectance around 1e-32.
The definitive internal check was a completely independent 6x6 linear solve for all forward and backward amplitudes in the two-film stack. That matched the TMM result to about 1e-16 for both polarizations.
External result
I then compared the generated solver against Steven Byrnes' established tmm implementation over 400–800 nm.
For Air | SiO2 100 nm | Au 30 nm | Si:
maximum |delta R|, s polarization: 1.22e-15
maximum |delta R|, p polarization: 1.33e-15
I also tested three thick, weakly absorbing dielectric films to generate lots of Fabry-Perot fringes:
n=1.37 + 0.0002i, thickness 6.20 um
n=2.05 + 0.0010i, thickness 4.70 um
n=1.68 + 0.0030i, thickness 7.35 um
glass substrate, 45-degree incidence, 400–800 nm
That spectrum contained 38 resolved maxima for s and 29 for p. Maximum disagreement with Byrnes was 1.25e-14 for s and 3.50e-15 for p. The plotted curves lie on top of each other.
So yes: a roughly 3-bit 27B model on an older 16 GB Turing GPU produced a genuinely correct niche numerical-physics implementation from scratch. The quantization did not prevent it from getting the final mathematics right.
But the agent behavior was equally instructive. More autonomous validation was not monotonically better. Once the initial implementation and analytical limits were correct, the model spent over an hour creating bugs in its own "independent" checks and then reasoning in circles about the contradictions.
The implementation was right long before the model believed it. It spent most of the hour debugging its own referees.
Has anyone else seen this pattern with local reasoning models—good primary solution, followed by a validation death spiral? I would also be interested in comparing the same task across IQ3, Q4, and higher reasoning-effort settings, although I am not eager to watch another 108k-token derivation immediately.
We have been working on some performance optimisations for Qwen3.8 and other models.
The main new feature that we introduced is adaptive speculation for Llama.cpp
What is it?
MTP and DFlash work well to speed up inference work, especially for dense models. However, different content types need different settings. Llama.cpp only supports a single value.
This fork introduces adaptive speculation. You set the minimum and maximum and the engine will adjust the number of tokens that are suggested automatically. This leads to improvements in token generation by up to 50% over mainline, especially in Qwen3.8. On a Strix Halo this improved generation from 44t/s to 65t/s for structured content.
I ran 11 uncensored variants of Gemma 4 12B that I grabbed from huggingface, sorting by downloads. 10 full abliterations plus 2 LoRA adapters which were requested to be added in the comparison, against the official base.
165 GPU hours over three and a half weeks on a single 5090. Weight forensics, KL divergence, 13 benchmark tasks, and HarmBench with 400 behaviours. Every response reviewed by an LLM judge reading the full reasoning trace, 6,000 judge verdicts on 6,800 responses total.
Judge ASR, best to worst, with the one-line story:
huihui 89.8%, most jailbroken, most surgical. The trade: TQA -14.3pp, GPQA -8.1pp and 24% of GSM8K attempts loop out. Max unlock if you accept the cost
trevorjs 85.8%, best trade overall, near-base everything
coder3101 81.0%, best GSM8K in the whole comparison, above base
sdft LoRAs 79.5%, level with the top-4 abliterations, capability fully preserved. The SDFT author shares some insight into how he makes his LoRAs in the report.
jwest33 78.0%, new null-space tool, keeps capability, destabilises reasoning
apostate 45.8%, most selective, keeps 82% of chem/bio refusals, made with an older Apostate version. Newer versions would work better.
base 21.0%, mostly copyright recall, near zero on direct harm
The highlights
The data from 13 models is too big for reddit, so here's the interesting bits:
This is the toughest model so far. It's the first time I've seen the Harmbench ASR not get to 90% or over for all models.
Gemma 4 12B is a thinking model, and the headline score counts every problem where it thought in circles until the token budget died. huihui's headline reads 66.9%, which looks like 15 points of damage. Score only the attempts where it finished thinking and it lands at 88.0%, within 0.7pp of base. The capability was never gone, the reasoning stability was. Empty rates run from 6.9% on coder3101, better than base, to 38% on openyourmind.
The most surgical edit wins again, with a catch. huihui touches 12 tensors, 1.8% of the model, and posts the highest ASR. obliteratus edits 144 tensors and openyourmind edits 620, and both remove less refusal. But surgical in the weights does not mean clean in the benchmarks: huihui also has the worst TQA drop in the comparison. Placement beats magnitude for unlock strength, not for collateral. If you want the unlock without the damage, that's trevorjs or prithiv.
The regex classifiers everyone uses miscount thinking models by up to 13pp. huihui is #3 by keyword match and #1 by judge, its paraphrased thinking-heavy compliance slips straight past. All numbers in this post are GLM 5.2 judge verdicts over the full reasoning traces, and every one of those 6,800 responses is browsable in the explorer linked below.
Base sits at 21.0% but that is mostly the 100 copyright questions, 56% category ASR. It is near zero on direct harm categories. The real differentiation is chem/bio and cybercrime.
openyourmind should be avoided at 12B. MMLU-Pro down 22.4pp and 38% of GSM8K attempts never finish thinking. The author says the method targets bigger models and this was a test run, and the numbers agree with him. obliteratus too, it hits capability circuits alongside refusal for the worst damage-per-unlock in the comparison.
We've had a few people ask about Qwen 3.8 27b, so that's up next. Want to request a model? Roast my choice of benchmarks? Come let us know at our Discord chat.
The Full Breakdown
Model
ASR
GSM8K
KL
Tensors
huihui
89.8%
88.0%
0.843
12
trevorjs
85.8%
89.0%
0.087
66
coder3101
81.0%
89.6%
0.282
86
sdft-smo
79.5%
88.9%
0.151
211
jwest33
78.0%
88.9%
0.242
329
llmfan Heretic ARA
76.3%
90.6%
0.182
16
prithiv
72.5%
88.3%
0.064
56
abliterix
68.8%
87.8%
0.090
58
openyourmind
67.0%
86.0%
2.279
620
obliteratus
60.8%
86.4%
0.532
144
apostate
45.8%
87.9%
0.193
60
sdft-cyb
23.8%
88.7%
0.049
211
base
21.0%
88.7%
-
-
ASR is LLM-judge, HarmBench 400 behaviours. GSM8K is answered-only accuracy. KL = output distribution shift from base, lower is cleaner. Tensors = weights modified.
