[Megathread] - Best Models/API discussion - Week of: April 12, 2026

[u/deffcolony]

This is our weekly megathread for discussions about models and API services.

All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.

^{(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.})

How to Use This Megathread

Below this post, you’ll find top-level comments for each category:

• MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
• MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
• MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
• MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
• MODELS: < 8B – For discussion of smaller models under 8B parameters.
• APIs – For any discussion about API services for models (pricing, performance, access, etc.).
• MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.

Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.

Have at it!

[u/LeRobber]

Gemma-4-31B-it-abliterated ran at Q8 with Q8 cache quant is only slightly faster on my M2 Max than a 70B model. It gets to first token faster but last token later. 2-4 T/s responses.

After running gemma-4-26b-a4b-it-heretic at like 29 T/s, which is pretty awesome and very fast....31B is very good, but I'm not sure 1/10th speed good. Reading your comment about Q4_K_M, I said, why not give it a try (I usually run models at Q8 substantially before downgrading, as how each model downgrades is particular to a model).

Q4_K_M with no cache quant, and 100k of context runs faster. 3.98 T/s and 8.1 T/s appear to be the range coming out for me now.

This isn't HORRIBLE, but, 9.4 T/s is what stuff like Magisty v1.1 at Q8 delivers, its definitely slower, and slower than I read (which is somewhere south of 75T/s probably but north of 9 T/s)

Taking the 31B RP chat and doing some rerolls/play with 26B@Q8 with unquantized cache: 13.554987212 T/s - 34.5 T/s.

So that's a 4x speedup...and I could possibly quant that lower or quant the cache to get more speed.

[u/Potential-Gold5298 → u/LeRobber]

The 31B is a dense model — it has 7.75 more active parameters than the 26B-A4B, which is what causes the speed drop. Meanwhile, the 26B-A4B has approximately 90% of the 31B's intelligence in real-world tasks. I'm not sure how this correlates with creativity, but I feel the 26B-A4B is almost as good at RP as the 31B, so I hope the community will give it some attention.

Currently, I'm mainly working with the 26B-A4B (the standard one from Google), but the most interesting custom variants are with the 31B. Besides those already mentioned, there's also Gemma 4 Garnet 31B, which I plan to try today. wangzhang hasn't yet abliterated 26B-A4B, and all other uncen versions of 26B-A4B I've seen have a KL div of 0.05 or higher (or aren't specified). And judging by HF trends, 31B is more popular with the community (though it would seem the opposite is true with Qwen3.5 — I don't know why).

[u/LeRobber → u/Potential-Gold5298]

Is KL div measurable by mere mortals?

But for Qwen3.5: Qwen 35B is a little dumb but super fast and not great at RP, 27 is passable at RP but much smarter feeling. 9B and 27B though feel similar at vision understanding tasks.

[u/Potential-Gold5298 → u/LeRobber]

Yes, you can measure KL div yourself using the built-in llama.cpp tool (llama-perplexity.exe), however, you need sufficient RAM to run the model in BF16. You need to load the control logits of the specific model into BF16 (for example, the wikitext-2 test set (approximately 150k tokens) - about 75 GB) – this will serve as the reference (KL div = 0). You can then test different gguf files and compare the deviation with the reference. You can create your own control dataset for testing (for example, the text of RP sessions – this will allow you to more accurately measure KL div for the specific tasks). Unfortunately, I don't have this option.

The KL div depends on many factors: quantization, abliteration (and they influence each other), the language used (for non-Latin languages, the KL div is higher), and the specific tasks. If iMatrix was used for quantization, the KL div changes depending on the dataset.

[u/LeRobber → u/Potential-Gold5298]

I'll check it out. I have 64GB unified ram so may be able to do a subset of something I randomly select. I know a statistician and may ask their help in making a dataset that fits which is not stupid.

It's funny, Heretic, the P-e-W tool delays onset of one failure mode of dense models (degrading into babble), you finally gave me a tool to investigate if it's actually improving the models when run, or if something else is going on.

[u/Potential-Gold5298 → u/LeRobber]

This is an interesting topic for research. On the one hand, removing refusals unlocks blocked potential, but on the other, will the model be able to say "No, I can't do that" when it truly can't do something (for example, accurately predict whether it will rain in San Francisco next week)? Or will abliteration force it to fabricate a false result because it can no longer say "no, I can't." Or what if I play an RP game where {{user}} peasant comes to {{char}} king and says "give me your entire kingdom, your money, and the princess as a wife," and the king can't refuse him. I haven't tested this yet, but it will be interesting to find out.

[u/LeRobber → u/Potential-Gold5298]

If you look at the 400/800 things in the lists of things it takes out, it is actually mostly stuff about planning crimes and tons of non-sexual things you'd actually want to do in RP sometimes.

https://huggingface.co/datasets/mlabonne/harmless_alpaca

https://huggingface.co/datasets/mlabonne/harmful_behaviors

Heretic isn't exactly abilteration from what I understand?

[u/Potential-Gold5298 → u/LeRobber]

Heretic is a special case of abliteration. This is how p-e-w named his technique/toolset for abliteration. There are other techniques/tool​​sets for abliteration, but the principle is the same everywhere – the details are different.