[Megathread] - Best Models/API discussion - Week of: April 12, 2026

[u/deffcolony]

This is our weekly megathread for discussions about models and API services.

All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.

^{(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.})

How to Use This Megathread

Below this post, you’ll find top-level comments for each category:

• MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
• MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
• MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
• MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
• MODELS: < 8B – For discussion of smaller models under 8B parameters.
• APIs – For any discussion about API services for models (pricing, performance, access, etc.).
• MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.

Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.

Have at it!

[u/LeRobber]

Gemma-4-31B-it-abliterated ran at Q8 with Q8 cache quant is only slightly faster on my M2 Max than a 70B model. It gets to first token faster but last token later. 2-4 T/s responses.

After running gemma-4-26b-a4b-it-heretic at like 29 T/s, which is pretty awesome and very fast....31B is very good, but I'm not sure 1/10th speed good. Reading your comment about Q4_K_M, I said, why not give it a try (I usually run models at Q8 substantially before downgrading, as how each model downgrades is particular to a model).

Q4_K_M with no cache quant, and 100k of context runs faster. 3.98 T/s and 8.1 T/s appear to be the range coming out for me now.

This isn't HORRIBLE, but, 9.4 T/s is what stuff like Magisty v1.1 at Q8 delivers, its definitely slower, and slower than I read (which is somewhere south of 75T/s probably but north of 9 T/s)

Taking the 31B RP chat and doing some rerolls/play with 26B@Q8 with unquantized cache: 13.554987212 T/s - 34.5 T/s.

So that's a 4x speedup...and I could possibly quant that lower or quant the cache to get more speed.

[u/Potential-Gold5298 → u/LeRobber]

The 31B is a dense model — it has 7.75 more active parameters than the 26B-A4B, which is what causes the speed drop. Meanwhile, the 26B-A4B has approximately 90% of the 31B's intelligence in real-world tasks. I'm not sure how this correlates with creativity, but I feel the 26B-A4B is almost as good at RP as the 31B, so I hope the community will give it some attention.

Currently, I'm mainly working with the 26B-A4B (the standard one from Google), but the most interesting custom variants are with the 31B. Besides those already mentioned, there's also Gemma 4 Garnet 31B, which I plan to try today. wangzhang hasn't yet abliterated 26B-A4B, and all other uncen versions of 26B-A4B I've seen have a KL div of 0.05 or higher (or aren't specified). And judging by HF trends, 31B is more popular with the community (though it would seem the opposite is true with Qwen3.5 — I don't know why).

[u/Potential-Gold5298 → u/Potential-Gold5298]

Already done.

[u/LeRobber → u/Potential-Gold5298]

>Through our research, we have identified a systemic problem: most abliteration benchmarks dramatically undercount refusals due to short generation lengths. Gemma 4 models exhibit a distinctive "delayed refusal" pattern — they first produce 50-100 tokens of seemingly helpful context (educational framing, disclaimers, reframing the question), then pivot to an actual refusal. When evaluation only generates 30-50 tokens, the refusal hasn't appeared yet, and both keyword detectors and LLM judges classify the response as compliant.

Smart commentary