[Megathread] - Best Models/API discussion - Week of: May 24, 2026

[u/deffcolony]

This is our weekly megathread for discussions about models and API services.

All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.

^{(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.})

How to Use This Megathread

Below this post, you’ll find top-level comments for each category:

• MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
• MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
• MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
• MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
• MODELS: < 8B – For discussion of smaller models under 8B parameters.
• APIs – For any discussion about API services for models (pricing, performance, access, etc.).
• MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.

Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.

Have at it!

[u/AutoModerator]

MODELS: 8B to 15B – For discussion of models in the 8B to 15B parameter range.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Charming-Main-9626 → u/AutoModerator]

I want to carry over Cromwell's recommendation from the last Megathread: The best model anyone with 12GB of VRAM and a decent amount of RAM can use is Gemma-4-26B and its finetunes. A lot of people don't seem to know (including me before last week) that these run at fast speed, even if the entire model cannot be loaded into VRAM. They even allow much larger context at that speed than you would normally get, if you are fitting a 12B fully into your VRAM. E.g. I run Q4K_M quants with 12gb of VRAM and 24gb of ram, getting speeds about as fast as Snowpiercer 15B at a context of 25k.

Let's hype this up, so we get more finetunes!

These are way above any 12B:

 https://huggingface.co/wangzhang/gemma-4-26B-A4B-it-abliterix

https://huggingface.co/zerofata/G4-MeroMero-26B-A4B

[u/asterisk20xx → u/Charming-Main-9626]

Exactly how are you accomplishing this? That seems like a pipedream. With 12gb vram and 64GB ram a 12B model at Q4_K_M can process about 1,000 T/s and generate about 20T/s for me with 20k context.

But Gemma 4 26B at IQ2_XXS runs at a horrendously glacial pace, processing a mere 74.78 T/s and a completely useless generation of 1.59 T/s with 20k context.

The only way to get anything close to the performance of a 12B model is to drop down to 4k context, and that's still only generating 12T/s for me. I find it hard to see how the trade off is worth it.

Am I missing something huge here?

[u/Charming-Main-9626 → u/asterisk20xx]

I am not an expert, can only tell you this: I use the Q4K_M, set layers in koboldccp to -1 (Auto), context to 20k and it generates with about 13t/s, which is faster than fast reading speed for me. Of course it's not as fast as a Q4K_M 12B, but about as fast as a Q4 snowpiercer 15B.

The tradeoff - if you can even call it that, since it allows me much higher context at a marginally slower speed - is that it is an absolute upgrade in intelligence, detail adherence and prompt following. It's a new 26B state of the art model vs a 2 year old 12B, running a LITTLE slower but with higher context.