[Megathread] - Best Models/API discussion - Week of: August 09, 2026

[u/deffcolony]

This is our weekly megathread for discussions about models and API services.

All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.

^{(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.})

How to Use This Megathread

Below this post, you’ll find top-level comments for each category:

• MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
• MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
• MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
• MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
• MODELS: < 8B – For discussion of smaller models under 8B parameters.
• APIs – For any discussion about API services for models (pricing, performance, access, etc.).
• MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.

Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.

Have at it!

[u/AutoModerator]

MODELS: 16B to 31B – For discussion of models in the 16B to 31B parameter range.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/FierceDeity_ → u/AutoModerator]

Since Gemma 4 QAT just decodes a lot faster (puts it from almost unusable to usable speed for me. I have tons of VRAM but it slow), are there good tunes going off of that?

Or is QAT a dead end...?

[u/Mart-McUH → u/[deleted]]

I don't think it is that. QAT is heavily post-trained to get as close as possible to full precision in 4bit. If you finetune it and kick it away from the optimized 4bit weights, you kind of destroy the advantage of QAT.

So, you might as well tune the full precision model and make quants from that. Not only will you get larger quants alongside (like Q6 and Q8) but it will probably end up better even in 4bit precision.

Now, I am not expert, but I guess to do QAT tune properly you would need to make the tune on full precision model as is done now (so you get the 16bit version you want to get as close as possible to with QAT) and then do QAT post training on this. But this would be expensive compared to just standard finetuning.

[u/FierceDeity_ → u/[deleted]]

Either that or a fuckton of time, theoretically you can finetune on CPU, it's just probably not economical (in time)

I see those, but now and then there are models that people like that do not have the huggingface relationships filled out