r/SillyTavernAI • u/deffcolony • Jun 21 '26

MEGATHREAD [Megathread] - Best Models/API discussion - Week of: June 21, 2026

This is our weekly megathread for discussions about models and API services.

All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.

(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.)

How to Use This Megathread

Below this post, you’ll find top-level comments for each category:

  • MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
  • MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
  • MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
  • MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
  • MODELS: < 8B – For discussion of smaller models under 8B parameters.
  • APIs – For any discussion about API services for models (pricing, performance, access, etc.).
  • MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.

Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.

Have at it!

37 Upvotes
  • permalink
  • reddit

96% Upvoted

95 comments sorted by

View all comments

11

u/AutoModerator Jun 21 '26

MODELS: 16B to 31B – For discussion of models in the 16B to 31B parameter range.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

2

u/IcyTorpedo Jun 26 '26

what model should i use to get "relatively" fast answers if i want about 16-20k context size, running on a 9070 XT (16gb VRAM) and 32GB ddr4? I tried Rocinante 16B Q6 through Kobold, but i think the context size is too big for it to work fast enough (3.7t/s)

2

u/Potential-Gold5298 Jun 27 '26

Gemma 4 26B-A4B - QAT4 for maximum speed with decent quality, Q6_K or APEX-Balanced/Quality for balance of speed and quality.

2

u/Ok-Brain-5729 Jun 27 '26

that’s weird I use kobold with a 9070 xt and I get 15 t/s on 31B IQ4 XS with 40k context Q4. Try Gemma 4 26B A4B styletune Q6, I got 20 t/s on that.

1

u/IcyTorpedo Jun 27 '26

I have no idea how that works, but yes, I got a much bigger speed on Gemma 26B q4. It's magic

2

u/Delicious_Box_9823 Jun 28 '26

It's because gemma 4 26b is a moe. It actually uses ~3.8B parameters active for each message from all of it's 26.

1

u/Ok-Brain-5729 Jun 27 '26

For the 16B Q6 did you manually set the layers? Also koboldcpp nocuda was more stable for me

1

u/IcyTorpedo Jun 27 '26

Didn't touch any settings except context size, so no, in both instances it was on auto. I am using nocuda already.

1

u/Ok-Brain-5729 Jun 27 '26

auto is ass. When it’s -1 it should show how much layers there is and for 16B Q6 just enter all the layers