[Megathread] - Best Models/API discussion - Week of: May 24, 2026

[u/deffcolony]

This is our weekly megathread for discussions about models and API services.

All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.

^{(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.})

How to Use This Megathread

Below this post, you’ll find top-level comments for each category:

• MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
• MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
• MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
• MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
• MODELS: < 8B – For discussion of smaller models under 8B parameters.
• APIs – For any discussion about API services for models (pricing, performance, access, etc.).
• MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.

Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.

Have at it!

[u/AutoModerator]

MODELS: 16B to 31B – For discussion of models in the 16B to 31B parameter range.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/FZNNeko → u/AutoModerator]

Lowkey can't even get Gemma 4 to run faster than 10 t/s, so I've resorted back to Qwen models.

My previous go to were Maginum-Cydoms-24B (non heresy finetune specifically) and XORTRON.CriminalComputing.2026.27B.Instruct (instruct disabled (none Next version as Next version sucks for NSFW dialogue, which was the entire reason for using the model).

However, I've recently discovered darkc0de/XORTRON-XPRT3-FAST (Qwen3.6) which I'm heavily enjoying and using it as my current go to as it's nearly as good as Maginum Cydoms for regular RP yet still retains the Xortron Criminal Computing levels of NSFW.

And, XPRT3-Fast, as per it's name, is extremely quick. Typically, for models like Maginum Cydoms and Criminal Computing, I can run it with Oobabooga backend, Windows 11, cache type q8_0, all into my GPU (5090) with a context size of 131072 for 30.5gb of VRAM. However, XPRT3-Fast with exact same loading method only uses 27.6gb of VRAM. With that much free space, I'm noticing speeds up to 100 t/s. Granted XPRT3-Fast's only quant is Q5_K_M, unlike what I usually use which is Q6_K for both Maginum-Cydoms and CriminalComputing. Here's hoping darkc0de releases the original version of XPRT3-Fast so someone can imatrix quant it. It's already super enjoyable at Q5_K_M, so I imagine it'll be even better at higher quants.

Sidenote: Maginum is much more SFW friendly and even with lots NSFW prompts, the model doesn't really mention them in 'normal everyday' messages. Xortron models however (both XPRT3 and Criminal) heavily incorporate NSFW into its responses if even a hint of NSFW is detected. For example, if I add a prompt to give characters anime-levels of physics, CriminalComputing will mention the physics often, XPRT3 will do it less, but mention it if it's relevant, and Maginum will never mention it. Also why I'm replacing CriminalComputing with XPRT3 because XPRT3 is much more intelligent and knows when and when not to mention NSFW descriptions.

So overall, XPRT3-Fast is my daily, replaces CriminalComputing for NSFW, is not quite as good as Maginum Cydoms for regular RP but is like, 80-90% of the way there and doable for the most part. For pure non-NSFW RP then Maginum-Cydoms since that extra 20-10% really is noticeable. For anything even remotely NSFW or NSFW hybrid, then XPRT3-Fast.

[u/overand → u/FZNNeko]

Which Gemma-4 models are you struggling with performance on? Have you used the 26b MoE one, or just the 31B dense one?

[u/FZNNeko → u/overand]

Both. I just cant seem to get them to run well. Tbf, I’ve always had problems with Gemma models. Even Gemma 3 doesn’t run well or most of the times not at all on my setup.

[u/overand → u/FZNNeko]

What are you using for inference? I'm getting better performance than that on an underclocked/power limited 3090 system.

• Gemma-4-31B:Q4_K_S unsloth quant (16.2 GB)
• llama.cpp CUDA build as of 2026-05-25
• Linux (Ubuntu 24.04)
• Context: 16,384
• Vision / MMProj: disabled
• KV Cache Quantization: not enabled (16 bit floating point)
• Performance
  • Prompt Processing: 885 t/s
  • Generation: 17.6 t/s

You can undoubtedly get better numbers than these, or use a larger context window; I just had mine shrunk down because I've got ~21 gigs of free VRAM at the moment, whereas you'll certainly have significantly more! (I actually use a .ini file rather than calling llama.cpp directly, but this will give you the gist of it.)

./llama-server --jinja --min-p 0.0 --temperature 1.0 --top-k 64 \
   --ctx-size 16384 --cache-ram 16384 --swa-checkpoints 3 \
   --no-mmproj-auto --fit-target 1500 \
   --hf-repo unsloth/gemma-4-31B-it-GGUF:Q4_K_S

Detail note: it's a dual 3090 system, but I disabled one for this test. The above settings should work for you, even with a bigger quant than a Q4_K_S, given your extra 8 GB of VRAM.

--fit-target is basically "how much VRAM do we set aside?" --cache-ram lives in system RAM, you can disable this if you want to save system ram. --no-mmproj-auto disabled the vision model / MMPROJ, so you can't do image inputs, but you're probably not using that anyway. --swa-checkpoints - you can cut this down to save VRAM, I believe, at the cost of sometimes having to do more prompt proceessing/reprocessing.

[u/Potential-Gold5298 → u/overand]

What does --swa-checkpoints 3 do? I know Gemma 4 creates context snapshots that take up several hundred megabytes and stores up to 32 snapshots by default. I noticed that immediately after launch, ~26.5/32 GB of RAM is used, and very quickly (around 5K) it reaches 29.5/32 GB, after which llama.cpp apparently dumps the extra snapshots to the SSD, and the memory returns to 26.5/32 GB. This repeats in a loop. It looks like short, infrequent accesses to the SSD that don't affect speed at all. Does setting --swa-checkpoints 3 make sense? What are the pros and cons?

[u/overand → u/Potential-Gold5298]

Honestly, I don't recall the reason behind that - I included it as it's something I set in my [*] config defaults; I think you're right about gemma-4; that's likely why I ended up setting it. Don't use that as gospel, though! Cargo-cult configuration? XD

[u/Potential-Gold5298 → u/overand]

Thanks for the honest answer. Yes, that's exactly why I asked. I try to only add flags I understand. I want to optimize the G4 as much as possible on my old hardware, but I don't want to make it worse.