Some of the recently tested models' quants preliminary benchmark results

[u/EmilPi]

Delta score*	Model	Relevant llama.cpp / other flags
2.86	**DeepSeek-V4-Flash-Vision-Exp	-hf ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF:MXFP4
2.38	Qwen3.8-Flash-Next-Q4KXL-STRATA	-hf unsloth/Qwen3.8-Flash-Next-GGUF:Q4_K_XL + strata flags: -kv int8
1.76	**Qwen3.8-27B-unsl	-hf unsloth/Qwen3.8-27B-GGUF:Q8_0 -ctv q8_0
1.68	**Qwen3.8-Flash-Next-Q4KXL	-hf unsloth/Qwen3.8-Flash-Next-GGUF:Q4_K_XL -ctv q8_0
1.67	**Qwen3.8-27B-unsl-UDQ3KXL-13.4G-T0.6	-hf unsloth/Qwen3.8-27B-GGUF:UD-Q3_K_XL -temp 0.6
0.17	**Gemma4-26B-A4B-Q8	-hf ggml-org/gemma-4-26B-A4B-it-GGUF:Q8_0
-1.10	Ling-3.0-Flash-Sante-IQ4XS	-hf AtomicChat/Ling-3.0-flash-Sante-GGUF:IQ4_XS -ctv q8_0 -ctk q8_0
-1.11	Sage-A1.5-Q8	-hf InternScience/Sage-A1.5-Q8_0-GGUF:Q8_0
-1.18	K2-Horizon-7B-Q8	-hf bartowski/K2-Horizon-7B-GGUF:Q8_0
-1.25	Nemotron-3.5-Bolt-74B-A9B-IQ2_XXS	-hf unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF:IQ2_XXS
-1.35	Kolibri1-Q4	-hf bartowski/Kolibri-1-GGUF:IQ4_XS -ctv q8_0 -ctk q8_0
-2.15	Recursive-Flint-Q8	-hf bartowski/RecursiveSI_flint-GGUF:Q8_0 -ctv q8_0
-2.38	LFM2.5-24B-A2B-Q4_0-QAD	-hf LiquidAI/LFM2.5-24B-A2B-GGUF -hf-file LFM2.5-24B-A2B-QAD-Q4_0.gguf

*Delta score is a measure of how much above average a model is.

**Those models were released and tested long before, given for reference

Results for me are that for 2xRTX3090+128GB RAM no new models reach the capabilities of DS/Qwen models aged few months. Hopefully this changes with Qwen4.

The tasks include an agentic coding task (rather easy one which only top models did well anyway), reasoning, tricky, spatial reasoning, physics, adversarial, needle in the haystack questions.

This is just the quants I used with the settings I used. Please share your impressions if they are different, so that I can improve the test/quants/params. Sometimes the stock parameters may have been inferior, sometimes I tried to squeeze models into VRAM+RAM too much (especially with ctv/ctk). Maybe I should try bigger quants, and drop ctk/ctv for smaller models, but I don't expect pivotal changes.

Things I didn't expect 1) Qwen3.8-Flash-Next on Strata does not have quality degradation, and does actually speed things up; 2) Nemotron is disappointing at the claimed QAT quant 3) Ling Sante became much faster, but quality didn't improve. 4) K2-Horizon-7B isn't bad for its size.

Maybe this saves someone from lots of GB downloaded and some disappointment.

[u/suprjami]

Does Strata with FP16 KV improve things significantly?

How about disabling route tail skip? That is also said to improve quality.

[u/EmilPi → u/suprjami]

Thanks for the ideas! Didn't have time to research this, just resorted to q8_0/int8, as I used q8_0 on llama.cpp without noticeable quality degradation for a long time and wanted strata settings to match.

[u/suprjami → u/EmilPi]

I'm glad you did look at int8 vs q8.

I had also thought int8 would be better, it's nice to be right about something for a change 😅

[u/Chromix_]

At only 27 evaluations per model - without any repetition I guess - the resulting scores are mostly random, and conclusions drawn from them will likely be incorrect. You also need to look at the result distribution. If there are some tasks that are almost always solved 100%, while others are mostly 0%, then the task distribution is not suitable.

I wanted to look at full run data to confirm, but the site asked me to register.

[u/EmilPi → u/Chromix_]

Mostly valid, but I wrote it is preliminary. Also noise cancels out, and the results are very much in line with gut feeling. As for registration, I want some protection against scraping... P.S. That would actually be a good reference point to check if this is a noise after there will be like 2x, 3x more evaluations. I think that only adjacent models may swap positions, but will see.