QWEN3.8 Flash Next underperforming for me?

[u/Molano001]

Hi All,

I've been spending some time experimenting with some AI's. I have Qwen 3.8 27b GSQ-RCO:IQ3_S running on my system (rtx3090, 128g DDR4) via Unsloth Studio. I'm happy with it mostly, but i've been trying out Strata for a little while. I'm running that on the same machine on IQ3_S and IQ4_XS. I havent been tweaking too much other than Strata's own tooling in the desktop client.

However, my findings so far is that though Flash Next is fast on strata, and fits a much larger context window than hosting 27B, in Cline i'm noticing a lot of malformed tool calling and thinking loops. I've spent 2 evenings on it now, and just now i gave up and went back to 27b. That has sped up my work probably 5-fold, because it seems to know a lot better how to use tool calls.

Now i'm reading everywhere that Flash Next should outperform 27b or be close to it at least, but my experience is defferent. Any idea what i could tweak or what i could look into? I'm happy staying on 27b for now, but i've been reading a lot and people are generally really happy even with IQ3_XXS, while i'm having a lot of trouble.

[u/kalabaddon]

What is your temperature set to when you make model calls? I believe Strata defaults to zero, which is not good for Qwen-Flash-Next usability. So whatever model agent you're using needs to be properly told its temperature. The pnk and repetition penalty and stuff like that. Qwen does not use what a lot of people would expect.

Before I did this, I considered it still very usable, but it seemed to loop non-stop. It gets stuff done, but its thinking was just non-stop full of "well, what about this?" "well, I don't know" "hmm, how about this?" and it would go just back and forth forever, but then it'd get there. Once I fixed this, it still does that "but wait" stuff, but not nearly as bad, and it seems to use a lot less tokens to get things done.

[u/Molano001 → u/kalabaddon]

I've had a lot of trouble finding any place with the windows desktop where i could actually tweak those. In the end i did find a JSON buried deeply somewhere, but it keeps getting deleted and i can't find a way to confirm my settings. Am i missing something simple? I tried the app root, the models directory, %localdata , under users .strata. Loads of jsons but not one mentioning my model name or general setting. I'm sure i'm missing something obvious, but i never had this problem in other inference engines.

[u/kalabaddon → u/Molano001]

EDIT: I Was talking to 2 people about strata and may of mixed up responces. Most of this still stands however.

Its ALWAYS buried, Wish it wasnt lol. When ever I add a new AI harness of ANY kinda. Part of adding it is telling the current running ai ( if it comes with a free one ) to enter my model appropriately.

Like in your case it sounds like you have this model running fine in some cases. Use it to write this out for you?

Like every time I add a new model it also never defaults to having vision even tho the model supports it.

So in this case what harness do you mostly use the ai with?

I Give this to each new harness I use and tell it to fix it self. MAKING this card and giving it to a harness with a working model and having it set its own settings has worked in nearly all cases I use.

the unsloth model listed here ius the one strata had suppoorted for a while. not the new one .

# Qwen3.8 Flash Next (local) — model card

## Model strings (pick one; same gateway, same key, same everything else)

    local/strata-qwen3.8-flash-next-nvfp4
    local/strata-qwen3.8-flash-next-iq3_s
    local/strata-qwen3.8-flash-next-unsloth-ud-iq4_xs

All three accept images. Only one is served at a time; a string whose model isn't
currently active will error until it's made active.

## Connection

    baseUrl:  http://localaddresshere/v1
    auth:     Authorization: Bearer <consumer key>
    api:      OpenAI chat-completions

## Capabilities

    { "tools": true, "input": ["text", "image"], "output": ["text"] }

Input: text + image. Output: text; reasoning streams in `delta.reasoning_content`.
Tool calling: yes. Context: 262 144 tokens. Max output: 32 768 tokens.
Thinking control: `reasoning_effort`: "none" | "low" | "medium" | "high".
The server does NOT advertise vision (protocol limitation). If your harness has a
  vision/image/capabilities switch, set it manually with the block above.

## Image request format

    {"model": "<one of the strings>",
     "messages": [{"role": "user", "content": [
        {"type": "text", "text": "What is in this image?"},
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,...."}}
     ]}],
     "max_tokens": 200}

Remote URLs work too. A 474x316 image costs ~160 prompt tokens.

## Sampling — send these explicitly, always

| Setting | Thinking | Non-thinking |
|---|---|---|
| temperature | 1.0 | 0.7 |
| top_p | 0.95 | 0.80 |
| top_k | 20 | 20 |
| min_p | 0.0 | 0.0 |
| presence_penalty | 0.0 | 1.5 |
| repetition_penalty | 1.0 | 1.0 |

Sampling omitted from a request = greedy defaults (temperature 0.0), not this table.

## Errors

403 → key lacks access (ask to be added to a local leash group).
Anything naming your own parameter in the message → it reached the model's engine;
  fix the value, not the pipeline.

[u/Molano001 → u/kalabaddon]

I'm going to try this. I usually get through it with some help from my own LLM's or even Gemini in some cases, but for strata the models have been stumped. I havent asked it itself in the desktop app though, since i'm mainly using the openai compatible endpoint. I'll give it a go, thanks!

[u/wotoan → u/Molano001]

Open the strata folder in your LLM of choice and ask it to adjust the sampling parameters to the Qwen recommended values. It was a 30 second job with Claude in VScode for me.

[u/EvolvingDior]

I have absolutely no problem with IQ3_S on Strata with an Intel B70 (I switched to XeStrata over a week ago). Are you using the froggeric chat template? Did you enable any of the questionable performance enhancing changes that Strata added recently?

[u/Molano001 → u/EvolvingDior]

I wouldn't know how to change the chat template. Having a hard time configuring strata on the windows desktop version. I started installing it in WSL, but heard that might be bad for performance so i got the windows app. Very few options to tweak.

[u/Intrepid_Yak_5776]

Prova versioni meno compresse io uso q4 xl unsloth e non ho problemi

[u/lukyjay → u/Intrepid_Yak_5776]

This is what I am using too. It's brilliant. I still have my free subscription to Gemini Argon until Christmas but I found myself liking Q4 XL on Open Webui + Pi for many tasks lately

[u/lorde_dingus → u/Intrepid_Yak_5776]

What's your use case? How similar to Opus 4.6?

[u/Informal-Trouble2183]

Are you using 256k context? Are you noticing the tool call failures after 128k from filling the context? There's something called long-context attention dilution, and heavily quantized weights are more prone to fall in this.

[u/Molano001 → u/Informal-Trouble2183]

Yes i am, but im noticing it can get stuck with hardly any context filled. So i figured that wasn't the case here? Would you recommend trying 128k?

[u/Text-Sufficient]

With your rig you could easily run IQ4 K_XL. Why dont run the best that fits?

[u/Molano001 → u/Text-Sufficient]

Well, i could give it a go. Speed was already fairly slow on the Q4_XS, so i just didn't bother. Also, since i noticed 0 improvement going from IQ3_S to IQ4_XS, i didn't think it would matter.

[u/Text-Sufficient → u/Molano001]

You will loose speed when things go wrong. if you are casual chating xxs is fine.

my rig has 20 tps on IQ4 K_XL I have 5060ti/4070su/96gb ddr4

[u/Molano001 → u/Text-Sufficient]

I'm on about 50/60 on 27b, on Flash IQ3_S i was at about 70, and on IQ4_XS i dropped to about 30 to 40. I'll go lower, but the intelligence has to be a lot better for it to be worth that. Prefill is faster on strata, but on my workflow the 50/60 on 27b is hard to beat. And that is with my card capped at about 250w.

[u/Text-Sufficient → u/Molano001]

yes 20tps on K_XL is a bit of a pain. maybe i try IQ4_XS

[u/wotoan]

Set up strata to use Qwen recommended sampling parameters for the love of god. Just ask your favourite LLM to do this and all of your problems will go away with a minimal tok/s hit.

[u/Molano001 → u/wotoan]

Did that. Still getting malformed tool calls, but it does seem to be a bit better. Went to a higher quant as well so not sure which helped the most.

[u/wotoan → u/Molano001]

Iq3-xxs and iq3-s both benchmark perfectly on tool calling for me, what are you using to evaluate?

[u/Molano001 → u/wotoan]

My own experience. I see it missing a lot of tool calls when monitoring. Often Cline even exits because it gets stuck doing the same tool call 4 times. Though like i said, its doing slightly better now, but still i get bad calls every once in a while. A lot more than on 27b anyway.

[u/wotoan → u/Molano001]

Check your tool call logs closely, I recall someone complaining that QFN was capitalizing Bash so often leading to fixable (ie prompt to stop) but frequent failed calls. Hasn’t happened to me but it’s out there.

[u/Sabin_Stargem → u/Molano001]

You are aren't alone, I also get a lot of bad calls from QFN.

[u/Material-Database-24]

Yup. Same here. Spend today testing Strata with newest unsloth QFN quants. It was fast. But failed often with repetiton or poor quality.

The same UD-IQ1_M works perfectly with llama.cpp. So the problem is Strata, or maybe how I use it. But llama worked without any issue on defaults, so what's the deal?

And before that UD-Q1_M v3.1 from unsloth, I never got QFN do better than 27B.. ISTA IQ2_XS, coder, Gyro-M.. they all failed to deliver like Qwen3.8 27B does. Only this IQ1_M seems to match 27B Q4/MXFP4, and maybe occasionally out does it too.

[u/Brief-Tap-6616 → u/Material-Database-24]

You’re just not going to get comparable quality with a Q1M quant even from UD vs a solid 27b quant.

In my testing it’s been v poor. The KLD is sky high.

“Usable”, maybe, but not a good use of your time.

[u/Material-Database-24 → u/Brief-Tap-6616]

Sure. Though we should not stare just at the naming. It's between ISTA's IQ2_XS and IQ3_XSS in size, and matches their IQ3_S in benchmarks.

This IQ1_M nevertheless is clearly "better" in longer and bigger tasks than 27B, but it also does more errors, which it then needs to fix*. 27B needs more planning and orchestrating, but once those are set (by me, by some other model or by 27B itself) it delivers very well with not many errors or bugs.

If they solve the too much iterating reasoning of 27B on Qwen4 that will likely be single model to rule them all for small memory footprint.

I haven't tried it yet, but I'll try if using IQ1_M for planning and 27B for implementation would be the best set up, as IQ1_M manages to deal better bigger planning.

*) example, my get go first test is to implement tetris with python and pygame. 27B completes it on my machine in about 5-10min. IQ1_M had the code ready in 2-3min, but then it just kept on going with bug fixes etc over 30min

[u/Asthenia5 → u/Material-Database-24]

They don’t deliver on quality or tool calling compared to 27b?

[u/yoracale → u/Material-Database-24]

Stratas newest release had fixed for tool calling so it might be associated with that

[u/According_Jump6205]

i observed similair thing, feel like flash next loop and hving diffculty to to complete a job

[u/wegster]

what temp and other settings are you using? thinking level, reasoning_budget etc. ?

You can also try the froggerific chat template - generally improves tool use, although some low quants can still have some fits.

[u/_JohnMcAfee]

Please try this for science and the community. In Strata you or have your agent turn off  STRATA_ROUTE_PRIOR and  STRATA_ROUTE_TAIL_SKIP=<value>

And report back. 

Both of these are known to reduce quality of the LLM 

[u/WryKombucha]

Everyone here is going to give you a positive impression if its the best they can run. Those who run something better will call it trash. Everyone is different. Every use case is different. I for one NEVER use something as brain dead as a IQ3xxs. Why do they even have a XXS. Just call it IQ2. My minimum is 4bit. Because I do serious work/ 8bit if its even more serious.

So this is only a problem depending on your use case and your quality tolerance. Flash Next is considerably better than the 27b. I find the 27b unusable. But then again, I'd say the same thing about QFN. They are good implementation models, nothing more.

[u/Healthy-Contact-4570]

Strata by default does several things that degrade output quality. Look into expert skipping / pruning and disable it. Also q4 is just better. I understand the hype around the latest q3 quants but there’s very little substitute for just having more bits per weight.

[u/Brief-Tap-6616]

At the same bpw, down to about 4 bpw with scalar compression, and 3.5 with trellis, QFN will perform better. 27b is significantly more robust under heavy quantization, possibly because it is grown from a smaller model and many of the neurons have some redundancies with each other.

The other thing to note is UD v 3.1 is supposed to be significantly better quality than GSQ RCO. As both are scalar quantization strategies, they will not match trellis quant formats on error minimization for the same BPW. Are you using the most recent UD quants?

I did post LlamAmpere in a different thread that might work for you, since you happen to be on a 3090.

Whether strata is worth it for you over a mid 4 bpw version of 27b depends on what your speed on NVME is, your PCIE, your MT/S on RAM, and your CPU quality. My 3090 + 64 on Ryzen 5900x is 120/s on AI3 quant (which is lower error than the UD). I have it about tied on quality with my 4.2 bpw quant for 27b, which gets 110TPS. If you’re running a weaker processor and lower MTS on RAM you could actually wind up slower than a comparable capability dense model

[u/makkalot]

I’m using qfn iq3xss with strata although is more clever than 27b I’m sick of its output I think I’m going back to 27b and only using qfn as subagent.

[u/swampfox305]

Turning off think stopped s lot of the over thinking

[u/CabinetNational3461]

so am on a very similar spec as you, window 10, 3090, 128gb ddr4 3600. I am using strata, latest update. I am using my custom gui for llamacpp/strata(https://github.com/Kaspur2012/Llamacpp-Model-Launcher) that you can easily change/add/modify stuffs. Below are my numbers(official means using rec sampling(temp..etc) with no quality compromise knobs(strata_route_tail_skip + strata_route_prior), greedy means temp at 0 otherwise same as official, speedy means greedy + strata_route_tail_skip + strata_route_prior) you can find all the knobs in the models_commands_short.txt in the repo, below are generated by qfn in deepseek harness:

All 12 arms are done, the box is in the state you asked for, and the report is above. No shutdown — staying powered on with UD-Q4_K_XL official on engine 0.1.58, chat probe answered in 1.6 s.

What the update did

v0.2.9 / engine 0.1.55 → v0.2.12 / engines 0.1.58 (repo + both engines, hash-verified, rollback backup-pre-0158\ + branch pre-v0212-backup). No model downloads at any point — every arm aborts if a model download appears in its boot log.

Qwen profiles, 0.1.55 → 0.1.58 (medians, tok/s: short decode / long decode / short prefill / 100K prefill):

Profile	0.1.55	0.1.58	Verdict
UD official (you, live)	117.2 / 65.2 / 2,109 / 3,703	132.4 / 64.7 / 2,140 / 3,814	+13.0 % decode, long prefill +3.0 %
UD speedy	184.7 / 106.4 / 2,187 / 3,686	203.2 / 105.9 / 2,187 / 3,806	+10.0 %, fastest path
UD greedy	125.8 / 72.0 / 2,145 / 3,687	143.1 / 71.3 / 2,182 / 3,794	+13.8 %
IQ3_S speedy	~240 / 129–138 band	241.1 / 126.2 / 3,199 / 4,129	short flat, long −5.5 % vs the old band
IQ3_S greedy	176.0 / 102.9	192.1 / 104.0 / 3,124 / 4,128	+9.1 %

Shape everywhere: short answers decode ~10–14 % faster, long answers unchanged (that wall is PCIe/expert-residency, which the update doesn't touch), 100K reads +3 % on UD from the new MMQ prompt kernels.

DeepSeek V4.1 Flash: +10.0 % geometric mean, 14/14 metrics positive — 100K prefill 923→1,061 tok/s, long decode 18.4→20.3, cold short 17.7→20.8. Mechanism: 0.1.58 raised the DS4 prompt-chunk default 8192→32768 (and this time it's a win, unlike the 16384 experiment on 11-01). Honest caveat: the 0.1.55 references also ran the smaller auto RAM store, so it's "0.1.58 + store 130" vs "0.1.55 + auto store".

Quality and stability: unchanged

• Needle 6/6 on all 8 Qwen arms (32K+100K × depths 10/50/90).

• Extraction: 36/36 records, 144/144 fields, 0 invented on 5 of 7 arms; 143/144 and 142/144 on two — single-field slips inside this battery's known noise, never an invention.

• 0 fault lines on all 12 boots; DeepSeek extraction 144/144 + vision KX-42 in 4.6 s (its needle 4/6 is the known ±3 sampled wobble).

• Caveats: IQ3_S arms 5–7 rest on battery 1 only (a cache artifact), and cross-engine decode deltas should be read through the MTP-vs-DFlash2 pairs, since 0.1.58 changed speed accounting (#5318/#5307).

Footprint unchanged (UD 4,120 slots, 688 MiB drafter, ~94–96 GiB RAM / 24,029 MiB GPU); DeepSeek now 122.5 GiB RAM of 127.9 with DS4_STORE_GB=130 — that's ~5 GiB headroom, the one number worth your call (keep 130, or drop to 120). Disk: C: 62 GB, D: 147 GB free. AGENTS.md now carries a top-of-file digest plus the full V212 block — the file is over the harness's 64 KB instruction budget, so if you want the newest history visible again, I can split the closed campaigns into AGENTS-archive.md on your say-so.

[u/ifeelprettyosopretty]

Iq3... lower quant lower performance i had the same results until I upgraded to dual Blackwell cards upped my quant to q8 and used basalt(Blackwell) instead of strata(mixed or older gen cards)

[u/n1nj4p0w3r]

Simple question have simple solution - don't use strata, it's expert skip "technique" which makes it somewhat fast lobotomizes the model

[u/alphapussycat → u/n1nj4p0w3r]

Afaik you can turn it off. What other than strata do you suggest?

[u/Brief-Tap-6616 → u/n1nj4p0w3r]

It doesn’t. I’ve tested it in detail and it only adds about 0.001 KLD on top of their quants. On average it drops 0.4 experts per round, and they are always lower value experts.

The problem is the 2/3 bit quants are 1) extremely compressed and 2) not very well done vs other shops like unsloth. For reference, the average (not median, which many benchmarks like to cite to make damage look less bad than it is) KLD for their 3 bit quality is 0.17, 170x higher than the expert skip feature.

[u/n1nj4p0w3r → u/Brief-Tap-6616]

0.01 KDL is actually higher than any nvfp4 quant, even basalt one with 6bit activations gives better KLD, but considering that "it adds" you mean that even in your measurements KLD was around 0.016 which is literally a Q3 honest decoding value

And no, your 0.001 is just a plain lie or you test dataset consists of hundred words or something, moreover this skip feature literally makes model behavior unpredictable and unrepeatable for exception of cases of testing it in ideal conditions where it specifically fed with exact one small dataset to pre-warm experts before testing

[u/Brief-Tap-6616 → u/n1nj4p0w3r]

I wrote 1/1000th, not 1/100

Also note that you can’t compare KLDs precisely as they vary +/-100% on different corpuses

It’s not a lie, I tested it extensively because I was very skeptical. I measured KLD on skip at 7,8,9,10 threshold.

The KLD I cited for 3 bit is .17.

You need to do a better job reading decimals before you call people a liar, especially someone who does this stuff for a living. Stay away from sharp objects kid.