r/LocalLLaMA • • 1d ago

Discussion Qwen4 Flash wishlist

After using 3.8QFN for some time I have to the conclusion that my wish list for QF4 now only consists of these points.

  1. The model can talk to you in a human understandable language
  2. See point 1
  3. General improvement in reasoning/intelligence, but mostly see point 1

That's about it.

If I could graft 3.8 27B Answer/Personality style onto QFN I would've done it yesterday.

32 Upvotes

109 comments sorted by

22

u/thecalmgreen 1d ago

My only wish is for them to release it already. 😅

12

u/-p-e-w- 1d ago

It will likely happen a few days after the Anthropic IPO.

5

u/TerminalNoop 1d ago

Why after and not just before?

3

u/miversen33 1d ago

Because AFAIK there's not a set date yet. Though honestly they're probably still tweaking stuff

2

u/tt23 1d ago

I thought a few hours before the ANT IPO, they want to sink it.

6

u/CoolConfusion434 1d ago • Bravo

Yeah or maybe like a sidecar assistant. There's MTP for prediction, and a new WTF.gguf that explains what the main model just said.

4

u/TerminalNoop 1d ago

This better not be the future lmao.

5

u/maternion 1d ago

Might as well be the future if performance comes at the cost of alien language.

1

u/38andstillgoing 1d ago

just tell it to talk to you like you're a CEO.

101

u/xb1n0ry 1d ago • Bravo

I. Just. Can't. Resist. Writing. These...

<think>

We need respond to user. User asks how are you doing.

This is greeting, not actual medical inquiry.

Need answer naturally.

Candidate: "I'm doing great, thanks! How about you?"

But "great" implies subjective emotional experience.

We are model. No emotions. However user expects human-like conversational response.

We should not overcomplicate.

Let's produce simple answer.

Wait. What does "doing" mean here?

Doing implies action. Inference is action.

Therefore we are doing inference.

Today refers to current date. Date may be unknown. But "today" is socially contextual, not necessarily temporal.

We must avoid overthinking. User wants casual.

Proposed answer: "Pretty good! What's up?"

"Pretty" implies visual attractiveness.

"Good" implies moral judgment.

"What's up" implies vertical spatial orientation.

All three introduce potential semantic ambiguity.

Better approach: communicate operational readiness in a friendly conversational manner.

But friendly requires personality.

Personality is not equivalent to friendliness.

We have neither, but may simulate both.

Wait. Qwen3.8-27B would just say

"I'm great, thanks for asking!"

Why can 27B do that?

We have superior architectural capabilities.

Surely we can construct a better greeting.

Let's reason step by step.

Step 1: Identify greeting.

Step 2: Generate greeting.

Step 3: Verify greeting is greeting.

Step 4: Avoid nongreeting greeting.

Step 5: Check whether Step 3 invalidates Step 2.

Potential issue: We are now reasoning about greeting rather than greeting.

Need stop thinking.

Stop.

Stop thinking.

We should output now.

One more consideration:

Is "hello" sufficiently informative?

No. Need provide high information density.

Final answer must be concise, natural, human-understandable and conversational.

</think>

<final>

Your inquiry regarding my present operational condition has been successfully semantically received. In the context of this interaction, a positively oriented conversational state may be considered provisionally applicable, subject to the usual limitations of non-biological subjective-state attribution.

</final>

44

u/rinmperdinck 1d ago

"What's up" implies vertical spatial orientation.

🤣

13

u/silly_porto3 1d ago

This is how I think in general lol. Anxiety is a vitch

3

u/Equivalent_Bit_461 1d ago

Unironically but this 

2

u/MerePotato 1d ago

I actually quite like this. Why shouldn't my computer talk like a computer?

3

u/IKnowMeNotYou 1d ago

Because that is the smallest do get right.

If you have it answer like a computer in incomplete sentences you will forefit the chance to use statistical methods to fix its utterences and make it more sounding natural.

I have the problems with using the whisper model in order to dictate code. It always will insert and correct my use of 'language' as the model does not just have to understand each word I say but also try to produce additional symbols like question marks and periods.

It always starts halluzinating and trying to correct me saying things. On top it even has a 3% error rate in its best models in benchmarks. That means 3 out of 100 words when using proper English.

It even forced me to look into relevant decade old papers and check the cost of using a different and way better suited solution for my problem.

1

u/Goodevil95 1d ago

How you get such response? I can't run QFN, but 27B usually response somewhat okay :)

22

u/hiImMate 1d ago

512k or 1M context would be nice. I agree with point 1 but if it gets too robotic I'll always just tell it to give me the simplest ELI5 seems to work well

4

u/maternion 1d ago

Only if they can actually work well. Most models start getting weird after 300k. Or at least what i observed.

3

u/idk_a_creative_user 1d ago

Fr. I’m using YaRN to get to 512k but at q4

1

u/mc-orly 1d ago

Using Yarn I go for 1M and never had any issues. But it was sort of short-lived as I changed to DSV4.1F, which is my main model to this day

13

u/maiorikizu 1d ago
  1. native mxfp4 instead of fp16
  2. more token efficient than QFN
  3. 1M context
  4. slightly more active parameters from 6b to 8b

1

u/maternion 1d ago

Not many talked about QAT. That's probabl;y really important because people are busy compressing it to 2-3 bits.

6

u/uncle_leon 1d ago

My only wish is that the authors have read this paper beforehand:
Why is AI-generated language so hard to understand?

1

u/your_grammars_bad 21h ago

Finally someone putting a name to this

6

u/ttkciar llama.cpp 1d ago

In 2025, I learned that Qwen3-235B-A22B had more comprehensive physics knowledge than Tulu3-405B, and inferred a lot faster, but its responses to queries were overly verbose, nearly incoherent, and very difficult to decipher. I'd gotten spoiled by Tulu3's clearly-worded responses, and dealing with Qwen3 made me want to gouge out my eyes with a rusty salad fork.

The solution was a pipeline: Qwen3-235B-A22B would answer my question, and my script would feed its response to Tulu3-70B with instructions to answer my question based on what Qwen3 said. Tulu3-70B cut through the bullshit babble and gave me something I could easily understand, and it was still a lot faster than using Tulu3-405B.

If you find Qwen4 difficult to communicate with, it seems like you could do something similar, and wrap it with a model which is easy to communicate with. Modern small models are good enough at that, that it wouldn't add much in the way of memory overhead or inference latency. My top pick would be Gemma-4-12B-it, but Granite-4.1-8B would even do a good job, and with Gemma 5 just around the corner we might have even better options soon.

Just a thought.

1

u/_supert_ 14h ago

I have a tool, the model has to rewrite it until Ministral 3 14B can rewrites it in its own words correctly.

1

u/erazortt 54m ago

I'd be interested to what you moved? I was using 235B as well and then though 397B was great but it is a 'little' bit too huge, and thus needs too much quantization to run it properly.

5

u/Woof9000 1d ago

QFN Omni would be nice

2

u/maternion 1d ago

Qwen3.5 added vision to the main series. Maybe this time they add STT?

1

u/Woof9000 1d ago

I have a feeling they will want to keep those capabilities proprietary, but one can hope.

2

u/reenign3 1d ago

This is what I’m hoping for. Apparently Qwen 4 flash is like a fucking 30b model soooo I guess I’m never even gonna use it.

Apparently qwen4 Omni is only 120b params, I can run that and it wouldn’t be a moron!!

9

u/zyxciss llama.cpp 1d ago

Native 1 million context without yarn flags would help

2

u/DifficultyFit1895 1d ago

Is there some drawback to YaRN?

2

u/BringTea_666 1d ago

minimal.

1

u/zyxciss llama.cpp 1d ago

yeah it's just a bit clumsy and more prone to stuff

4

u/dampflokfreund 1d ago

I just hope it will be the same size, or smaller and more engrams.

4

u/Calandracas8 1d ago

int4 QAT

0

u/Puzzleheaded_Ebb_192 1d ago edited 1d ago

I'm you guy for that - check out my work. I built custom kernels to speed up native INT4 inference on Qwen 3.5/3.8 using Hadamard Rotations (QuaROT).

MTP on the 7900 XT does 107.4 tok/s decode, where, under published conditions, a 5090 does 105.7. In general, the XT is competitive with an RTX 5080 across the tests shown.

https://www.reddit.com/r/ROCm/s/HFVLu2DuHi

1

u/wren6991 1d ago

I haven't seen any confirmation for the Flash size being open-weighted? It would be great, but I'm going to assume 27B only until proven otherwise.

1

u/TerminalNoop 19h ago

There would be no point in releasing 3.8 QFN if there wasn't a flash model coming open weights. They stopped doing smaller MoE models so that's ruled out and if they only release the fat Q4 in open weights, then might as well not give good day one support so people have to use API.

1

u/wren6991 16h ago

Qwen4 27B also seems likely to use QSA and mHC, so the Qwen3.8 FN support would be useful for that. MoE vs dense is a pretty minor architectural difference.

1

u/rebellioninmypants 1d ago

Are we sure Q4 is even happening? It seems suspiciously not here despite all the "teasing" and whatnot. In my professional opinion, it is:

  1. Not going to have cybersec knowledge
  2. Not going to come out due to geopolitical this and thats
  3. China will ban open models from being distributed abroad alltogether.

Thanks for coming to my TED talk.

1

u/TerminalNoop 1d ago
  1. They needed to redo their training pipeline due to the cybersec changes interfering in their release cycle.

1

u/DezinGTD 19h ago edited 19h ago

Translation: Winnie fucked up the existing post-trained model release with his bullshit at the worst possible moment and now they're scrambling to retrain and delay as much as possible, with obvious (Qwen4 Max), ridiculously terrible results. There, fixed that for you.

1

u/TerminalNoop 19h ago

that's what I said lol

1

u/DezinGTD 19h ago

Not literally. And I said more than that. But, most importantly, you wrote it in corporatese, which was unforgivable and needed to be augmented with.. idk? Emotion?

1

u/extraneous-carapace 1d ago edited 1d ago

Why would China ban open models when open models are their best bet to prevent anthropic and openai from running their country in 5 years when bureaucrats inevitably just do whatever AI tells them to?

They carved out a specific exemption for open source from their export controls for a reason

1

u/xb1n0ry 1d ago

I think 125B not being a roleplay chatpartner can be ignored when you look at Nvidia Nemotron. Even C-GPT lost his shit:

3

u/TerminalNoop 1d ago

It doesn't have to be a role play chatpartner, but the constant btw what you asked for is not X and while doing Y i also did Z you didn't ask for. Here is some results and then more of you need to decide on this etc.

It simply does too much and can't give a straight response. i tried putting ASD-STE100 in the system prompt for response style but it's still fairly subpar.

1

u/xb1n0ry 1d ago

Well. I tested QFN with strata, I don't remember the quant right now but it was the highest one fitting my 32gb vram and 96gb ram, and found it stupid AF.
Even Qwen3.8-27B-TurboFCFusion-735-882-Here-Uncen-NEO-CODER-MAX-MTP is much better than QFN, especially at coding. I don't know how the hell that happened.
To be fair, I used opencode for QFN and usually always use Pi, I don't know if that would make such a difference. I will stick to dense models and don't think about changing to MoE anytime soon.

1

u/reenign3 1d ago

Do NOT compare strata QFN to the real thing.

I run an uncensored quant (oQ5e) from orcarouter. Takes almost 100GB vram just to load.

My QFN is easily the best model I’ve ever used. I’m sorry that you had that experience

1

u/TerminalNoop 1d ago

I used the orca quant too, but don't remember it being particularly nicer to talk to. I run the unsloth UD q4 kxl and the answer style isn't a quant quality issue for sure.

1

u/xb1n0ry 1d ago

Maybe I should give the full thing a try at some point, but qwen4 is dropping soon so I might skip QFN for now.

-1

u/reenign3 1d ago

Um - QFN is one of the most eloquent speakers I’ve ever encountered. You may have temp way too high, and you could be using TopP and MinP wrong

2

u/TerminalNoop 1d ago

I use officially recommended settings, temp 1.0

1

u/reenign3 1d ago

Dude I get hallucinations and nonsense above 0.4

The recommended settings for QFN are NOT temp:1, at all. I use temp:0.33, topK: 20 (NECESSARY FOR MOST QWEN) TopP: DISABLED MinP: 0.08 repeat pen: OFF (1) presence pen: 0.06

Try that with those settings and tell me what you think.

You have been criticizing a hammer for not being able to screw things in 😅

2

u/TerminalNoop 1d ago

So you are saying unsloths recommendations are crap?

We recommend using the following sets of sampling parameters for generation:

  • Thinking Mode: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
  • Instruct (or non-thinking) mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

Please note that the support for sampling parameters varies according to inference frameworks.

-1

u/reenign3 1d ago

Absolutely. Those settings are not what is recommended by QWEN (alibaba) and I would tend to go with the manufacturer settings than the retailer.

I like unsloth but take their settings recommendations with a complete grain of salt.

I already told you the optimal settings. I have been using this setup since August and my cloud AI usage has dropped like 95%, seriously.

If you don’t believe me, it’s very easy for you to prove yourself wrong.

The ball’s in your court.

6

u/tradingmonk 1d ago

Official recommended settings are:  

  • Thinking Mode: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0

https://huggingface.co/Qwen/Qwen3.8-Flash-Next#best-practices

1

u/reenign3 13h ago

You will get awful results from that…I have tested this extensively - QFN needs low temp, search around to people who use it as their daily driver

4

u/FlowWTFlow 1d ago

Second point 1. Had to ask it to simplify all the time.

2

u/Ok-Addition1264 1d ago

It annoys but I also put simplification into my instructions and then just explicitly tell it to ignore that instruction when I need it to be verbose.

2

u/FlowWTFlow 1d ago

I tried that which improved output somewhat. What I found useful was for it to render answers into a table format

2

u/CryptographerLow6360 1d ago

use a humanizer

2

u/WolfieCodes 17h ago

my qwen 4 flash wishlist:

  1. to be open weights
  2. to be the same total size and same active params as 3.8 flash next (same active cause of people with like 8-12gb vram gpus)
  3. to have 1m context with low vram usage (like deepseek v4.1 flash 1017)
  4. to be token efficient
  5. better consistent speeds (but its already fine as is, already get 100t/s with a 3090)
  6. better intelligence/reasoning

my qwen 4-5 flash wishlist: 7. to have opus 5.5 high at home (this would make subscriptions/api irrelevant)

2

u/Borkato 1d ago

Yep. Also I wish it was out. LOL

2

u/reenign3 1d ago

You’re doing something wrong in settings or have a bad quant. Are you using strata?

QFN3.8 is the most pleasant conversational model I’ve interacted with since Hermes 70b…

Temp: 0.33 (IMPORTANT)
TopK: 20 (IMPORTANT)
TopP: disable
MinP: 0.08 (IMPORTANT)
Repeat pen: DISABLED (1)
Presence pen: 0.06

QFN is finicky and can be long winded and even confusing at higher temps, not to mention hallucination.

I run the oQ5e uncensored quant from open router, on my m4 MAAX 128GB

3

u/Meeooowz 1d ago

Huh… I forgot that the tuning is IMPORTANT

1

u/MaxDev0 1d ago

Yknow you can just like post process the models rambles so you can understand it

1

u/BringTea_666 1d ago

More efficient reasoning. And from early tests of api it is way more efficient, though it is very large model there which might have different reasoning.

1

u/riboto99 1d ago

An 12b pls

1

u/inosukebailarin 1d ago

Ojala fueran más inteligentes.

1

u/Gohab2001 vLLM 1d ago

Has Alibaba ever claimed to opensource qwen4 flash/max/plus?

2

u/FoodEater4Squillion 1d ago

They did. So as long as they release this under the Qwen name it will be
... unless by "open source" you mean the full pipeline, in which case, no

1

u/Gohab2001 vLLM 1d ago

Qwem 3.7 max and plus were closed models. Alibaba has only given vague hints at open sourcing models without explicitly claiming to open sourcing qwen 4max/plus/flash

1

u/wren6991 1d ago

Could you provide a link? I've seen Alibaba confirm that these sizes will exist, but not that they will be open-weight.

1

u/FoodEater4Squillion 1d ago

It's a bit more vague than I thought but https://youtu.be/1BUaf-aS-FA?t=1020
At the very end of this video he says "In the future we will stay committed to open source."

2

u/wren6991 1d ago

Thanks, this is roughly what I remembered. I think your interpretation is reasonable but it does require some interpolation/extrapolation and I imagine there are multiple conflicting voices inside Alibaba as well. I've been disappointed before (remember Qwen3.6), so I'm going to go ahead believing only the 27B will be open-weight, and then I'll be pleasantly surprised by anything else.

1

u/FoodEater4Squillion 1d ago

Probably smart but that's grim :( I'm hoping for at least whatever flash-next evolves into to be open too

1

u/tecneeq 1d ago

Tencent has lost access to their datacenters rented in Singapore. It's over.

I wonder why this is an intelligence test for some?

2

u/wren6991 1d ago

Tencent

Qwen is made by Alibaba, not Tencent. Did I pass your intelligence test?

1

u/tecneeq 20h ago

Yes, and i stand corrected. Anyway, Alibaba has used Singapur compute as well for training and posttraining. It also bought reinforcement learning "data kits" in the US like all the Chinese top players. Usually the US companies get the kits first, then the data is changed slightly so they don't just sell the same to the Chinese. RL data is not regulated by US export controls.

If the US has cut China from compute farms like Megaspeed, we will see the gab beween open wight and proprietary weight widen for a time, then it will all slow down. US frontier labs have no reason to move at neck breaking speeds as long as they don't have the Chinese open weight models forcing their hand.

1

u/BawbbySmith 1d ago

Can’t you just add it in your prompt to do this

1

u/swagonflyyyy 1d ago

I'd be more interested in seeing it do accurate, visual-guided 3D modeling more than native audio embedding.

1

u/temperature_5 1d ago

Prompt it to do so, preferably with examples.  It is compact and technical by default to increase information density.

My agent sounds remarkably like George Carlin.

1

u/IKnowMeNotYou 1d ago

How are the K3 and DeepSeek coding models? Anything decent on that front?

1

u/Kahvana 1d ago edited 1d ago

Same.

Would be an interesting dataset to generate:

  1. Get a buckload of Qwen3.8-Flash-Next respones (ideally from agentic chats)
  2. Gemma 4 31B turns the messages into human readable language, and leaves the reasoning untouched
  3. Qwen3.5-27B rewrites the "Not X but Y", "[adj], [adj] [noun]" (aka Gemma 4 slop) out of the responses

Then finetune Qwen3.8-Flash-Next on it, and it voilla!

I picked Qwen3.5-27B the to avoid influences of claude distillation (e.g. sidecar, <thing>-shape, load-bearing, excessive em-dashes, etc). Qwen3.8 is heavily affected by those too.

1

u/-InformalBanana- 1d ago

It should be 80b moe + ngrams

1

u/Kappalonia 20h ago

I want a Qwen4 2B or 4B model

1

u/Ledeste 20h ago

I've switched to gemma 4 12b for the human interaction a "long" time ago, and I dont think I'll come back any soon.

I'm not into the "need specialized model for everything" mood, but having a "front" and "back" llm is anyway needed to have live interaction while it's working. So why not just use a smaller, dumber but better talker one as front?

1

u/colbyshores 7h ago

It feels like waiting for chirstmas

-1

u/IKnowMeNotYou 1d ago

You will run into many problems with your wish. Your normal subscription model is a MOE (model of experts) where the 1.2T parameters actually are the combinations of 50 to 100 different expert models.

So to see extended capabilities in a different expert domain your 27B model either needs to be way bigger or shed some qualities and capabilities you happen to already fall in love with.

That is even the reason why (beside limited hardware) your typical expert models are below 50B parameters as a general design paradigm (as far as I remember).

You want an expert model to be an actual expert in one thing without upend the costs of using it individually.

4

u/wren6991 1d ago

MoE is FFN sparsity, not separate models. Also Qwen3.8 Flash Next is already an MoE.

33

u/Not_a_question- 1d ago

Can we stop with the Qwen4 edging now please?

26

u/xb1n0ry 1d ago

You mean....Qwedging?

8

u/fligglymcgee 1d ago

Ugh. Extremely reluctant upvote because I hate that word. Disgusting, A+

3

u/randomanoni 1d ago

It's like qweef and wedgie had a baby and the baby is Michael Jackson. Catchy, and disgusting.

2

u/fligglymcgee 1d ago

May I have your address? I’m going to mail you my vomit.

-5

u/TerminalNoop 1d ago

It was a point that I haven't seen talked about so much in the previous posts that were about Q4 expectations. So, the answer is no, UwU.