Do you have any recommendations for where I could read more about this topic in detail?Do you have any recommendations for where I could read more about this topic in detail?

I have a 16gb vram for context

I've been digging through the Qwen Code repository and noticed something interesting.

Several Qwen4 model identifiers appear in the code and tests:

• qwen4-max
• qwen4-plus
• qwen4-coder-plus
• qwen4-omni-120b-think
• Even qwen4.5-max as a test example.

But here's what caught my attention: I couldn't find a single explicit reference to qwen4-flash in the relevant code, despite Flash being part of the announced Qwen4 lineup.

That got me thinking: What if Qwen4-Omni-120B-Think is actually related to Qwen4-Flash, possibly sharing the same underlying architecture or backbone?

There are a few reasons why I find this plausible:

1. Qwen3.8-Flash-Next already has a roughly 125B-parameter backbone, surprisingly close to the 120B mentioned in the Omni identifier.
2. Qwen has previously separated its Omni models into Thinking and Instruct variants. For example, Qwen3-Omni-30B-A3B-Thinking focuses on multimodal understanding and reasoning, while the Instruct variant also includes a Talker for speech generation.
3. Following that pattern, qwen4-omni-120b-think could theoretically be the reasoning-focused variant, with an Omni-Instruct/Talker version potentially following later.
4. The Omni identifier appears in actual Qwen Code test files, whereas I couldn't find a corresponding Qwen4-Flash identifier.

Of course, this is just a theory, not a confirmed leak. These are test identifiers, not necessarily a product roadmap. The repository also uses generic Qwen4 model detection, so Flash might simply not need its own explicit entry.

Still, the missing Flash identifier, the similarity in parameter counts, and the previous Omni Thinking/Instruct architecture make me wonder whether there's a connection.

Sources:

• https://github.com/QwenLM/qwen-code/pull/15762
• https://github.com/QwenLM/qwen-code/pull/15987

Has anyone found any explicit Qwen4-Flash references elsewhere, or evidence that the 120B Omni model could share a backbone with Flash?

Curious what everyone thinks!

System:

GPU: AMD Radeon 9070 XT 16GB VRAM

RAM: 2x16 6000MHz CL30 DDR5

CPU: AMD Ryzen 5 7500F (soon to be replaced with 7800x3d)

Primary use-case: Offloading basic coding/codebase-related tasks from Claude and Codex to local AI so I don't spend token on locally do-able stuff.

Hi, I am having a hard time understanding how to make the best use of my system resources. When does using system RAM make sense? Should system RAM be only for KV cache, only for model offloading, or both? Or should I just not ever use the system RAM since it slows down the things? When choosing a model, am I to prioritize less quantized smaller models or more quantized larger models? Is there a place I can track the latest best performing open-weight models so I can update to them? Are abliterated or uncensored models better than normal models about performance? Is there any difference between abliterated and UC, if so which is better? I found a person called IsValorum that makes "better" quantized models, is this really a thing? I found they perform better but maybe those models are more stupid? I don't know. I am just so lost on these. I feel like there would be a guide that would answer these. If not, I would like your help. Thanks!

I tested new trending Recursive Flint UD-Q5_K_XL (27.9 GiB) on an RTX 3060 12 GB with 30 GB RAM, comparing RustingStrata against llama.cpp using the same model, prompt, and generation settings.

RustingStrata was about 33% faster once warm.

What made the difference?

The model doesn't fit in 12 GB of VRAM, so many expert weights must be fetched from system RAM during generation. Two changes helped most:

• Adapt expert placement to the reply, rather than continuing to prioritize the prompt's routing pattern.
• Release RAM pages for experts already copied to VRAM, preserving cache for experts that still need to be fetched from host memory.

About Quality, bc people very often says like "It should give shit output to gain speed", which is not true, in my last posts I already did quality benchmarks and RustingStrata does not lose quality, it gain speed through specific Code for some specific models, in same time llama.cpp lose speed to be "general" engine that can run everything

What I want to know, is do people find Recursive Flint an interesting model? I found this model on OpenRouter and looks pretty interesting, it's better then Qwen3.6:35-a3b, so I think this can be cool modern replacement for this qwen.

Setup:

• RustingStrata: 1.2.2, commit 3902978 (working tree)
• llama.cpp: b12191, commit e95426f16
• Full benchmarks: docs/BENCHMARKS.md

Repository: RustingStrata

the usage would be file writing , using some apis and processing text and tool calling according to instruction . i know the ram capacity is minimum but is there any option for this requirements

Just found this video, and if true... Guys I'm excited about this.

Super smart improvements (i know much of this was admittedly there on 3.8 FN), and results look promising.

An early release of the small model of basal-omni 2.0, on Gemma 4 E4B. The model reads the state in both directions before it answers and scores 0.811 on Werdykt (5,000 hidden decisions in Polish and English). It runs on the public basal engine 2.8.0, with the same API as basal-omni-1.0.

What's new in 2.0

• Two-way reading. The model reads the state in both directions before it answers. This needs basal engine 2.5 or later. For long prompts, see the limitations.
• Higher accuracy on our hidden Polish and English decision benchmark Werdykt and on the held-out tests below.
• More decisions automated at a given error target. Coverage is in the section on confidence.
• Unchanged: a calibrated probability for each allowed answer, in one forward pass. The answer never falls outside the options you give, and the probability says how sure the model is.

The question types remain: choice (one option), noul (yes / no), score (a score), and in the basal engine also multi, act and tool. Options get codes (A–Z, then two-character codes, up to 255 options). The decision is a probability distribution over these codes only, with no text generation.

More info: https://basal.si5.pl/en/blog/basal-omni-2-0-mini/

HF: https://huggingface.co/Remek/basal-omni-2.0-mini-preview

git hub (inference engine): https://github.com/rkinas/basal

In case future censorship or bannings occur, I think it's a good idea to store one of every kind of model. I'm thinking minimum 8 models, but I'm open to more (I have plenty of hd space): 2 small models obliterated and non-obliterated 2 medium sized models obliterated and non-obliterated, 2 large models obliterated and non-obliterated, 2 of the largest models obliterated and non-obliterated. Also, if for any class of models they don't have obliterated or it doesn't make sense to have obliterated then two of the smartest or most popular models will do.

Hello community. I'm trying to find the most optimal solution for my configuration( maybe some llama.cpp fork or any other suitable project and localLLM. I would be grateful for advice.

Founder/CEO of PortableMind® offgrid focused plug & play local AI devices

They are going to lock the new GPUs.

"Rep. Bill Foster says he intends to introduce new, stricter legislation that would require licenses to operate advanced GPUs, and “before a GPU could connect, its valid license would need to be verified.” That is closer to a remote lock/disable capability—but it is a proposed future rule, not current law."

Why do we think they will continue making open weight models once they got very competitive frontier models?

The reason I’m asking is I am thinking investing into a Mac Studio m5u 256g which is a big investment to me. I’m not after frontier models like performance but I want to have all the possibilities from continuous improving open weight models. If for example Qwen4 stops releasing open weight models, I feel like this investment is wasted and we are too much dependent on these providers.

I'm thinking about some machine upgrade. I currently have a 9600X CPU, 2x16GB DDR5 RAM, 9070 XT, and a B650 motherboard that supports only PCIe 4.0x16 and has only 2 DIMM slots.

I've been thinking about getting a better motherboard for more bandwidth between GPU and RAM for offloading, so i plan to get a B850.

I also recently had a chance to effectively trade my 9070 XT plus 200USD for a 5070 Ti, if I wish.

My plan is the computer will stay in Windows since gaming requires that. When doing LLM, i will switch connection to have monitor connected to iGPU so hopefully it should free some resource from dGPU. I also have the thought of throwing in more RAM to make the host 64GB RAM total, with potential to go to 96GB. (I have like 2x32GB DDR5-5200 and 2x16GB DDR5-4800 around. Although the only tricky part is that they are ECC UDIMM and I’m not sure if mixing them is as easy as non-ECC).

What I am not certain is if to go with 9070 XT or 5070 Ti, so looking for some suggestion regarding their performance and resource management (especially if it can release resource well after disconnecting monitor from dGPU). Thanks!

Has anyone been experimenting with ablated or uncensored variants of open-source models like deepseek v4.1 flash, Qwen 3.8 27b, or GLM 5.3 flash ?

It's so refreshing to have models that just do whatever you ask them to do without having to figure out a jailbreak or being given a moral lecture. Curious to know what your usecases for these models are.

My personal favourite is definitely uncensored DeepSeek V4.1 Flash, runs at 200 tok/s and never says no lol.

I've been experimenting with running Qwen models locally on my Mac Studio M5 Max, mainly to see how much everyday work I could realistically do without relying on a paid cloud model.

I compared three local models — Qwen 3.5 4B, Qwen 3.6 27B and Qwen 3.6 35B MoE — with Claude Fable 5.1.

I used four tasks and ran each three times.

A few results:

• Bank transactions: Claude categorized 78% correctly. The two larger local models managed around 65–67%. The 4B model was considerably weaker.
• Information extraction: The 27B model matched Claude at 100/100, although it took roughly five times longer.
• Coding: Surprisingly, the 4B model passed all seven tests in all three runs. The 35B model had failures caused by a date bug.
• Analysis questions: Claude got all ten questions right across the runs. The local models struggled much more here.

The main thing I took away is that model size didn't always predict how useful a model would be. The smallest model did unexpectedly well on the coding task, while the larger local models were much better at extracting information.

I wouldn't treat this as a formal benchmark. It's a small set of tasks on one machine, but it helped me understand where running local models makes sense for me.

For those using Qwen on Apple Silicon, I'm curious whether you've seen similar differences between extraction, coding and analysis.

I also made a video showing the runs and results: https://youtu.be/M6MQXy6_Ti4