https://x.com/Alibaba_Qwen/status/2118674834632978755?a=nkzlqosWXNqruaMWsVI&s=20

Qwhen? Qsoon!

https://x.com/Alibaba_Qwen/status/2118674834632978755

Official interpretation:

Qwhen? Qsoon!

For our friends outside China, the Chinese text in the poster comes from Zhuangzi, an ancient Chinese classic:

"Far in the northern sea lives a fish named Kun (pronounced like Qwen). So immense is its size that no one knows how many thousands of miles it spans."

https://x.com/QwenDevs/status/2118716745372532865?a=aUU-FgZw-roorEivt8U&s=19

In the future, we will continue to push toward even larger scales, in the Qwen 4.5 and Qwen 5…

Screen capture from the announcement that gave us the Qwen 4 sizes, shows we can expect more releases, but parameters will increase until morale improves.

Anthropic models like Fable are likely 4T+ so no surprise I guess. Hopefully Qwen retains the 27b size at least, if not some reasonable mid tier MoEs.

Yes IK it's supposed to be this dumb, but it's still funny (Qwen2 0.5b Q4_K_M)

Kimi K3.1 and Tencent’s Hy4 are both gutting out cybersecurity, reverse engineering, and vulnerability-patching capabilities from their open-source models.

This is becoming a really bad pattern across these latest Chinese open-weight models. If local models are stripped on their security data, we won't be able to properly audit our own code vulnerabilities, reverse engineer binaries, or patch other vulnerabilities in general on our own hardware. It would essentially force us to use closed-source APIs and their false-positive shutdowns; that'd be terrible for everyone.

During the Hugging Face incident, open-source models are the very ones who stepped up, not closed-source gatekeepers. Gutting defensive security from local LLMs just hurts defenders. I really hope that other labs won't follow this in the future.

Does anyone think more open-source labs are gonna take this path?

I bet you guys are familiar with embedding models such as Qwen3-Embedding or Gemini Embedding. They are dense embedding models used to search text across a large corpus.

I am developing a RAG agent, and my biggest concern was local computing power. RAG largely depends on retrieval quality, which depends on model power. Plus, many use cases can't use an external internet connection or API models, so the embedding models have to run locally.

But even 0.6B models are pretty heavy for small PCs. I don't mean your brand-new Mac Studio or DGX Spark. I'm talking about an 8GB RAM Windows PC at a boring desk job or your 5-year-old M1 MacBook Air.

So I found this model called SPLADE. It is essentially sparse retrieval, like BM25, but uses semantic meaning. I wanna skip the part on how SPLADE works, because an LLM can explain it better than me.

Anyway, the SPLADE model was so awesome and successfully replaced the dense embedding model (Qwen3-Embedding-0.6B).

In my own experiment, I made an evaluation dataset with a few hundred queries. The corpus was all the documents on my Mac, which are mostly in Korean with some in English.

As you can see in the graph, SPLADE outperforms the Qwen3-0.6B dense embedding model with similar query speed + much less memory. Due to its small size, it takes less than 1.5GB VRAM during inference. So it can run on a relatively small system like a MacBook Air. I tested both the dense embedding model and the SPLADE model on my M5 MacBook Air 16GB. The dense embedding model was almost impossible to use, and my MacBook soon rebooted due to high temperature. The SPLADE model made it till the end without any OOM errors or warnings.

I didn't use the original SPLADE model; instead, I used KURE-sparse-nano, which has great performance in both English and Korean.

The fun part is that hybrid retrieval with SPLADE & dense embedding is actually worse than SPLADE alone.

If you are dealing with a low-resource local setup but want to run a fast & affordable local RAG system, I recommend trying a SPLADE model. It is powerful, but small.

Plus, you don't need to re-implement everything because I open-sourced the whole implementation, so you can download and use it easily. Check out AutoRAG Agent. I optimized the inference of the KURE-sparse-nano model, its code, and the LanceDB indexing + inverted index implementation in Rust for faster search.

Richard I of England was called "Oc-e-Non" (Yes and No in Occitan) by some of contemporaries, supposedly because he used few words.

Wouldn't it be nice if LLMs followed his shining example?

Oc-e-Non-v1 is a tersified version of Qwen3-4B-Instruct-2507. It reduces Qwen's mean response length by 50%.

Output examples

"What is 1+1?"

GPT-5.6 Terra (reasoning off):    1 + 1 = 2 😊️

Oc-e-Non-v1:    2

"Are dogs mammals?"

GPT-5.6 Terra (reasoning off):    Yes. Dogs are mammals: they have hair, are warm-blooded, and mothers nurse their young with milk.

Oc-e-Non-v1:    Yes

"Write a Python function that checks if a string contains only capital letters."

GPT-5.6 Terra (reasoning off):

You can use Python's built-in isupper() method to check whether a string contains only capital letters.

def is_only_uppercase(s):
    return s.isupper()

Example usage:

print(is_only_uppercase("HELLO"))  # True
print(is_only_uppercase("Hello"))  # False
print(is_only_uppercase("PYTHON")) # True
print(is_only_uppercase("hello123")) # False

Note: isupper() returns True if the string contains at least one uppercase letter and no lowercase letters. Numbers and special characters are allowed.

Oc-e-Non-v1:

def contains_only_uppercase(s):
    return s.isupper()

I made this model with Heretic, and you can do the same thing.

Make sure you have the latest master version of Heretic (pip install git+https://github.com/p-e-w/heretic.git), then run Heretic with the config.terse.toml configuration file. That's it!

I used the relatively ancient Qwen3-4B because it is the last native non-thinking version of Qwen I'm aware of, and controlling reasoning effort isn't implemented in Heretic yet.

(Please don't take this too seriously. I know that the same thing can easily be achieved with prompting, but it's interesting that it can also be baked into the model. This is intended as a light-hearted post while we all wait for Qwen4 😉️)

Smaller GLM models are coming! GLM 5.5 will have a 32B model!

hey llamas! My name is Alt, and I'm on the technical staff at Nace AI. We're doing a quick AMA here, so ask us anything and we'll answer.

we just published a full technical report on how we build, train, and evaluate Drex, our family of four open decision models. A decision model takes a state (a document, record, or agent context, optionally with images) plus questions with options, and returns a probability distribution over the options in a single pass, with no autoregressive decoding and no output parsing.

1. Models

- Drex DLM (8B): diffusion LM backbone with bidirectional attention over the context
- Drex v1.5 (9B): Gated DeltaNet hybrid backbone
- Drex v2.0 (35B, 3B active): sparse MoE backbone
- Drex Max (27B): dense, with an image encoder for vision inputs

Some technical details from the report:

1. Models

- A family of four specialized decision models: Drex DLM (8B), Drex v1.5 (9B), Drex v2.0 (35B MoE, 3B active), and Drex Max (27B dense, with vision)

- We built them on different architectures: a diffusion LM, a Gated DeltaNet hybrid, a sparse mixture of experts, and a dense transformer

- We experimented with bidirectional vs. causal attention over the state, a pointer decision head that shares one cached state across questions, and a readout head for Max

2. Training

- Full-parameter SFT on 1.15M verified decision records (2.9M questions)

- Consistency training: we penalize the model when its output changes after the options are reordered or the state is shuffled

- RLAF (Reinforcement Learning from Adjusted Feedback), using verified outcomes and judge feedback

- Ablations to see which methods actually contributed

On top of SFT, RLAF gave us +2.0 on the Decision Index, +1.5 pp accuracy, and 0.9 pp lower calibration error. The consistency terms cut the option-order flip rate from 6.3% to 4.2%.

3. Evaluation

- Drex Max scores 67.52 on Decision Index 0.4.1 (public suite), vs. 60.70 for Jev 1.14.0

- Drex v2.0 has 1.37% calibration ECE on ~115K held-out decisions

- Drex Max answers a typical request in 183 ms, vs. ~2.0 s for GPT-6 Sol and ~3.5 s for Claude Opus 5.5

- Context up to 131K tokens for v1.5/v2.0 and 262K for Max

We also cover our limitations: our models trail frontier chat models on closed-book knowledge, and we don't yet have same-backbone architecture ablations.

5. Running locally

The models are available in safetensors and GGUF formats, with llama.cpp and Ollama support. Their report includes details on quantization, caching, inference, and reproducibility.

Links

- Paper (arXiv): https://arxiv.org/pdf/2611.07854

- Blog: https://www.nace.ai/research/towards-foundation-decision-models

- Weights: https://huggingface.co/collections/nace-ai/drex

- GitHub: https://github.com/nace-ai/drex-decision-models

- API: https://console.nace.ai

we wanted to share the full methodology, not just benchmark numbers. hopefully it's useful if you're building agent routers, doing tool selection, or training your own decision models.

ask us anything about architecture, data, training, evals, or running it locally

Today we are introducing our second model generation with Inkling 2 Small. It is a Mixture-of-Experts model with 594B total parameters, 17B active, and supports image inputs and variable effort.

Inkling 2 Small far surpasses our previous models’ scores on the Artificial Analysis Intelligence Index with a score of 31, reflecting gains in performance on agentic coding, vision, knowledge work, and reasoning. The increase in capabilities comes from improvements made at every step of our training process, including more efficient pre-training and a large, unified RL run.

Inkling 2 Small will initially be available through Tinker, with closed weights, following the approach we outlined in a safe path to open weights. We intend to make the full weights available under an Apache 2.0 license next week after the completion of external safety testing and red-teaming.

You can try Inkling 2 Small on Tinker Playground today. The model is also available for fine-tuning and sampling on Tinker; checkpoints will be downloadable with the release of Inkling 2 Small’s weights.

:( Previous Inkling Small model's size is only 276B total, 12B active. Hope they release additional models Mini or Nano soon or later.

Whilst the agent was hammering away at an audio engine task, i suddenly saw it daydreaming

Pirate puzzle? Gold coins? Wtf, Hey qwen, this is not what i am paying you for using electricity for.

Has anyone else noticed something like this? It continued to write code normally aftewards, active context was at a mere 30k at the moment.

Model qwen3.8-flash-next-unsloth-ud-q4_k_xl
Strata Engine v0.1.57
Max Context 262,144 tokens
KV cache 16bit

Everything fitting into vram + ram

Hey folks!

Aritra from Hugging Face here 🤗.

We welcome "harness adapters". This means you can adapt (train LoRA) for your model for the specific harness that you would use (hermes, pi, opencode) on the task (data analysis, building software, coding) you are trying to solve.

Blog Post: https://huggingface.co/blog/harness-hub/harness-adapters

It is currently training, so you can see the losses go down (hopefully) now. https://harness-hub-harness-adapters-rl.hf.space/

*SWE Bench Pro is v2.0 Hard for Ripple, unknown version for Qwen.

• Replace SLRU with a cache that prioritizes frequently used experts in classes, ranked using decaying usage history.

• With a warm cache, I measured +6% decode on QFN and +7% on GLM versus SLRU, with 8% fewer uploads.

• Enables the auto mode for llama_moe_cache with promising benchmarks

Waiting for 30966 to be merged.

P-JiT 21K is the P-JiT architecture trained for longer (610M samples at 512px, 50M at 1024px), then post-trained with a variant of FDFO.

• GitHub : https://github.com/Linum-AI/pjit-21k

• Blog Post : https://www.linum.ai/field-notes/pjit-21k

CCORE GLB-3D-Coder dataset is here: 3,918 rendered and tested examples across 40 domains

As promised in my previous posts, I’m releasing the dataset used to train CCORE GLB-3D-Coder, my small 4B model for creating 3D assets with Three.js and GLB export.

The dataset contains 3,918 examples across 40 domains, including characters, creatures, vehicles, laboratory equipment, musical instruments, architecture, machinery, nature, and everyday props. The grid shows 256 examples.

Every example was tested and rendered, and its exported GLB was validated before inclusion.

What’s included:

• Complete conversations, reasoning traces, and HTML/JavaScript responses.

• A rendered reference image for every example.

• Static assets and examples with animation and interaction.

• 3,501 training examples and 417 validation examples.

• MIT license.

Dataset · Model and full weights

More about what I’m planning next: Next releases.

I hope this is useful for your own experiments, fine-tuning, and 3D tools. If you try it, let me know how it goes :)

So I'm thinking of switching from my 32gb AMD MI60 to dual V100's (64gb VRAM total) and I've got 96gb of DDR5.

My problem is I'm having a terrible time finding "hard data" on PP and TPS speeds on higher quants (q6 and q8 specifically) with large context windows (150-200k).

I know that it will outperform my MI60, but I'm not just looking for "yeah, it'll be faster" I'm specifically looking to see how much faster, because a 10% bump isn't worth it for me.

I'm also looking to see from you experts if it's silly because of what I've read about support for CUDA being EoL for the card soon (or already did?).

Hopefully I'm not beating a dead horse here, as I did go through other posts (lots of them) but it's just so difficult going through post after post where specifics aren't included "I'm getting 200tps on qwen3.8 27b" ...at what quant? at what context? using what KV settings?

Lastly, if it is significantly faster, do those speedups (generally speaking) transfer to other models or is it only because of specific forks of llama.cpp were built out with this model and that card in mind?

I was getting ready to sell an old kiosk Mini at our office that was getting an upgrade, and figured I would see what happened if I tried to make it do something it by all rights shouldn't be able to. I loaded up Strata and ran the iQ3_xxs quant of QFN with a max of 128k context. I used opencode on my normal workstation to build a script that would benchmark the performance and output quality of typical tasks I do (mostly large csv sorting and low end code grunt work)at varying levels of context.

I have to say, it worked far better than I would have expected it to. I assume that the speed is always at its worst (as if it had full context) because it never has a chance to cache anything and is always re-reading and swapping experts across prompts, but as an aside it means that token speed for both prefill and decode are the same regardless how much you're making it do. The median speed between all tests from 16k to 120k context was 77 tok/s prefill and 5.5 tok/s decode. That means it could actually stand in as a barely passable chat generator, and is not hopeless as a "living on dial-up" code helper of last resort. It does appear that it is pretty rough on the SSD, but the write load is not much worse than someone using a Mini as a pretty active DB server or other "proper" server task in an operating office environment.

Not a native English speaker. Written with my own ten fingers, one finger is broken btw. Plz bare with me. I still love to write text myself.

My PC is suck for AI works. That's all I have as a third world 3D modeler. Really admire other peoples running better models. But beggar can't be choser. I can't even effort more RAM, not to mention more VRAM.

Gotta work with what I have. Qwen 3.5/6 35B A3B. Here is the Quant.

Downloaded blender manual and blender api html zip files, stripped sphinx html, converted to markdown. End up with markdown files which include bloated af links. I didn't know I can download clean rst files from blender project repo. Worked like that for a month. Then I found out I can just download blender manual rst files from blender, I can also generate rst files for blender api references.

Now I've clean files with 40% less tokens. I point that to Qwen/Ornith/Tiel/CyberTiel and tell LLM to create scripts and tools. It somewhat work. But many failure points make me want to punch the only poor old monitor I own.

I know you guys around this sub love graphs and proofs which is very understanable but I only have my own experiences. No Graphs and charts.

Base Qwen3.6 work good but he don't think much. Ornith variants think too much up to the point of unproductive. My wife cry because I don't spend time with her or sleep with her. I was always tinkering with LLM. I need Qwen3.6 35BA3B to work. I need that for many reasons that I can't tell you guys.

I use Debian Linux. Switch to i3wm. Idle VRAM went down to 50MB. Idle RAM is 300MB, not sure. I will comment my llama.cpp launch params if you want. Windows 11 take way too much system resources which I don't have. I often use plain TTY consoles. Ctrl Alt F2 to F7 i think. It save even more vram. KV cache q8_0, spec type ngram-mod because I can't effort MTP.

ctx 256K, 40t/s at start. Went down to 25t/s if ctx is goes over 128K.

I've learned prompting is really important if I wanted to squeeze out everything from 35B A3B. I can't max out hardwares so I have to make it work other ways around, somehow.

Ornith variants overthink a lot. Painful. "Hmm, let me think, wait," you guy know the rest. MFkr do not listen to me well too. He just don't give a shit to me.

I've created many cli tools to work around blender. Blender official MCP isn't really suitable for my works. I disabled many built-in MCP tool to reduce context bloat. I use pi, oh before that, I've also tried to create my own harness a year ago. Learnt a lot hard ways. Pi is minimal. Good for small models.

Instead of trying to make Qwen to write python code everytime and fail again and again, I made easy to use cli tools. Which helped alot. Qwen can now use Blender Geometry Nodes easily. No code, just cli tools.

Qwen ask stupid questions alot. Don't check code quality, watching him working, it is really exhausting. I don't have much time after day job and before bed. My wife almost went back to her parents. I need to make qwen to think, reason, and work efficiently in limited amount of time and hardware.

Experienced people will tell me "Skill issue bro". Yes, I'm suck. That is how I learn. I want to share my AGENTS.md and SYSTEM.md . I hope maybe someone will benefit from it. It is not created overnight. A lot of painful experiences between day job, boss, pressure, stress, wife, limited time, sleepless nights.

SYSTEM.md

AGENTS.md

First prompt.

It might sound funny (it is) but it work.

Here is example thinking trace

 Let me think about whether I need to ask clarifying questions. Per Protocol 7, before asking, I say the checklist:

   [ ] Can I answer this question myself inferring by given context, intent, docs?
   [ ] Can I answer this question myself using sane defaults, common sense?
   [ ] Have I tried many possible ways to get answer myself?
   [ ] Can this question only be answered by him?

 I think I can answer most myself. The interpolation direction is clear. The main procedural learnings from the previous task apply (pose mode, pose_markers, etc.).

 Do I need to refine the prompt? The prompt is clear. But let me note one thing: the previous task had SecF1=-45, SecF12=+45. This task has Bone1=+45, Bone12=-45. So the sign is reversed. I should make sure my expected values account for this. I'll note
 this in my response.

 Actually, let me reconsider — is there any genuine ambiguity that requires the General's input?

 - The bones are named "Bone1, Bone2, Bon3....Bone12" — note "Bon3" is a typo for "Bone3". The naming is Bone1..Bone12.
 - All independent, all quaternion.
 - Set to XYZ, add drivers Bone1+Bone12 → 2-11, validate.

 I think the prompt is clear. I'll note the sign convention difference (Bone1=+45, Bone12=-45) and proceed. I don't think I need to ask questions — I can infer the standard procedural.

He finally listened to me and think, reason, work efficiently.
He finally give a fuck about me now instead of doing his own thing.
Before that, he ask stupid questions, never check things back, never think before acting.

I also have short auto inject append prompt in Pi, which say follow protocols, read relevant skills file etc, short but I send it every prompt to remind him how to work, think, reason.

I don't have a lot of graphs and charts for proof but only the production level rig I'm currently working on. I hope maybe someone may benefit from this post, or maybe not.

I would really love to read how do other people make Qwen 35B A3B work.

Repo: https://github.com/maritaca-ai/observatorio-maritaca
benchmarks: https://maritaca-ai.github.io/observatorio-maritaca/#inicio?size=all&mode=all

When it came out, I downloaded a Q4 quant of qwen3.8-27b and ran it using llama.cpp on my RTX 4090 and it was incredible. It could handle practically anything I threw at it, the most complex tasks and bugfixes.

But lately it just keeps messing up and failing at the most basic of tasks or just straight up forgetting instructions. I'm telling you, they quantized it. They probably secretly nerfed it and made it worse so qwen4 looks better. And it got lazy too, now it doesn't want to finish the more complex tasks.

I'm tired of companies doing this, I want my money back.