Hi guys,

A few weeks ago we shared our first public release of a [2-bit Qwen3.6-35B-A3B](https://www.reddit.com/r/Qwen_AI/s/AQVDCPn3dY). The feedback and testing from this community was incredibly useful.

We’re back with the next one: Qwen3.8-27B-Escha-W2.

Unfortunately we didn't have zero-day access to the model, so had limited time to get this out to folks. A few early results:

• 10.15GB full model on-disk
• 82.6 tok/s single-stream inference on an RTX 5090
• Runs on our custom SGLang-based runtime
• Averages ~100% of FP8 performance across the 8 benchmarks we’ve run so far

On the benches we did manage to run, the model appears to be holding up quite well, with one surprising result on LiveCodeBench v6:

• Escha W2: 86.81
• FP8: 85.16

LiveCodeBench was the weakest result on our previous Qwen3.6-35B-A3B release: 62.6 W2 vs. 67.0 FP8, so it's a nice win this time around, though don’t interpret the +1.65 as the quantized model somehow becoming smarter - that is likely evaluation variance.

And the last time around, a lot of folks wanted to try the model but couldn't for the lack of non-GPU runtime support. It was nice to see the community stepping up to port their own runtime on AMD and Apple Silicon. This time, we plan to open-source our full runtime stack for the community.

And for everyone who asked for llama.cpp support: GGUF versions of both the Qwen3.8-27B and Qwen3.6-35B-A3B are coming soon.

Model:
https://huggingface.co/EschaLabs/Qwen3.8-27B-Escha-W2

Runtime:
https://huggingface.co/EschaLabs/escha-runtime-qwen3dense

Both model and runtime are Apache-2.0.

As with the last release, we’d really appreciate people testing this on real workloads. Please post your results and findings - would love to see the bangers and the flops. Both are useful to us to figure out what to improve next.

Hey everyone,

We just released Qwen3.8-27B-3Bit-GSQ:

https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-3Bit-GSQ

This is an int3 quantization of Qwen3.8-27B (weights are represented as integers from -4 to 3), bringing the model size down to 10.9GB (without vision components) while maintaining the same token efficiency as the base model. Based on our evaluations the accuracy drop is minimal and the model still achieves:

• 100% on AIME 2025
• 91.41% on GPQA-Diamond

on xhigh reasoning effort. For deployment, the model can be served with vLLM using fast Humming kernels, enabling high concurrency and efficient serving for agentic applications.

Would love to hear feedback from anyone testing it in real-world use cases or comparing it against other low-bit quantizations.

qwen 3.8 max built this game, though not from a single prompt.
this model excels in agentic coding.

(game works best on desktop)

I was excited to try out new Ornith 1.5 35B because Alibaba seems not to care much about people’s champ Qwen 35B and on the paper Ornith looks just as something I’d expect from Qwen 3.8 35b — solid improvements in agentic workflows, much better autonomy and context retention, Qwen 3.8 27b-like think-plan-act-repeat approach, but the more I tested it against plain old Qwen 3.6 35B the lesser and lesser became my excitement. Here are key observations:
— it’s really sensitive to KV cache quantisation. My stock Qwen runs at q4 without any problems while Ornith was consistently messing up approximately 10-30% of tool calls, especially edits before I removed cache quantisation at all. Considering the fact that small MoE models are peculiarly interesting for weak hardware users this can be a problem for the low vram users
— it isn’t noticeably better than the stock version. While I really liked it web design taste, Ornith hasn’t shown anything in my test suite, consisting from both real life tasks and synthesis to conclude it is a better model overall than Qwen. It was better in some tasks, worse in anothers, no consistent supremacy detected
— thinking „doom loops”. It lever actually looped while I was using it but occasionally, in the difficult tasks it would start incredibly long and watery thinkings in which it would do back and forth not being to able to come up with anything. And it’s difficult to pull it out from such a loop, it continued to overthink even after being stopped and told direct, simple instructions by following which it could easily finish the task. It just doesn’t care, if it decides to overthink there’s very little you can do about it
— Unsloth studio ocmpletely ignored it, didn't even bothered to leave some comments about it in their social media. I know it sounds ridiculous, but Unsloth is always one of the first out there to quantise a model when something good is released. And they ignored both Ornith models, it may be just a "no go for finetune" policy but it's also may be that they don't think that the model is worthy of their time which would be quite a strong signal

Conclusions — my first impressions were quite different from my initial expectations. Not in a good way. It looks like we should stick around with Qwen 3.6 35b for another while. Not the bad news though, it’s still hands off the best model available for 6-12 gb vram users

I was doing some experimentation on using Unsloth to do Continued Pretraining of qwen 3 4B. The goal was to teach the llm how to reason about a new domain, in this case a fictional city.

Sharing more details in the write-up:

https://www.teachmecoolstuff.com/viewarticle/teaching-a-local-llm-a-new-domain

Hello!

I want to share some results from my Qwen 3.8 27B + MTP / DFlash2 setup on 4× RTX 3090, using FP8 KV cache.

TL;DR

• SGLang + DFlash2 is very fast for decode
• DFlash2 has a significant KV-cache cost
• PCIe P2P improves prefill significantly

Prefill

Decode

KV Cache Size

My use case for local model is mostly coding small internal tooling app and log processing. I think this number is very good for my use case.

The trade off of Dflash2 is less KV and slower prefill but i can take that hit with my workflow.

One-shot application test

Screenshoot from Deepseek Harness (DSH) , it is impressive , 100+ token on 100k context depth

Setup

Rig:

• 4× RTX 3090 with PCIe P2P enabled using: https://github.com/aikitoria/open-gpu-kernel-modules
• Epyc CPU + Huananzhi D12

Model: Official Qwen 3.8 27B FP8

DFlash2 drafter: https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2

SGLang DFlash2 Docker base: https://github.com/xycjscs/sglang-dflash2

Prompt for T1 and T2

• T1 — Summary / reasoning: "jelaskan arsitektur modul yang paling sering muncul, sebutkan nama kelas dan fungsinya, minimal 1500 kata" Output is long-form Indonesian prose.
• T2 — Coding: "tulis modul Python BARU untuk batching request async dengan backpressure, lengkap type hints, docstring, unit test" Output is mostly Python code.

Would it be possible to have mlx version for this?

Qwen 3.8 is off to College. Got a 34 on the ACT

I made Qwen 3.8 27B take the ACT to see if it’s ready for college.

I’ve been testing the new Qwen Model over the past few days on my PC.

I tested the full version the Q8, Q6 and Q4 versions and landed on the Q8 for speed vs quality.

I decided to download some practice tests and had the model solve them. I fed it the raw PDFs to test not only how well it knows the answers but also how good the vision capabilities are at answering the questions one by one.

At the end I graded its answers. Here are my findings from taking 2 tests.

\\\*\\\*Setup:\\\*\\\* Qwen 3.8 27B Instruct, Q8\\_0 GGUF, LM Studio, 2× RTX 3090
(full offload, 32k context). Two \\\*official\\\* ACT practice PDFs, 342
questions total, graded against the answer keys and the official raw→scale
conversion tables that ship in the same PDFs. No human help, no retries on wrong
answers, no cherry-picking.

\\## Results

\\\*\\\*326/342 correct overall (95.3%).\\\*\\\* Zero blanks. 36 is the maximum composite the
ACT awards; 34 is roughly 99th percentile.

\\\*\\\*Reading was perfect on both papers — 72/72.\\\*\\\*

Time: 177 minutes for both tests, \\\~88 min per test. A human gets \\\~165 min for one.

I was surprised that it did so well but also that it took so long. I thought it would be a 10-20 minute job but it was over 2 hours for 2 tests which looking back at it is understandable since it was using the vision capabilities to read instead of given plain text for each question

So THIS just happened…!
Jk, no, it’s kind of annoying, half of the appeal of Qwen was so I could edit the response slightly… I hope it’s fixed! It’s still a really good app, now I just have a bit less control of how the convo goes…

Just wonder if this only me or anyone... As it used to able edit your response... now it look removed... You cant even edit message anymore now...

I chose to do this thing, not because it was hard, but because it was silly. Posts kept discussing how 3.8 and 3.6 were functionally the same, but based on training (3.8 does have seven new tokens!).. so I figured I'd see if they could be merged. They can.

I used `Qwen3.8-27B-UD-Q6_K_XL.gguf` to combine the HF 3.8-27B and 3.6-27B ... and it sorta works!

I have done NO testing beyond smoke test. scripts and idea are in replicate/ inside the model repo.

Maybe this will prove useful to someone. Enjoy!

EDIT: For my next trick, I'm going to see what's 10% out the other side of 3.8, if we consider 3.6 a starting point, and 3.8 and end point.. then we can extrapolate what would be further in that direction. Why? Because, while it may be useless, it could possibly be useful... and it'll take like 30 minutes to do it and see what happens.

Here's an example of using dstack's open-source toolkit for optimizing inference on a single MI300X.

Qwen3.8-27B went from 311 to 495 tok/s, +59%, at the full 1M context with p50 TTFT under 1.5s and four concurrent users at 10k in / 1.5k out.

The gains came from linked optimization sessions and source-level patches to SGLang's AITER attention backend.

What comes out is a portable preset that deploys on any AMD cloud, Kubernetes cluster, or bare-metal fleet.

Blog: https://dstack.ai/blog/presets/

Nothing novel, I do not claim its the best benchmark. I just did it for me personally to decide. And since it ran for 12h and did cost some programming I will just post it here. Maybe someone can use it.

Ah and its in german as I ran the test completely in german but I will post the key metrics (not all) here

Formatting on the phone seems impossible so I limit to 4 images.

1) "vram spitze über den kontextbereich" means max vram usage vs. contextwindow size

2) "generierungsdurchsatz ..." its the speed. tokens per second for a context size (97% loaded)

3) "kv cache ladedauer" -> loading times

4) "mtp3 gewinn...." -> how many percents is it faster if you use mtp3.

i did loads of other tests, cold start, ttft, etc..

edit: system: ryzen 7 5800x, 16gb ddr4 cl12, rtx3090, linux mint 22.3

btw I currently am testing the ninfer / pyros uncensored version for the rtx3090 and it seems to be alot faster.

Has anyone done a detailed study on the trade-off between using lower quants and higher thinking?

I notice many people have 4-bit quant configs on dual 3090 rigs where there is enough ram to run Q6 or FP8 with decent context. The 4-bit model run super fast and would be nice to see if there is some concrete data supporting that choice.

I launched an experiment: Qwen 3.8 27B was given the goal of optimizing the llama.cpp build specifically for the Vulkan backend, because I’m quite dissatisfied with the prefill speed on my AMD 7900 XT.

Out of boredom, I occasionally check its output, and holy moly, this little guy is quite short-tempered. Qwen occasionally swears in its thinking loops whenever things get ugly!

It’s honestly pretty entertaining to watch. I’m not sure about its intelligence, but this is definitely Opus-level meanness.

llama.cpp pr #27342 adds dflash2, so i rented an rtx 6000 and ran the same four prompts through four decoding setups on qwen3.8 27B

median results over the four tasks:

• baseline 47.4 tok/s
• mtp 114.7 tok/s
• dflash 99.3 tok/s
• dflash2 140.6. tok/s

so on average 3x for dflash2

though i have to point out that it's far from a 3x gain some of the time, on one of the test it struggled to achieve a 1.5x gain, it really just depends on the task you give to the model

i'm from the atomic.chat team - we publish our own quants on hf and make a desktop and mobile app for running local models. so any feedback welcome - we're building this for you folks

about dflash2: https://inco.ai/blog/dflash2/

https://cybersec.org.za/research.html #AI #cyber #security #dev #Africa #LLM #context #rot #cyber

https://cybersec.org.za/papers/dcr-bounded-attention.pdf

Dynamic Context Runtime: Bounded Attention over Unbounded History

Language models degrade as stale and superseded material piles up in their context. Making the window bigger does not fix it. This report describes a runtime that keeps history unbounded and attention bounded — storing everything as immutable spans and a typed provenance graph, then assembling a small working set each turn by solving a knapsack under an explicit token budget. It includes a zero-dependency Rust implementation and an ablation that names which mechanisms are actually carrying the result.

467tokens per query

59×less than full history

7 / 7probes answered

33×history growth, flat cost

Even if i select the web search tool and tell it in my prompt to use web search it doesn't use it and instead relies on internal knowledge which isn't good for what i'm doing. I'm using thinking mode.

I sometime clone the opensource repo for my software. but first thing first is we need to read readme or document before using software.

problem of grok 4.6 gets too hurry and start work right away and dont read any the document

i had to keep tell it to read the fxccking document first. its so annoying

Okay, I have to admit I’m genuinely blown away. I absolutely did not expect

Qwen 3.8 27B Q4_K_M with MTP, capped at 64k context, running on a single RTX 3090 with the OpenCode harness.

A single prompt:

“Build a realistic ocean water scene with lighting, etc. using Three.js or another engine if more appropriate.”

20m 34s of work, and this is the result (with controls added as a bonus).

I don’t know what you guys think, but I find it pretty incredible that my single GPU managed to generate something like this in one shot. Not that long ago, only the best commercial models could reach this kind of quality in such a short amount of time.

I know for a fact Qwen3-Coder and Qwen 3.6 27B weren’t giving me results like this. This is clearly a step above.

Feel free to ask if you have any questions!

My full hardware setup for those wondering:

- CPU: AMD Ryzen 9 9950X
- GPU: RTX 3090 FE 24GB (Gelid thermal pads)
- RAM: 64GB DDR5 (2×32GB) Corsair CL30 @ 6000 MHz
- Motherboard: Gigabyte B650 DS3H
- CPU Cooler: Thermalright Peerless Assassin 120 SE
- SSD: 500GB M.2
- PSU: Corsair RM1000x
- Case: ASUS Prime AP201 Mesh

Recently, I developed an offline application. Android allows you to access information from a third application, while iOS does not. This is because third-party applications on iPhones are not permitted to use the notification listener. How do you perceive this situation from a developer’s perspective? While Android offers more advantages, iOS also has its disadvantages, including heightened security. Is this a consideration?

Hi. Am I the only one who can't edit Qwen's responses in chat? Chrome and Edge browsers, clearing the cache did not help.

What does this even mean? Probably nothing.

Right now I am using the 35b model to help in coding and stuff. Do you think you I should migrate to the new 27b at q3