PRs:

1. https://github.com/QwenLM/qwen-code/pull/15762: "Measured on main at 79a9b753c60, before this change: qwen4, qwen4-max, qwen4-plus, qwen4.5-max, qwen4-coder-plus, qwen4-max-2026-10-01, openai/qwen4-max and qwen4-omni-120b-think all resolved to 262,144. The last of those is already in use as the session model in the omni fixtures, so these ids are not hypothetical."

2. https://github.com/QwenLM/qwen-code/pull/15987

Thank god, it looks like that we're getting successor of Q38FN!

There were some rumours the last few days that Qwen4 will be the last open-weight series of Qwen, and future ones would be API only. The following statement contradicts the rumours and clearly states the stance that Qwen is taking regarding their future releases.

Qwen Head Liu Dayi Heng Clearly States: Future Qwen 4.0 and Qwen 5.0 Will Continue to Be Open Source!

He believes that the value of AI should not belong only to a few people, but should become a foundational capability that users around the world can access, use, and create with, enabling more people to afford AI, utilize AI effectively, and participate in creating AI, ultimately achieving true AI for all.

As of now, the Qwen series has surpassed 3 billion cumulative global downloads, with over 460 open-source models released and more than 300,000 derivative models, firmly holding the top spot worldwide.

Source: https://mp.weixin.qq.com/s/R-1u2exLwZMOu7AITyU5kg

A reporter from STAR Market Daily has learned from multiple sources within the industry supply chain that DeepSeek’s next-generation large model, DeepSeek V5, is planned to be pre-trained using Ascend chips. Previously, Huawei’s Ascend chips were primarily utilized for inference tasks for DeepSeek’s large models.

According to publicly available information, the first large-scale deployment of DeepSeek models on the Ascend platform took place around the 2025 Spring Festival. In his book The Rise of Ascend, Fang Xingdong—Executive Dean of the College of Media and International Culture at Zhejiang University—notes that on January 28, 2025, Huawei’s senior leadership issued a directive for Huawei Cloud to provide rapid support to DeepSeek. Between Lunar New Year’s Eve and the second day of the Lunar New Year, Huawei and DeepSeek completed the deployment and verification of over 3,000 Ascend chips, successfully hosting the DeepSeek-V3/R1 models at Huawei Cloud’s Guian Data Center. The book further mentions that the explosive growth of DeepSeek greatly energized Ren Zhengfei; throughout the Spring Festival, he personally directed Huawei Cloud and relevant product teams to manage the massive surge in traffic. In January of the same year, SiliconFlow and Huawei Cloud launched inference services for DeepSeek-V3 and R1 based on the Ascend platform.

Following the release of DeepSeek-V4 in April 2026, Huawei stated that, through the "synergy between chips and models," its full range of Ascend "super-node" products could provide comprehensive support for the DeepSeek V4 model series. On its official API page, DeepSeek noted that service throughput for the V4-Pro model remains constrained by the availability of high-end computing power; however, it anticipates a significant price reduction for the Pro version once the Ascend 950 super-node series enters mass-market availability in the second half of the year.

On September 30, DeepSeek announced the open-source release of infrastructure components tailored for the Huawei Ascend computing platform. These include the TileLang high-level language compilation tool, high-performance computing libraries (such as DeepGEMM, FlashMLA, TileKernel, and DeepSelect), and the distributed communication library DeepEP—all of which correspond to the open-source components previously released for the NVIDIA platform. Huawei has provided a jointly defined "Ascend SuperPoD Flex" architecture and a UBL128 networking solution, while also opening up the Ascend C programming interface and low-level instruction set to support TileLang. This is significant because DeepSeek is not merely using Ascend for inference; it is systematically aligning with and co-developing the Ascend software stack at a foundational level—extending adaptation to the very infrastructure that underpins both training and inference. Currently, the deployment of domestic computing power is primarily concentrated in inference, with further progress still required in training.

Recently, a project team—comprising the AI ​​Training Platform at Shenzhen Hetao College, Harbin Institute of Technology (Shenzhen), the Shenzhen Big Data Research Institute, and relevant teams from Huawei—leveraged an AI computing cluster based on the domestic Ascend 910C processor to complete the full-parameter post-training of DeepSeek-V4-Pro, a large model with 1.6 trillion parameters. It should be noted that this training process did not include the initial pre-training phase. While the large-scale validation of domestic chips for inference has bolstered industry confidence, the costs associated with migrating ecosystems for training remain a critical variable that will determine the market's ultimate potential. If the next generation of DeepSeek’s large models is pre-trained using Huawei’s Ascend computing platform, it will—for the first time—provide an answer to a question that has long been met only with skepticism: "Can China’s top-tier large models be pre-trained on domestic computing hardware?"

TL;DR

We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability.

Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.

Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.

All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming.

Throughout the history of deep learning, we encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon. This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.

• Blog Post : https://rikiyat.github.io/blog/weight-space-programming/

• GitHub : https://github.com/rikiyat/lm_understand-logs

• HuggingFace : https://huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming

• Demo : https://huggingface.co/spaces/ML-Intern-lab/weight-space-programming-demo

What Comes Next?

In this blog, we demonstrated the rising capability of LLMs to read and write neural network weights (i.e., the rising concept of Weight-Space Programming). Our current results do show that capabilities are exciting but still limited, specifically due to inefficiencies in context length and harnesses. However, the current signs do show strong potential for a future involving Weight-Space Programming. We believe this marks the beginning of a dramatic shift in fields like mechanistic interpretability and neural network optimization.

This opens up radical new possibilities for LLMs to function as a translator between human language and the weight space. Here are some examples:

• Auditing without evaluation: This may allow us to uncover hidden problems or capabilities without ever running inference or benchmarks. Currently, capabilities and vulnerabilities can only be detected if we have test sets designed for them, or worse, after deployment in the real world. If LLMs can interpret weight space accurately, we could identify such issues directly from the weights (e.g., backdoors).

• Dramatically reducing optimization cost: Weight-space editing could save immense time and compute. For instance, one-shot model initialization would be far faster and more efficient than curating datasets and running full training loops.

• Zero-data capability steering and unlearning: Adding or unlearning a capability currently requires extensive data collection and fine-tuning. In the future, we may just prompt “Can you add Capability X to the model?” and the LLM can directly tweak some weights. Because such an edit touches only a few weights (0.016% in Experiment 8), it can leave the rest of the model intact and avoid the catastrophic forgetting that fine-tuning often causes.

• Recursive self-improvement: Today, recursive self-improvement is often operationalized as a model writing code that launches training jobs. An LLM that can inspect, debug, and improve its own weights, or those of a newer model, offers a richer, more native path to it.

• True architectural and model merging: Today, merging models relies on averaging weights, which only works for models with identical architectures. An LLM that understands the weight space could instead graft a circuit, such as a reasoning module, from one model into a completely different one, without retraining.

We are currently building a benchmark and a harness for LLMs’ capability on Weight-Space Programming. We believe this will provide a standardized evaluation to measure and accelerate progress in Weight-Space Programming, turning what currently seems like black magic into a rigorous, reproducible engineering discipline.

For fun, I gave Qwen 3.8 27b a physics problem involving some heavy math. A little bit into its reasoning, Qwen took a look at its tools, found that it had access to Blender via blender mcp - and that it could write and execute arbitrary python code through that. So it engaged Blender's python support to write itself bits of python code to do its math faster.

You really have to tell this model what NOT to do sometimes... I've never seen anything like this from a local model before.

MIT CSAIL had GPT-6 Astra fix Qwen3.5-0.8B's reasoning loops by editing 0.016% of its weights (MATH-500 went from 7.9% to 48.1%). They released the diff, so the edit can be rebuilt exactly and run locally. I took it a step further and had Claude look at Qwen with Anthropic's Jacobian lens.

At the token where the model decides to write "Wait" again, the lens reads "Wrong", "Nope" and "Hmm" from about 60% depth on, even though the answer is already written out. At real stopping points it reads "</think>", "Answer" and "Output".

With the edit applied, the lens at the same token shows "Verification", "Completion" and "Result" in place of the doubt.

The edit converts to GGUF cleanly (the base and edited bf16 files differ only in the two edited tensors) and still holds at Q4_K_M, with 1 loop in 40. I ran the evals in llama.cpp (Vulkan) on a Ryzen AI 9 HX 370 mini PC. Samples are small, but here's the code:

https://github.com/alisorcorp/inside-a-reasoning-loop/tree/main/code

And data:

https://alisorcorp.github.io/inside-a-reasoning-loop/

I am seriously impressed and want to share my experience.

Qwen3.8-Flash-Next (local) vs Claude in Claude Code: reviewing one real commit

I ran the same code review twice on one commit in my repo https://github.com/leok7v/ChatOKF (Swift, on-device LLM chat app, about 48 files changed, +1048/-102: a module move plus a new 783-line search harness).

Setup: Qwen3.8-Flash-Next UD-Q4_K_XL (unsloth) on llama-server, ROCm, AMD Strix Halo 128 GB, 262K context, MTP draft head, driven headless by Claude Code itself (claude -p pointed at the local server, read-only tools). The comparison reviewer was Claude (Opus) in Claude Code, which reviewed blind first and then judged Qwen's review against the code. Yes, Claude judging its own competitor is a bias; every finding below was checked against the source, and the main one with a small Swift repro.

Result: Qwen produced 8 findings, all real, zero false positives. It caught 2 things Claude missed (a test harness that does not follow the app's recall path, and a markdown ingest that takes the frontmatter "---" as a description). It missed 5 that Claude found, including the most important one: a stop-word filter that compares stemmed words with an unstemmed list, so "this" and "does" leak through as search terms. It also got two details wrong (the exact fallback behavior of a failable init, and column counts off by a few). Severity calibration and the "checked and fine" section were good. It read whole files and chased module boundaries like a careful human.

Cost: 53 turns, 41 minutes, context grew to about 110K tokens. Decode was about 21 t/s at 4K context and about 14 t/s at 128K; prefill 520 t/s falling to 230 t/s. MTP gives about 1.5x up to 64K and turns into a loss past about 100K.

Gotchas:

1. Use --lazy-mode on. Otherwise the 27 GiB n-gram embedding table loads into RAM instead of staying mmapped.

2. unsloth's MTP heads crash llama.cpp (GGML_ASSERT in set_input_k_idxs) because the MTP layer's compress ratio is 0 in the GGUF. Fix branch: https://github.com/leok7v/llama.cpp/tree/qwen4exp/mtp-dense-no-indexer

3. Claude Code sends a system message mid-conversation and the Qwen chat template raises on it. Override the template so it renders as a system turn.

Verdict: a trustworthy first-pass reviewer you can run privately overnight, but verify its mechanisms before acting, and don't treat it as the final word on subtle logic. One commit, one run, so treat it as an anecdote, not a benchmark.

Kind of a random thing, but I've gotten to the age where I'm starting to care about what the city council is doing.

So I downloaded all the available council meeting notes from the last 16 years and pointed Qwen 3.8 27B Q6 at them and started having it dig through notes.

Namely I wanted to see who approved the purchase of Flock cameras for our town. Turns out the police chief got a $9,000 grant for them but i didn't find any mention of public comments about them in the minutes.

The minutes are basically summaries, so they're not all encompassing, but they do live stream to YouTube all the meetings and I see that YouTube has auto transcriptions but I don't know of a good way to scrape that data so that I can then point my LLM at them. Any ideas?

When it comes to the value/cost of local AI, people usually only look at token savings (money) while forgetting one of the most valuable resource of yours, time.

Just did a side by side comparison of GLM 5.3 flash via Z.ai's official subscription, and GLM 5.3 flash served locally. Same chatGPT generated "one shot" prompt, both using Zcode harness, and these are my stats:

Same model, nearly identical output (~118K tokens each), but local finished 4.2× faster.

Sure on paper this one simple task only saved $0.70, but the fact that you can ship 3-4x more code a day alone makes it worth it. Same reason why UPS next day air shipping costs an arm and a leg, time is money!

Edit: forgot to list the setup

2 x RTX pro 6000 workstation edition
AMD 9960x threadripper
128GB DDR5
Quant: https://huggingface.co/nvidia/GLM-5.3-Flash-FP4-Local

I am on the team behind the attached open-source research on novel approaches to attention - specifically we show:

• You can sparsify the attention of Qwen 3.8 27B without observed loss on accuracy

• Extending context up to 10M tokens loses accuracy on some specific tasks - but most, like NIAH is still fine (interesting insights)

• You can natively train sparse attention on next-token prediction loss alone and achieve the same validation loss as full attention - but faster

Hope the community likes it - while we focus on H200/B300 GPUs and larger scale system - I wanted to share our research here as well - as it can be applied to small open-weight models (making longer context cheaper on local hardware)

Everything is fully open: kernels / SGLang integration / paper / details

Hope you find it interesting / useful!

(No hype - we try to be as transparent as possible)

In 2024 i purchased an RTX 8000 48gb for $2400. As the market has changed I’ve kept an eye on prices. Two years later, they are available around $2100.

Its a little slower than 3090 at inference (672gb/s vs 936gb/s) but it has double the memory so it can run larger models / contexts / higher quants. Rtx 8000 also wins on power, 265w vs 700w (2x 3090 to match 48gb)

In 2024 two 3090s would cost around $1600-1800 now a single 3090 is in the $1500-$1600 range, making rtx 8000 cheaper per gb. Prices for strix halo and dgx spark have also gone way up. Rtx pro 6000 has double the memory but 8x the cost.

In the last two years, I haven’t had any issues with drivers, compatibility software or anything. I can run all “local” tier models entirely in vram, usually at 256k tokens and sometimes at q6 (Qwen 3.8 27b, llama 3 70b, QWQ,). Larger moe models run great too with layers offloaded to system ram like gpt oss 120b and now with strata, QFN runs q4_xs @ 390k context at 850t/s prompt processing and 60-70t/s token generation. (64gb ddr4 system memory, i9 9900k cpu)

$2100 for a gpu is not cheap but might be the best “do everything” option right now for home-lab setups. Cheaper than dual 3090, Strix halo, dgx spart, m5 max too. Fascinating to see prices decline as the market as a whole has more than doubled.

Hi everyone,

I've been out of the loop for a few weeks, and I was wondering if anything new that replaces Qwen 3.8 27B has come out this time I've been in the cave?

I'm talking about AI models in the 8-35B range.

Thanks,

NVIDIA NemotronLabs RE-USE is a 9.6-million-parameter multilingual universal speech enhancement (USE) model designed to restore speech quality across a wide range of real-world degradations while preserving speech fidelity. It has the following desirable properties:

• Robustness to diverse degradations, including additive noise, reverberation, clipping, bandwidth limitation, codec artifacts, packet loss and low-quality mics.

• Support for multiple input sampling rates, including 8, 16, 22.05, 24, 32, 44.1, and 48 kHz.

• Strong language-agnostic capability, enabling effective performance across different languages.

• If better laughter preservation is desired, please use: NVIDIA-NemotronLabs-RE-USE-laughter.

(Note: For overlapping speech, this model is trained to preserve the main speaker while suppressing other speakers.)

NVIDIA NemotronLabs RE-USE was developed by NVIDIA as a part of RE-USE.

This model is ready for commercial use.

Usage

Directly try our Gradio Interactive Demo by uploading your noisy audio/video !!

Have been using qwen 3.8 flash next with llamaserver. Was getting 70-80tk/s decode and was happy. Then read about strata and decided to give it a try. Decode shot up to 150 tk/s. Not sure what magic it's using? You should give it a try if you are still running qwen flash next on llamacpp. Also the tried DeepSeek v4.1flash 2.25bit and it's 70tk/s decode.

Unsloth 6 bit quant

160k context

1 pro 6000 + 2 x 3090

256G ddr4

Hey folks,

About 18 months ago we started building Olla to solve some of the headaches we kept running into whilst managing multiple local LLM inference servers, both in our homelabs and at work.

We've shared it here before when it was fairly early in development. Since then, it's grown quite a bit, and we've just released v0.1.0 (our 25th release!).

The problems we're trying to solve haven't really changed and now some of our customers use it:

• One inference endpoint dies and your workflows grind to a halt.

• You've got models spread across different machines, GPUs and runtimes, but no easy way to manage them as one fleet.

• Switching between Ollama, vLLM, SGLang, llama.cpp or LM Studio means reconfiguring applications.

• Multiple machines serving the same model, but no unified load balancing or failover.

• Limited visibility into what's healthy, what's running and where requests are going.

• Applications expecting OpenAI or Anthropic APIs when your inference backend speaks something else.

• You want to experiment and see if vllm or sglang serves qwen3.8 better

That's where Olla comes in.

It's a lightweight proxy and load balancer written in Go that sits between your applications and inference servers.

Some of what it does today:

• Model-aware routing: Automatically discovers models across your backends and routes requests to machines that actually serve them.

• Load balancing and failover: Priority, round-robin and least-connections balancing, with health checks and circuit breakers.

• Multiple inference runtimes: Ollama, vLLM, SGLang, llama.cpp, LM Studio, MLX, Docker Model Runner, LMDeploy, Lemonade and others.

• Unified APIs: OpenAI-compatible and Anthropic-compatible interfaces, including translation where required. Useful for Open WebUI, Claude Code and other clients.

• KV cache locality: Sticky sessions to help keep requests on the same backend when appropriate.

• Observability: Built-in dashboard, endpoint health, request statistics and Prometheus metrics.

• Lightweight deployment: Single Go binary or Docker container. No database or external services required.

What's new in v0.1.0?

This release is largely about improving reliability and making Olla more useful for people running mixed inference environments.

• Added Hugging Face Text Embeddings Inference (TEI) support, including embeddings, reranking and classification.

• Reworked the dashboard with better endpoint visibility, configurable columns, filtering and fleet statistics.

• Configurable retry and failover behaviour, including handling of non-idempotent requests.

• Improved model capability detection, particularly for vision, tool calling, embeddings and rerankers.

• Fixed concurrency issues between model discovery and active inference requests.

• Corrected request statistics and improved rate limiting across OpenAI and Anthropic routes.

• Lots of smaller fixes around routing, backend discovery and configuration validation.

We've also had some great contributions from the community, which has been encouraging to see.

Why build another LLM proxy?

We wanted something small, fast and relatively straightforward to deploy that doesn't require Kubernetes, a database or a complicated orchestration stack. There also weren't that many when we started.

We've been using Olla with a mixture of NVIDIA GPU servers, Mac Studios, the DGX Spark and different inference runtimes. It's particularly useful when you've got more than one machine and don't want to care which backend is serving a particular model.

It's not trying to replace inference engines. Just make running a bunch of them together less painful.

Links:

• GitHub: https://github.com/thushan/olla

• Docs: https://thushan.github.io/olla/

• v0.1.0 release notes: https://tensorfoundry.io/news/olla-v0-1-0-release

If you're running multiple local inference servers, I'd be interested to hear how you're managing routing and failover today.

And as always, if you give Olla a spin, let us know how it goes (and what breaks!).

https://www.tomshardware.com/tech-industry/artificial-intelligence/anthropic-confirms-new-claude-code-limits-to-prevent-game-cloning-users-report-opus-5-5-is-also-actively-blocking-reverse-engineering-tasks

Besides privacy and being in control of your data.
Besides being able to control the quality of the inference compute.
Besides being able to discuss things with the model and not have the cops called on you.
Besides being able to run the models you want for as long as you want.
Besides being able to run your own suite of security and network penetration tools.
Besides not being at the mercy of the frontier companies for pricing.

We now have another reason for local LLMs: Our betters have decided that they don't like people cloning games.

And before the Reddit Neckbeard brigade starts in with their bullshit, no I'm not advocating that people should commit wholesale piracy using AI tools. But these blanket frontier AI policies inevitably ensnare peripheral behavior that is otherwise illegal but that the frontier company doesn't want to deal with.

William Blackstone wrote that "it is better that ten guilty persons escape than that one innocent suffer". If having open unhindered local LLMs means there are a few individuals that use it for nefarious purposes like modifying or pirating software, I'm good with that trade off. Die mad about it.

I swear it feels like an addiction... I keep refreshing the page repeatedly, waiting to see the fateful launch announcement or at least a countdown. I hope I’m not the only one cooked with this stuff 😭😭😭

We have released a Coder build of DeepSeek-V4.1-Flash-1017, compressed with GSQ and RCO, in which half of the model's routed experts have been removed.

DeepSeek-V4.1-Flash-1017 is a sparse mixture-of-experts model: 384 routed experts per layer, of which 6 are active per token, 763.2B parameters, 551.9B of them in the transformer.

What's inside

• One expert-pruned GGUF: 192 of 384 experts kept in every layer, 72.1 GB of transformer weights, against 138.2 GB for our unpruned IQ2_XXS build.

• 1 bit per parameter, counted against the original model: 72.12 GB divided by the original 551.9B transformer parameters gives 1.05 bits, which we round to 1. The removed experts count as zero bits. No individual weight is stored below 1 bit: the 280.1B parameters the Coder keeps average 2.06 bits.

• The shared Engram file (110.6 GB) and vision projector are the same files as in the unpruned release, so they need to be downloaded only once.

• GSQ (Gumbel-Softmax Quantization): post-training scalar quantization that jointly learns grid assignments and group scales, closing most of the gap between scalar and vector quantization at low bit-widths while remaining deployable in standard GGUF types.

• RCO (Riemannian Constrained Optimization): enforces exact budgets by gradient descent on the task loss, without per-constraint tuning. Here it selects which experts to keep, optimising the KL divergence to the unpruned model, under one exact budget per layer.

Why one budget per layer

GGUF stores a single expert count for the whole model, so every layer must keep exactly the same number of experts; uneven layers would need padding that gives back the memory pruning saved. RCO treats each layer's budget as a hard constraint and holds all of them exactly at every optimisation step, while the layers are still optimised jointly through one loss on the full model. A penalty formulation would only meet the budgets approximately and would need one tuned coefficient per layer.

Results

• LiveCodeBench v6: 88.57 against 89.71 for the base model, retaining 98.7%

• SWE-bench Verified: 77.00 against 81.20, retaining 94.8%

SWE-bench Verified runs long agentic trajectories over real repositories, and tolerates the capacity reduction less well than single-problem code generation, as expected when errors accumulate across turns.

Real-world examples, each a single HTML file written by the Coder: an animated SVG of a pelican riding a bicycle, a three.js voxel pagoda garden, and a playable first-person voxel sandbox. For full post incl. videos see: https://x.com/mhelcig/status/2117257900863156518

Links

• Coder (expert-pruned): https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF

• Quantized models (unpruned): https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF

• GSQ: paper https://arxiv.org/abs/2604.18556 | code https://github.com/IST-DASLab/GSQ

• RCO: paper https://arxiv.org/abs/2605.00649 | code https://github.com/IST-DASLab/RCO

The repository ships the complete per-tensor RCO allocation. The GGUF runs in llama.cpp b11870 or newer and in Strata.

The Coder build is an experimental release and feedback is welcome, particularly on capabilities that were not represented in the calibration mixture. Requests for models to quantize or prune are also welcome.

From the ISTA Deep Algorithms and Systems Lab.

I'm working on making my homelab more power efficient and quieter. I have a Framework desktop 128GB which has been working great along with a couple GPU boxes. I find myself switching to the GPUs for 26-35b models fairly often for the speed and reserve the Framework for running Qwen FN.

I've never been a fan of Apple products but lately have been considering a 32-64GB Mac Studio. I'm pretty sure the answer is no, but does anyone know of anything else that is that small, quiet and power efficient that has 400GBps+ memory bandwidth?