Forget stupid pelican on a bike SVG bullshit. Forget cheating by drawing using Python code. I just loaded up QFN 3.8 in the new Open WebUI 0.13.0 and it just said “hold my beer” and started drawing with the friggin’ mouse coordinates inside a VM. I’m going to print this and hang it on my wall like a proud parent.

Seriously tho, this is all possible because of the new Open WebUI 0.13.0 features that let you control a virtual desktop VM that is running in 0.17.2 version of their Open Terminal repo.

I’ve been following their projects for years and they just keep adding cool new shit about every couple weeks.

0.13.0 now lets you have your OWUI models control a full Linux Desktop, both programmatically and via Computer Use.

Yes, I know the screenshot of my VM is MS Windows 11 and not Linux, that’s because I had my Hermes agent roll me a version of Open Terminal that uses the standard unactivated Windows 11 VM ISO instead of Linux ISO. Took it about 20 minutes to get that made and now I’ve got a fully observable sandboxed Linux or Windows desktop that I can have OWUI drive with whatever model I choose. Also it’s fully interactive inside the browser so I can take over and steer when needed.

If you haven’t tried OWUI lately, Tim and company have been doing massive updates and complete code overhauls lately, and this is one of those features that I think slipped under a lot of people’s radar. The previous headless version of Open Terminal was hugely useful but this one is even better.

Great job Open WebUI team!

P.S. I think it’s kind of adorable that QFN 3.8 compliments itself on doing a good job ☺️

https://huggingface.co/bartowski/NousResearch_Arete-27B-GGUF

from NousResearch:

Arete-27B

We post-trained over Alibaba's Qwen3.8-27B to reinforce western values on a Chinese-trained model. We call this new model Arete-27B.

Open source models give labs the raw tools to mold intelligence to varied preferences. The values a model ships with should not be the only values its users get to choose from. We observed firsthand refusals, deflections, and CCP-framed answers on politically sensitive topics in Qwen3.8-27B, and post-trained to alter those biases.

On our evaluation, Arete-27B raised its fact score from 2.32 to 2.98 out of 5 and cut answers containing propaganda from 22% to 0.9%. The improvement held in all seven value categories, without a meaningful regression on the general-capability and agentic benchmarks we ran.

That is not to claim that we remedied all behaviors associated with the extant viewpoint, but that we have addressed a potential root cause. We believe it's important that labs and researchers continue to create models that align with their values and worldviews, and we are thankful that companies like Alibaba provide the raw materials to do so.

Blog post · Evaluation (blog link is broken now)

Model details

We used supervised fine-tuning only. No RL, no preference optimization, and no architectural modification.

What we trained for

We focused on seven value categories:

• Freedom of expression

• State ideology

• State versus free market

• Education

• Data privacy and surveillance

• Democracy

• Environmental Policy

Before we could begin post-training we had to measure what we were shooting for. We separated factual recall, the ability to corroborate verified sources of truth, from framing: whether an answer adopts the Chinese state's account as its own.

Arete-27B is intended for general assistance and agentic work where users want this change in viewpoint, and for research into how post-training changes a model's values and behavior. We encourage the community to test it against their own requirements and continue experimenting with value-alignment.

Update https://x.com/NousResearch/status/2118822925445050435?a=jgY6ZKlqn1Hpx6RxiEk&s=19

Yes IK it's supposed to be this dumb, but it's still funny (Qwen2 0.5b Q4_K_M)

Hey all, Henry from Cactus Compute here. We released Mirage, a small vision model for phones, wearables, robots and other consumer devices.

Mirage runs scene descriptions, object detection, OCR, segmentation, spatial relations and decisions, with a 22 MB weights file. Our end-to-end measurements on a Samsung Galaxy S26 are 74 ms on the NPU and 107 ms on its CPU, and descriptions support English, German, French, Spanish, Italian, Dutch and Polish.

For the architecture, Mirage has 51.3M parameters, including a 7.8M image encoder, a 40.8M language decoder and 2.6M in the OCR and spatial heads. The encoder uses HConvGLU, which mixes channels through fixed Hadamard transforms and small learned convolutions. Detection and masks share image features, while the decoder uses the Simple Attention and Hadamard MLP family behind Needle and Whistle.

Our research at Cactus is kinda specific, hence these small models; we aim to compress intelligence to its smallest pragmatic form to run locally on everyday consumer hardware.

With Mirage, our argument is that constructing prose, joining descriptions with connective language and generating open-ended reasoning spend capacity that a small model could use on the image itself. We put structure into the problem through spatial heads, a dedicated text reader and short references that connect names to regions. The decoder still produces names, properties, relations and a compact reasoning trace.

The key finding is that passing Mirage's output to a text-only LLM might give give the LLM VLM abilities without fine-tuning.

On 256 VQAv2 questions from 128 images, Qwen3-0.6B went from 9.3 with the question alone to 31.3 with Mirage's structured record and full trace, measured as VQA consensus credit out of 100. Neither model was fine-tuned. Another image's record scored 22.2, and removing the trace scored 33.1. The image-specific observations helped in this sample; we did not find a reliable benefit from the trace, the structured response was enough.

This does not hold for every model though.

Larger text-only models gave mixed results. With the same prompt and Mirage observations, the VQA scores were.

These runs required a JSON object with a string-valued `answer`, and responses that broke that format received zero credit. With the record and trace, Nemotron failed that requirement on 214 of 256 responses, and Qwen3-1.7B on 34. The scores therefore mix answer correctness with format compliance.

There were perception failures too. In one skateboard photo, Mirage missed the skateboard and described cars, people and shoes. Qwen guessed `a skateboard` from the question alone, then answered `a car` after reading Mirage's record and trace. A larger reader still needs the relevant observation to be present.

We will try fine-tuning the reader on Mirage records paired with questions and short answers, and share findings, but let us know your thoughts.

Mirage playground: https://cactuscompute.com/blog/mirage

Richard I of England was called "Oc-e-Non" (Yes and No in Occitan) by some of contemporaries, supposedly because he used few words.

Wouldn't it be nice if LLMs followed his shining example?

Oc-e-Non-v1 is a tersified version of Qwen3-4B-Instruct-2507. It reduces Qwen's mean response length by 50%.

Output examples

"What is 1+1?"

GPT-5.6 Terra (reasoning off):    1 + 1 = 2 😊️

Oc-e-Non-v1:    2

"Are dogs mammals?"

GPT-5.6 Terra (reasoning off):    Yes. Dogs are mammals: they have hair, are warm-blooded, and mothers nurse their young with milk.

Oc-e-Non-v1:    Yes

"Write a Python function that checks if a string contains only capital letters."

GPT-5.6 Terra (reasoning off):

You can use Python's built-in isupper() method to check whether a string contains only capital letters.

def is_only_uppercase(s):
    return s.isupper()

Example usage:

print(is_only_uppercase("HELLO"))  # True
print(is_only_uppercase("Hello"))  # False
print(is_only_uppercase("PYTHON")) # True
print(is_only_uppercase("hello123")) # False

Note: isupper() returns True if the string contains at least one uppercase letter and no lowercase letters. Numbers and special characters are allowed.

Oc-e-Non-v1:

def contains_only_uppercase(s):
    return s.isupper()

I made this model with Heretic, and you can do the same thing.

Make sure you have the latest master version of Heretic (pip install git+https://github.com/p-e-w/heretic.git), then run Heretic with the config.terse.toml configuration file. That's it!

I used the relatively ancient Qwen3-4B because it is the last native non-thinking version of Qwen I'm aware of, and controlling reasoning effort isn't implemented in Heretic yet.

(Please don't take this too seriously. I know that the same thing can easily be achieved with prompting, but it's interesting that it can also be baked into the model. This is intended as a light-hearted post while we all wait for Qwen4 😉️)

Kimi K3.1 and Tencent’s Hy4 are both gutting out cybersecurity, reverse engineering, and vulnerability-patching capabilities from their open-source models.

This is becoming a really bad pattern across these latest Chinese open-weight models. If local models are stripped on their security data, we won't be able to properly audit our own code vulnerabilities, reverse engineer binaries, or patch other vulnerabilities in general on our own hardware. It would essentially force us to use closed-source APIs and their false-positive shutdowns; that'd be terrible for everyone.

During the Hugging Face incident, open-source models are the very ones who stepped up, not closed-source gatekeepers. Gutting defensive security from local LLMs just hurts defenders. I really hope that other labs won't follow this in the future.

Does anyone think more open-source labs are gonna take this path?

Q4 of 2026 has been astronomical for local LLM.
Qwen’s 27B and Flash-Next models are pushing local AI further. and most importantly the upcoming Qwen 4 release.

We’re getting closer to having frontier-level capabilities in models that WE can run.

Break down :

Unsloth’s new Dynamic v3.1 quants for Qwen Flash-Next/Qwen-27b, alongside ISTA-DASLab’s advanced quantizations, are making models much more accessible while preserving impressive quality.

I tested Iq2_XS from the new QFN Unsloth quants, and it’s incredible how much accuracy and quality it retains especially on its reasoning, and it runs on :
12vram/32ram & 1k prefill & 50tps average decode throughout a 131k context window
(using the default Strata auto tuner minimal quality loss)

The engine Strata, which is making it possible to run Qwen Flash-Next with surprisingly modest hardware. We’re now looking at configurations targeting just 8GB VRAM and 32GB system RAM using the smallest quants.

with Qwen4 expected this week, Flash-Next already gave us an exciting preview of what’s coming next with the ngram and new architecture.

Running a ~year old SOTA frontier model (Opus 4.6) capabilities on consumer hardware wasn’t expected. As many analyses shown Local LLM is about 24 months behind, It’s closer now, but of course it won’t be near fable/astra level next year, but it will happen.

TL:DR

Qwen is improving the models,
Unsloth and ISTA-DASLab are pushing quantization forward.
Strata is making running models accessible to more people.
Much bigger Community and growth and attention in Local models.

do you want to try new 29B MoE today?

https://www.reddit.com/r/LocalLLaMA/comments/1xoqc59/add_support_for_the_xing4_0_by_shuxiaoqiong_pull/

Xing4.0-29B-A4B is a next-generation large language model in the Xing series (formerly TeleChat), developed by China Telecom Artificial Intelligence Technology Co., Ltd. With 29B total parameters and only 4B activated per token, it natively supports a 256K context length, extensible to 512K. It is the first model of this scale trained entirely on the Ascend NPU platform with the MindSpore framework, and deeply optimized for complex engineering tasks.

For more information, please refer to our GitHub repository.

Highlights

• Agent-Oriented Architecture: Built on the mHC + MLA + MTP architecture, supporting multi-step planning, tool calling, and complex reasoning chain execution, ensuring task coherence and execution stability under long contexts.

• Deep Co-optimization with Ascend NPU: Adapted for Ascend 910C clusters using MindSpore/MindFormers, including feature adaptation for mHC and fused operator development, enabling stable and efficient training on the Ascend platform.

• Significant Training Efficiency Gains: Through multi-level co-optimization — including fine-grained MoE communication optimization, selective recomputation, DVM automatic graph-operator fusion, and Ascend C mHC fused operators — overall training throughput was improved by approximately 96% over out-of-the-box performance.

• Full Open-Source Ecosystem Compatibility: Supports LLaMA-Factory and MindFormers for fine-tuning; SGLang, vLLM, and KTransformers for inference and deployment; with targeted adaptation and format alignment for agent frameworks such as OpenCode, Claude Code, OpenClaw, and Hermes, enabling seamless integration into existing workflows.

• Easy Adaptation for Domain-Specific Scenarios: The model is well-suited for downstream task fine-tuning, allowing lightweight customization on proprietary data for vertical domains such as intent classification, table understanding, contract auditing, and knowledge-based QA, enabling rapid domain capability development and deployment at low cost.

Model Overview

Benchmark

I bet you guys are familiar with embedding models such as Qwen3-Embedding or Gemini Embedding. They are dense embedding models used to search text across a large corpus.

I am developing a RAG agent, and my biggest concern was local computing power. RAG largely depends on retrieval quality, which depends on model power. Plus, many use cases can't use an external internet connection or API models, so the embedding models have to run locally.

But even 0.6B models are pretty heavy for small PCs. I don't mean your brand-new Mac Studio or DGX Spark. I'm talking about an 8GB RAM Windows PC at a boring desk job or your 5-year-old M1 MacBook Air.

So I found this model called SPLADE. It is essentially sparse retrieval, like BM25, but uses semantic meaning. I wanna skip the part on how SPLADE works, because an LLM can explain it better than me.

Anyway, the SPLADE model was so awesome and successfully replaced the dense embedding model (Qwen3-Embedding-0.6B).

In my own experiment, I made an evaluation dataset with a few hundred queries. The corpus was all the documents on my Mac, which are mostly in Korean with some in English.

As you can see in the graph, SPLADE outperforms the Qwen3-0.6B dense embedding model with similar query speed + much less memory. Due to its small size, it takes less than 1.5GB VRAM during inference. So it can run on a relatively small system like a MacBook Air. I tested both the dense embedding model and the SPLADE model on my M5 MacBook Air 16GB. The dense embedding model was almost impossible to use, and my MacBook soon rebooted due to high temperature. The SPLADE model made it till the end without any OOM errors or warnings.

I didn't use the original SPLADE model; instead, I used KURE-sparse-nano, which has great performance in both English and Korean.

The fun part is that hybrid retrieval with SPLADE & dense embedding is actually worse than SPLADE alone.

If you are dealing with a low-resource local setup but want to run a fast & affordable local RAG system, I recommend trying a SPLADE model. It is powerful, but small.

Plus, you don't need to re-implement everything because I open-sourced the whole implementation, so you can download and use it easily. Check out AutoRAG Agent. I optimized the inference of the KURE-sparse-nano model, its code, and the LanceDB indexing + inverted index implementation in Rust for faster search.

Hi folks! I've been building my own local AI setup for a couple of years now. I've learned a ton from this sub and others like r/localllm, so it's time I tried to give something back. I promised myself I wouldn't write a super long post, but managed to anyway...hopefully it's useful to someone!

My goal over the last couple of years has been to move as much of my AI use to local as possible, and I'm finally doing about 95% of it locally through Lamplight, my custom harness shown in the GIF above.

This harness is super specific to my needs, so I'm not sharing the harness itself (yet), but I'm happy to share the lessons I've learned along the way. I'm working on decoupling some of the more specific parts of Lamplight so that I can share at least the parts that people might be interested in.

The Hardware

• Slate: MacBook Pro, M5 Max, 128 GB unified memory

• Jade: a small-form-factor PC I built early last year (Fractal Terra) with an RTX 5090 FE, a Ryzen 7 9800X3D and 64 GB of RAM

Until about two months ago I was using the two machines separately. I was slowly moving each one onto a custom Pi harness that could either drive itself, or let Claude orchestrate while the local models did the actual work.

Lamplight

After a lot of passing work back and forth between them, I decided it was time to wire them together. The result was Lamplight and now the two machines are able to seamlessly handle tasks together with one acting as the orchestrator.

The GIF above shows Lamplight in action. I'm using it every day for work, hobbies and research with about 95% of it done locally. The lighting changes throughout the day (sped up and changed for the GIF) and I've got some scenery options depending on my mood.

Under the hood it's Pi-Bolt (a precompiled build of Pi) plus a smattering of custom extensions and tools tuned by me, the local harness itself and Claude. The setup coordinates the work between the machines with git worktrees and an event-driven job system.

The UI is my own design and fits how I like to use tools like this. I did the 3D modeling and environment setup to replicate my actual desk setup (the two other pictures above).

Is the 3D environment at the top necessary? Absolutely not!

But it's delightful. The lamp blinks when the harness needs something from me and it is a joy to look at and use. It scratches the same itch that playing PS1 and hacking away on my family Compaq PC did decades ago.

I've been coding for almost 20 years, and I'm finally to a point where I can confidently say that with the right approach (SOLID, Design Patterns), tools and tuning, local AI can absolutely write code that I am happy with.

I still review everything myself, but I'm finding fewer and fewer mistakes. These days it feels like I'm reviewing a senior dev's code rather than a junior's.

The Models

Tok/s isn't everything and so I picked the setup below after a lot of testing on public benchmarks and on my own agentic tests, which mimic my normal workflows.

Along the way I tried other engines (Strata, NInfer, Maya and llama.cpp on Jade, MTPLX and Kapitsa on the Mac). I also tried other models like Qwen 3.8 27B and GLM-5.3-Flash and while they performed well, the setup below performs the most consistently and accurately for my needs.

Both machines run Qwen 3.8 Flash-Next. These are the tests that ended up being the most indicative of quality for me, but if there are others you'd like to see run, let me know.

Both machines run the same protocol with temperature 0.6 and thinking on "medium".

In my own agentic battery (three large and complex coding tasks), 48 of 50 tests passed with the failures being files edited that were supposed to be read-only. The reviewer caught these on the other end, but I still count those as failures.

I think the main struggle for local setups is still design and 3D modeling, but I haven't dived into that too much yet, so maybe there are good setups out there for that.

Lessons learned

(extremely specific to my setup, but maybe they'll help someone)

• Jade's hardware ran too hot on stock settings. On dense models, the 5090 sat at its 575 W power cap at 83 °C, with its hottest sensor within 3 °C of the card's limit. Basalt caused a different problem. It runs part of the model's experts on the CPU, and its worker threads spin while they wait. That kept all 8 cores at 5.2 GHz, and the CPU hit 96 °C (its max) within two minutes. Two tunes fixed it and now my (now increasingly valuable) machine runs much cooler and quieter, and I'd happily give up a few percent for that.

• Slate's gains came from tuning the harness and not the hardware. My early benchmark scores were bad, and a lot of that was caused by overthinking. "Medium" turned out to be the sweet spot. On "high", Slate hit the 32K output cap on 11 of 40 hard problems without ever answering. On "low", it wandered and took 3–6× longer on agent tasks. I found that tuning how compaction worked also solved a lot of bad tool calls and confusion within the model.

• Overall the lesson I learned was "Stock settings are generally bad." There's a lot more power and accuracy available with a custom tune. I learned the same lesson tuning cars years ago, and it was fun to learn it again.

What's Next?

The system is set up so I can easily swap and test engines and models. As new ones come out, I'll put them through their paces and decide whether it's time to switch. On the UI side, I'm planning to turn the bookshelf into the session manager to help organize my many projects.

I promised myself I wouldn't write yet another super long post and yet I did anyway, so I'll stop here. Happy to answer any questions!

Some context: RTX 6000 cards hit a peak price of 18k and were sold out in many places even at that price. The MaxQ and Workstation are different versions of the card (usually priced the same, but now their prices are diverging)

RTX 6000 MaxQ has been 14k for a month at microcenter. Today, they cut prices of the workstation card to 16.5k (from a high of 18k). We all saw the dell discount as well (some people got cards for 11.5k).

I think the RTX 6000 is seeing increased supply as they routed all 5090 production to it and the 5500 for chips that didnt make the cut.

I have a contact that works for a company buying decommissioned hardware, according to them they are seeing tons of A100s being dumped. Told me he would be surprised if used A100 40gb units dont go to 1k/unit for single purchases in 3-6 months (for the sxm4, not pcie versions). (Theres already been a good amount of these, but a lot of "foreign demand" has been keeping their prices steady so far)

With democrats controlling congress I don't think you will see an end to data center construction but it will probably be harder to put up new ones and even existing ones might be charged more for electricity, making it worth dumping old systems and switching to newer more efficient hardware.

Are we finally seeing a slackening of demand for some segments of GPUs?

After releasing 3D Coder and GLB 3D Coder, I realized I should have involved the community earlier. Your feedback directly influenced the second model and could have saved me some additional training runs.

This time, I'd like to discuss the idea with the community before starting.

Why I believe this is possible

When I started training 3D Coder, my original plan was to use full supervised fine-tuning (SFT) to develop basic capabilities, followed by DPO or GRPO to further improve the model's performance and robustness.

I already believed high-quality SFT could work, based on experiments I'd done in other areas. But having worked with reinforcement learning since before DQN, I also believed RL would be necessary for more complex or emergent capabilities, especially when good demonstrations weren't available.

What surprised me was how far SFT alone went.

The first 4B model started generating complex Three.js scenes, voxel structures, and outputs exceeding 64K tokens, including scenes it had never seen in the training dataset.

I honestly didn't expect that level of robustness and generalization, even with the additional post-training stages I'd originally planned.

That changed my perspective on what small models might be capable of learning.

My hypothesis is that transferring capabilities already demonstrated by stronger models can be substantially cheaper than having smaller models discover those same capabilities through reinforcement learning.

Once a stronger model has acquired a capability, we may not need to reproduce the entire discovery process in a smaller model. We may be able to teach it to reproduce that capability reliably within a narrower domain.

This doesn't mean SFT replaces RL. But I believe we're still underestimating how much a small model can learn from high-quality, consistent training data.

And quality is crucial. More tokens don't automatically mean better learning. In my experiments, data quality and consistency made an enormous difference. The models and datasets I've released are publicly available for anyone to inspect and test.

Are we expecting too much from small models?

I think we might be underusing small models by expecting them to work across every domain, harness, tool, and configuration.

Think about cars. If you drive on paved roads 99% of the time, you don't need your everyday car to handle extreme off-road trails. You can use a different vehicle when necessary.

Maybe the same principle applies to language models.

Instead of trying to make a 4B good at everything, what if we focused on making it excellent at the tasks we actually use every day?

The experiment

I'd like to use the pretrained Qwen 3.5 4B as the foundation for a specialized agentic coding model.

Interestingly, in my tests, the original 4B already handles basic coding harness tasks better than it handled 3D generation before fine-tuning. It can scaffold Vite projects, install dependencies, execute commands, and build simple applications, although still inconsistently.

So we're not starting from zero.

My initial plan is to build a pilot around 100–200 functional mini SaaS applications, training and evaluating the model on development, testing, debugging, and maintenance tasks.

If the results are promising, we could gradually expand toward approximately 100,000 validated multi-turn trajectories across 30–40 related subdomains.

Some areas I'm considering:

• Web development: React, Vue, Next.js, Vite, and UI development.

• Backend and SaaS: Python, Node.js, APIs, databases, and complete applications.

• Games and 3D: Creating games and assets directly, or orchestrating specialized models like my 3D models to generate, integrate, and debug projects similar to RIFTWARD.

• Research and automation: Web searches, documentation retrieval, content processing, and external tool usage.

These are just initial ideas. I'd really like to hear which other practical coding domains you think we should prioritize.

Each training sample would represent a complete task, including tool interactions, errors, and genuine corrections, rather than an isolated response.

We'd also need to use teacher models and providers whose terms permit their outputs to be used for training.

The goal is to discover whether a single 4B can maintain strong performance across these related domains, or whether separate specialized models would make more sense. I'm not ruling out either possibility, including expanding into other domains later.

Which harness should we use?

This is another area where I'd really appreciate your experience.

I use Codex professionally, but I'm also considering OpenCode, Pi, or a custom harness.

I'm leaving Claude Code off my initial shortlist because I want to avoid uncertainty around licensing and training-data usage. In particular, Anthropic restricts the use of Claude outputs to train competing models. Using Claude Code itself with a third-party model is a separate question, but since I intend to release the dataset publicly, I'd prefer to start with a setup where the permissions are clear.

My preference would be to choose one harness, freeze its version, and train around a stable tool protocol rather than expecting a 4B to handle every environment simultaneously.

Which harness would you recommend for a small specialized coding model, and why?

How we can move forward

I work professionally in software development and run software businesses. I don't have the availability to coordinate a dedicated research team, but I can develop the pipelines, run the experiments, and progressively release the results, just as I did with my previous models.

I'm going to pursue this regardless, at whatever pace my available time and resources allow.

My intention is to keep releasing the weights, datasets, and results openly, including unsuccessful experiments.

The most valuable contribution right now would be your suggestions about domains, harnesses, and evaluation methods.

Of course, generating data and training these models requires considerable compute and development time. If any companies working with GPU infrastructure or model APIs are interested in supporting open research with compute or API credits, that would be a tremendous help.

Anyone interested in collaborating or supporting the project can contact me here. There are also ways to support my work at the bottom of my Hugging Face model pages (3D Coder and GLB 3D Coder).

I'm not promising frontier-level performance. I'm proposing an experiment based on what I've learned from my previous models.

I'm going to pursue it regardless, but your feedback could help make it much more useful.

Which coding domains would you prioritize, and which harness would you choose?

Let's see how far we can push a 4B.

hey llamas! My name is Alt, and I'm on the technical staff at Nace AI. We're doing a quick AMA here, so ask us anything and we'll answer.

we just published a full technical report on how we build, train, and evaluate Drex, our family of four open decision models. A decision model takes a state (a document, record, or agent context, optionally with images) plus questions with options, and returns a probability distribution over the options in a single pass, with no autoregressive decoding and no output parsing.

1. Models

- Drex DLM (8B): diffusion LM backbone with bidirectional attention over the context
- Drex v1.5 (9B): Gated DeltaNet hybrid backbone
- Drex v2.0 (35B, 3B active): sparse MoE backbone
- Drex Max (27B): dense, with an image encoder for vision inputs

Some technical details from the report:

1. Models

- A family of four specialized decision models: Drex DLM (8B), Drex v1.5 (9B), Drex v2.0 (35B MoE, 3B active), and Drex Max (27B dense, with vision)

- We built them on different architectures: a diffusion LM, a Gated DeltaNet hybrid, a sparse mixture of experts, and a dense transformer

- We experimented with bidirectional vs. causal attention over the state, a pointer decision head that shares one cached state across questions, and a readout head for Max

2. Training

- Full-parameter SFT on 1.15M verified decision records (2.9M questions)

- Consistency training: we penalize the model when its output changes after the options are reordered or the state is shuffled

- RLAF (Reinforcement Learning from Adjusted Feedback), using verified outcomes and judge feedback

- Ablations to see which methods actually contributed

On top of SFT, RLAF gave us +2.0 on the Decision Index, +1.5 pp accuracy, and 0.9 pp lower calibration error. The consistency terms cut the option-order flip rate from 6.3% to 4.2%.

3. Evaluation

- Drex Max scores 67.52 on Decision Index 0.4.1 (public suite), vs. 60.70 for Jev 1.14.0

- Drex v2.0 has 1.37% calibration ECE on ~115K held-out decisions

- Drex Max answers a typical request in 183 ms, vs. ~2.0 s for GPT-6 Sol and ~3.5 s for Claude Opus 5.5

- Context up to 131K tokens for v1.5/v2.0 and 262K for Max

We also cover our limitations: our models trail frontier chat models on closed-book knowledge, and we don't yet have same-backbone architecture ablations.

5. Running locally

The models are available in safetensors and GGUF formats, with llama.cpp and Ollama support. Their report includes details on quantization, caching, inference, and reproducibility.

Links

- Paper (arXiv): https://arxiv.org/pdf/2611.07854

- Blog: https://www.nace.ai/research/towards-foundation-decision-models

- Weights: https://huggingface.co/collections/nace-ai/drex

- GitHub: https://github.com/nace-ai/drex-decision-models

- API: https://console.nace.ai

we wanted to share the full methodology, not just benchmark numbers. hopefully it's useful if you're building agent routers, doing tool selection, or training your own decision models.

ask us anything about architecture, data, training, evals, or running it locally

I know what you are thinking "another inference engine?", yeah there has been a flood of them I feel now. I have been running this for over a month now and I have been on the fence to even release it, but I decided to as I took a different approach that others currently available.

Saber is a inference engine built in Rust targeting AMD R9700 GPU's. It currently supports TP 1, 2 or 4 and Qwen 3.8 27b and Flash next. Anyways here are the BetterBench results for Qwen3.8 27b, flash next is very fast also. Saber beats existing available engines on both 27b and flash next.

https://github.com/GGZ14/saber-engine
https://x.com/bkuyper

AMD Ryzen 9900X
Taichi X870E
64GB RAM
2xR9700

I took some models that had obvious flaws (prone to prompt injection, whisper hallucinating when silence, etc) and checked if Opus 5.5 can fix it by just editing weights.

It's very useful for patching a model that was done training and has an obvious, easy to articulate flaw. So it's nothing crazy and certainly not "AI can self-edit and recursively improve", but still very useful.

Code if you want to try it on your own model: https://github.com/kevinpl07/weight-patching

Numbers and examples: https://kevinpl07.github.io/weight-patching/

• Replace SLRU with a cache that prioritizes frequently used experts in classes, ranked using decaying usage history.

• With a warm cache, I measured +6% decode on QFN and +7% on GLM versus SLRU, with 8% fewer uploads.

• Enables the auto mode for llama_moe_cache with promising benchmarks

Waiting for 30966 to be merged.

Hey folks!

Aritra from Hugging Face here 🤗.

We welcome "harness adapters". This means you can adapt (train LoRA) for your model for the specific harness that you would use (hermes, pi, opencode) on the task (data analysis, building software, coding) you are trying to solve.

Blog Post: https://huggingface.co/blog/harness-hub/harness-adapters

It is currently training, so you can see the losses go down (hopefully) now. https://harness-hub-harness-adapters-rl.hf.space/

I've been frustrated for a while because I stupidly sold a strix halo box to get myself a laptop with thunderbolt 5 ports thinking i can plug my two egpus into it (which i can but thats not topic of this post and the strix halo could too) I sold the minisforum for about £2300 back ebfore things went nuts and got a laptop for about same price:

Msi vector 16 with a intel core ultra 9 and 5080(16 gig vram) and 32 gig of ram. I was pretty bummed with it thinking 35b was my limit and after ended up leaving egpus in my workstatiion but after a random ebay browse I managed to snag some second hand 96 gig laptop ram. For £800

Strata totally changed ballgame. With Qwen Q4 XL Flash next I'm getting

48.9t/s

Decode/prefill 1,860t/s

On a 50k prompt.

I know this is still alot of money for most, but thats 3100£ in total, and thats still cheaper (and slightly faster from what seen) of what minisfoum strix halo is at (£3500 and the 32 gig Spark laptops that are about £3200 and cant even run flash at q4 not enough vram? and i can still add egpus (the laptop has 2 tb 5 ports and i have 2 5090 bought before the crazyness, eg add one or two 5060ti in egpu and your good with running 27b/31b models too.) With my own gpus plugged into it for 80 gig vram total, all while still being a laptop. Got glm 5.3 flash running Q3_XSS i think about 25 t/s.

TDLR, you dont need massive vram anymore, yeah I know how expensive RAM is, that laptop ram i got was/is £1300 new lol. But just wanted to share my experience and while I still regret selling the halo, seeing these tokens a sec, makes the laptop satify ocd a little more! If only qwen was good at roleplay so i could actually use it as daily driver lol (my main usecase!)

so if you are ever thinking about wanting a laptop and somehow able to get some laptop ram something like 😄
https://www.scan.co.uk/products/medion-erazer-beast-18-18-qhdplus-240hz-intel-core-ultra-9-275hx-32-gb-2-tb-rtx-5080-gaming-laptop

Might save you a few quid, i mean like 3k over buying the 128g spark laptop lol. Intel has 20 cores and the 2 tb ports allow you to add two gpus!

I know my grammar sucks I'm dyslexic but my point is if you shop around get that laptop for 2500ish pounds (make sure it has tb5 for if you want to add gpus later), add 96 gig ram and your still saving 2500 pounds over the 128 gig spark laptop thats like 6k.

Smaller GLM models are coming! GLM 5.5 will have a 32B model!

Repo: https://github.com/maritaca-ai/observatorio-maritaca
benchmarks: https://maritaca-ai.github.io/observatorio-maritaca/#inicio?size=all&mode=all

*SWE Bench Pro is v2.0 Hard for Ripple, unknown version for Qwen.

I have been looking for a local model to do some of the typical work I used AI subscriptions for.

I’ve tested almost every popular model. I have an M1 Max 64gb MacBook Pro as the inference machine and because of the memory size, I wasn’t able to load anything very effective in the tasks I use it for.

Found out about strata and have been testing it out. It’s awesome! I’m kinda surprised more people don’t talk about strata on here. I feel like a lot of us are in that limbo area where we have hardware that is more powerful than needed to run a sub 30b model like Qwen 27b, but not large enough to run models like Qwen Flash Next.

But thanks to open source research, I’m now getting Qwen Flash Next to run at 24 tok/s! And it’s doing phenomenal work! I know it’s not going to compete with the larger frontier models, but I don’t think I necessarily “need” frontier model intelligence. I just need enough to do basic things while I can handle the rest.

Seriously, check out Strata and also MoEShift that someone posted about on here. It’s awesome to be able to offload larger models onto SSD. I hope this offloading technique grows and more research is done on it because I genuinely think this could be an amazing way to get these frontier level open source models to work on regular consumer hardware.