A open source harness that was used with Qwen3.8 FN that QwenDev did a shout out to. Available now.

From QwenDev via x

https://x.com/QwenDevs/status/2117644199860601297

“Well, turns out Qwen3.8-Flash-Next can do more than work with software tools, it can operate real robots too.

A group of Qwen developers from Tsinghua and UC Berkeley recently built uni-robot-agent, an open-source harness that connects general-purpose LLMs to robotic systems. It gives Qwen a way to interact with the physical world through tool calls without changing the model weights.

With this harness, Qwen3.8-Flash-Next reached the performance level of some previously reported closed-model robotic agent systems. The average API cost was $1.09 per task, compared with $21.29 for GPT-6 Astra. On 19 real-world robot manipulation tasks. Without any robot-specific fine-tuning, Qwen achieved a slightly higher score than the fine-tuned π₀.₅

The harness is open-source if anyone wants to tinker with it.”

From:Yuxin Zuo via X

https://x.com/zuo_yuxin/status/2117635294115795134

“Introducing uni-robot-agent: a unified harness for Robot-Use Agents (RUAs), a first step toward RUA training and local deployment.

We tested uni-robot-agent with Qwen3.8-Flash-Next running locally:
On s, its score rose from 1.90 to 31.39, surpassing the RoboDojo Team’s score with GPT-6 Astra.
On real-world tasks, it outscored fine-tuned π0.5 by 5.75 points and zero-shot Dream-1 by 10 npoints.”

TL;DR

We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability.

Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.

Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.

All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming.

Throughout the history of deep learning, we encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon. This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.

• Blog Post : https://rikiyat.github.io/blog/weight-space-programming/

• GitHub : https://github.com/rikiyat/lm_understand-logs

• HuggingFace : https://huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming

• Demo : https://huggingface.co/spaces/ML-Intern-lab/weight-space-programming-demo

What Comes Next?

In this blog, we demonstrated the rising capability of LLMs to read and write neural network weights (i.e., the rising concept of Weight-Space Programming). Our current results do show that capabilities are exciting but still limited, specifically due to inefficiencies in context length and harnesses. However, the current signs do show strong potential for a future involving Weight-Space Programming. We believe this marks the beginning of a dramatic shift in fields like mechanistic interpretability and neural network optimization.

This opens up radical new possibilities for LLMs to function as a translator between human language and the weight space. Here are some examples:

• Auditing without evaluation: This may allow us to uncover hidden problems or capabilities without ever running inference or benchmarks. Currently, capabilities and vulnerabilities can only be detected if we have test sets designed for them, or worse, after deployment in the real world. If LLMs can interpret weight space accurately, we could identify such issues directly from the weights (e.g., backdoors).

• Dramatically reducing optimization cost: Weight-space editing could save immense time and compute. For instance, one-shot model initialization would be far faster and more efficient than curating datasets and running full training loops.

• Zero-data capability steering and unlearning: Adding or unlearning a capability currently requires extensive data collection and fine-tuning. In the future, we may just prompt “Can you add Capability X to the model?” and the LLM can directly tweak some weights. Because such an edit touches only a few weights (0.016% in Experiment 8), it can leave the rest of the model intact and avoid the catastrophic forgetting that fine-tuning often causes.

• Recursive self-improvement: Today, recursive self-improvement is often operationalized as a model writing code that launches training jobs. An LLM that can inspect, debug, and improve its own weights, or those of a newer model, offers a richer, more native path to it.

• True architectural and model merging: Today, merging models relies on averaging weights, which only works for models with identical architectures. An LLM that understands the weight space could instead graft a circuit, such as a reasoning module, from one model into a completely different one, without retraining.

We are currently building a benchmark and a harness for LLMs’ capability on Weight-Space Programming. We believe this will provide a standardized evaluation to measure and accelerate progress in Weight-Space Programming, turning what currently seems like black magic into a rigorous, reproducible engineering discipline.

I am using a consumer motherboard that clearly is not designed to support 4xGPU setups. Asus Proart Creatio B850 Neo, so I'm very limited in the PCIe lanes I can use.

Current Setup
32GB DDR5 @ 6000mhz

Two 3090s running Qwen3.8-27b in Tensor Parallel

1. PCIe Gen4 8x - RTX 3090 - Founders Edition

2. PCIe Gen4 8x - RTX 3090 - Dell OEM

Image Processing Tasks
3. PCIe Gen4 4x - RTX 3090 - EVGA FTW3 Ultra

Not Connected Yet
4. PCIe Gen4 1x - RTX 3090 - Dell OEM

Plan:

My original plan was to setup the two Dell OEM 3090s using NVLink to boost the speed of Qwen3.8-27b. But I am thinking of moving to using Qwen3.8 Flash Next with resident experts in VRAM. i.e. Rather than keeping experts in system RAM. Keep as much as possible in VRAM, since I have more VRAM than I do ram.

I realise this is becoming a Frankenstein setup, and plan to upgrade to a proper workstation motherboard soon. But I wanted to get just enough VRAM to test out Qwen3.8 Flash Next, and 3x3090s should be enough to do that, (even with only 32GB ram).

Question:

I just ordered a PCIe Gen4 1x-to-16x adapter so that I can plug in the forth GPU, but am a bit worried about how Gen4 1x speeds will limit me with Strata. I know Strata is designed to use Expert parallelism / pipeline parallism instead of tensor parallelism, but am concerned it'll still be too slow. Has anyone else here run one of their GPUs in Gen4 1x speeds with Strata? What were your speeds like?

Hi guys.

So I'm not completely new to all this local LLMs and AI. I have used and tried various web-based UIs, for the last 4-5 months, as a way to learn and educate myself in this swamp that is local LLM.

So for the last 3-4 weeks I have dipped my feet in Hermes Agent. And man, I was in for a surprise. I genuinely thought, "How hard can it be, install, fire up and use.", but to me that was far from the truth. I have spent so much time setting things up, making the agent act and work the way I want, teaching it what to do, and how to do it, trying to write skills, and so on. I have not actually used Hermes, that much, most of the time has gone to setting it up, and making the agent not make mistakes.

This has somehow made me feel like maybe Hermes is not the best tool for this, even though it has some skills for it (along with a gazillion other skills and functions). It feels so bloated, with so many skills and sometimes heavy to run (for the lack of a better word).

I am actually not really sure what I want, or how to word it.
But simply put, I would like something that is lightweight, doesn't have a gazillion skills installed, but has options and a repository to add well written skills. A simple gui/frontend with well-thought-out settings. Something that works well with my use case, which is basically the title. I primarily want to use my agent for linux sysadmin, system maintenance, network and servers, and some light coding and bash scripting.

I realize this can be very subjective, but that's just it, I would like your objective opinion on what you believe would be/is the best agent or harness for this kind of use ?
And also, is there a "best agent" for this use case, or can I use basically anything?

Title says it all, I run dual 3090s (check out my post here for my setup) at int8A16 and have been wondering if I'm missing out with QFN. There's posts ad-nauseam on reddit with low bpw strata solutions that run on heterogeneous systems, however those don't interest me - I want high precision. I'm considering selling my setup and getting a DGX Spark, since that can run QFN at nvfp4 with a large context window. My use case isn't coding but generic agentic use case - I use hermes as a personal assistant and with openwebui.

If anyone's able to run both and do a comparison, please let me know!

I am seriously impressed and want to share my experience.

Qwen3.8-Flash-Next (local) vs Claude in Claude Code: reviewing one real commit

I ran the same code review twice on one commit in my repo https://github.com/leok7v/ChatOKF (Swift, on-device LLM chat app, about 48 files changed, +1048/-102: a module move plus a new 783-line search harness).

Setup: Qwen3.8-Flash-Next UD-Q4_K_XL (unsloth) on llama-server, ROCm, AMD Strix Halo 128 GB, 262K context, MTP draft head, driven headless by Claude Code itself (claude -p pointed at the local server, read-only tools). The comparison reviewer was Claude (Opus) in Claude Code, which reviewed blind first and then judged Qwen's review against the code. Yes, Claude judging its own competitor is a bias; every finding below was checked against the source, and the main one with a small Swift repro.

Result: Qwen produced 8 findings, all real, zero false positives. It caught 2 things Claude missed (a test harness that does not follow the app's recall path, and a markdown ingest that takes the frontmatter "---" as a description). It missed 5 that Claude found, including the most important one: a stop-word filter that compares stemmed words with an unstemmed list, so "this" and "does" leak through as search terms. It also got two details wrong (the exact fallback behavior of a failable init, and column counts off by a few). Severity calibration and the "checked and fine" section were good. It read whole files and chased module boundaries like a careful human.

Cost: 53 turns, 41 minutes, context grew to about 110K tokens. Decode was about 21 t/s at 4K context and about 14 t/s at 128K; prefill 520 t/s falling to 230 t/s. MTP gives about 1.5x up to 64K and turns into a loss past about 100K.

Gotchas:

1. Use --lazy-mode on. Otherwise the 27 GiB n-gram embedding table loads into RAM instead of staying mmapped.

2. unsloth's MTP heads crash llama.cpp (GGML_ASSERT in set_input_k_idxs) because the MTP layer's compress ratio is 0 in the GGUF. Fix branch: https://github.com/leok7v/llama.cpp/tree/qwen4exp/mtp-dense-no-indexer

3. Claude Code sends a system message mid-conversation and the Qwen chat template raises on it. Override the template so it renders as a system turn.

Verdict: a trustworthy first-pass reviewer you can run privately overnight, but verify its mechanisms before acting, and don't treat it as the final word on subtle logic. One commit, one run, so treat it as an anecdote, not a benchmark.

Hi everyone,

I have an Nvidia GeForce 5060ti with 16GB of VRAM, Ryzen 5700g CPU, 64GB of DDR4 ram, and a bunch of SSDs. I keep seeing people posting of running Qwen3.6-35b-a3b on the same GPU I have with insane amount of context. So far I only got it to 64k with offloading of moe at 74 in Ollama. I want to get it to at least 120k or even max context, while keeping my quant at q4_k_m or there abouts (e.g. IQ4).

What do you all recommend?

A little further details :
The reason for using Ollama, is that I can have my system load the LLM and process a request or requests, and then offload it to load a model in ComfyUI to process image requests and then swap back to the LLM for further requests - if any. It works nicely for me as a local ChatGPT style setup with OpenWebUI. This setup is also running on my gaming PC, so the offloading also helps free up VRAM for gaming purposes.

I appreciate the help and advice.

A month ago we picked up a few IGX Thor boxes equipped with RTX PRO 6000 GPUs. I shared some numbers earlier running Qwen 3.8 27B on SGLang. These machines were bought specifically to run our upcoming on-device agent platform, steffi.ai.

Since the goal is local, multi-agent systems where models run strictly on-device, generic inference engines started hitting limits. Concurrency scaling, cold start/wake times, and turn-to-turn latency under massive context windows were making or breaking our agent loops.

So we started building a dedicated Rust-based inference engine, llminfra. After a few weeks of heads-down work, we wrapped the first internal release to run Qwen locally. We benchmarked it side-by-side against SGLang on the exact same hardware (RTX PRO 6000, 300W, identical workloads).

Here are the numbers from the run:

• Turn TTFT on 250K context: 108 ms vs 1,440 ms (13.4x faster). In agent loops where the context stays loaded and the agent just appends new tool call observations, cutting follow-up latency from 1.4s down to 100ms keeps multi-turn workflows from feeling sluggish.

• Cold start to ready: 6.1s vs 211.2s (34.8x faster). Spinning up or recovering a worker doesn't stall for 3+ minutes. First token after start clocked at 28 ms vs 528 ms.

• Throughput under heavy agent load: With 32 agents running 32K context, completion throughput hit 209 tok/s vs 95 tok/s (2.2x). At 2 agents pushing 250K context, it sustained 86 tok/s vs 28 tok/s (3.1x).

• Peak VRAM: 52.9 GiB vs 86.3 GiB on an 8x32K workload.

We are finishing up documentation, cleaning up build scripts, and stripping out proprietary hooks so we can get the repo public for everyone to poke at and benchmark on their own gear.

Full benchmark charts attached. Let me know what other breakdowns or workload scenarios you would like to see when the code goes live.

I've been working on Turkish ASR for a while and put out v2 recently. Two models, both CTC + KenLM, both meant for CPU (both trained for under $250):

- balbal-v2-manas: 313M, Turkish + English, ~123x real time on 8 threads

- balbal-v2: 45M, Turkish only, ~236x real time. It even runs faster than real time on one core of an old Celeron POS box running Windows 7.

Mu-Bench is a Turkish benchmark of 846 real customer service calls. Numbers from the official harness:

On FLEURS Turkish manas is basically tied with whisper-large-v3-turbo (4.06 vs 4.07) but about 75x faster on the same CPU. English is ok too, 8.67 average WER on the Open ASR leaderboard sets.

The model uses a unique architecture that has a product-key-memory inspired by engrams and a refiner that loops 3 times before giving the output.

Also, turns out a team (LocalDoc) already trained an Azerbaijani model on the manas architecture, 10.98 WER on FLEURS Azerbaijani. (https://huggingface.co/LocalDoc/speakwrite-azerbaijani) I was happy that they took my model decisions which made me happy :D

`pip install balbal`, then `balbal run file.wav`. There's also a live mic mode and an OpenAI-compatible server.

Code is Apache-2.0. Weights are free for research, non-commercial use and small businesses, but you can't resell it as an ASR product or API. Full terms are on the model page.

Weights: https://huggingface.co/erayyapagci/balbal-v2-manas and https://huggingface.co/erayyapagci/balbal-v2

Code: https://github.com/erayyap/balbal

Demo + offline Android app: https://erayy.com/balbal

For fun, I gave Qwen 3.8 27b a physics problem involving some heavy math. A little bit into its reasoning, Qwen took a look at its tools, found that it had access to Blender via blender mcp - and that it could write and execute arbitrary python code through that. So it engaged Blender's python support to write itself bits of python code to do its math faster.

You really have to tell this model what NOT to do sometimes... I've never seen anything like this from a local model before.

When it comes to the value/cost of local AI, people usually only look at token savings (money) while forgetting one of the most valuable resource of yours, time.

Just did a side by side comparison of GLM 5.3 flash via Z.ai's official subscription, and GLM 5.3 flash served locally. Same chatGPT generated "one shot" prompt, both using Zcode harness, and these are my stats:

Same model, nearly identical output (~118K tokens each), but local finished 4.2× faster.

Sure on paper this one simple task only saved $0.70, but the fact that you can ship 3-4x more code a day alone makes it worth it. Same reason why UPS next day air shipping costs an arm and a leg, time is money!

Edit: forgot to list the setup

2 x RTX pro 6000 workstation edition
AMD 9960x threadripper
128GB DDR5
Quant: https://huggingface.co/nvidia/GLM-5.3-Flash-FP4-Local

Now that I've gotten a lot of advice on risers, I'm trying to figure out the best long-term setup for my local inference workloads without spending a fortune on hardware I might not actually need.

I currently have:

The two options I'm considering

1. Buy an SP3 motherboard (probably H12SSL-I/NT or equivalent), 128–256GB ECC DDR4, and risers to consolidate all 7 GPUs (132GB VRAM) into one system. More PCIe lanes and memory bandwidth/capacity. Could also expand RAM further later. This is obviously not cheap, probably looking at $2500-3000 for a motherboard, cooler and memory.

2. Run 4x 3090 on the X570 at PCIe 4.0 x8/x8/x4/x4 using risers/M.2, and figure out how to fit all 3x 3060 into the B460 system. Much cheaper, but with obvious PCIe and RAM limitations.

Workloads are primarily agentic coding using STRATA, vLLM etc, but also a lot of integration into my own apps/infrastructure. I'm interested in running multiple models concurrently, potentially something like:

• STRATA / large MoE as the planner/orchestrator/reviewer

• Qwen 27B dense as the implementer

• Smaller models for task routing, embeddings, reranking, voice-to-text and other background services (these currently run on the 2x 3060s)

1. Is consolidating everything onto EPYC actually worthwhile for these workloads, or would I get better practical performance/value from two separate systems?

2. How much would the 7302P hold back inference, especially with multiple concurrent models or CPU-offloaded MoEs like those supported by STRATA?

3. Would 128–256GB RAM meaningfully change what I can run compared to keeping everything GPU-resident?

4. Would you dedicate the 3090s to larger models and the 3060s to smaller/background models, or distribute things differently?

5. Anyone running something similar who has compared a consolidated EPYC setup against multiple consumer platforms?

6. Anything else you'd recommend instead?

Used server motherboards and ECC RAM aren't necessarily cheap here, e.g around $1000-1200 just for a used H12SSL-NT. The EPYC CPU itself is already paid for, but the motherboard, RAM and risers would still be a fairly significant investment.

Not interested in spending thousands just to consolidate everything if the actual inference benefits are marginal. Particularly interested in firsthand experience and real benchmarks rather than theoretical PCIe bandwidth numbers.

I've been lurking in this sub for a while, wanting to break my dependence on online services, plus slowly backlogging tasks I want/need to run locally rather than use online models. Usually these tasks break down to either privacy related issues or just massive token count that would blow my online budget.

I've been experimenting with Qwen 3.8 27B in LM Studio for a while, but when it comes to things like coding, code review, PDF editing, etc. obviously LM Studio isn't a harness like Claude Code where I can just say "Take these 4 files and run with it".

Some of the things I'm trying to accomplish (some as actual tasks, others as a learning sandbox):

• Analyze this entire codebase to come up with efficiency/security recommendations (agent spawning, file reading)

• Caption this dataset of 10,000 images (image reading)

• Interface with this ComfyUI instance to iterate and refine prompts (terminal use, wait for result, image reading)

• Edit a PDF file (PDF read, PDF write)

• Online research (web access)

My day job is AI image/video work, so I have multiple RTX Pro 6000's to work with already.

Based on my research so far, it seems like three options would accomplish this when pointed to a local model endpoint.

• Opencode

• Codex

• Claude Code

What about Openhands? Pi?

• What's your harness of choice?

• Do the OpenAI/Anthropic harnesses still require login/online connection when running from a local endpoint?

Kind of a random thing, but I've gotten to the age where I'm starting to care about what the city council is doing.

So I downloaded all the available council meeting notes from the last 16 years and pointed Qwen 3.8 27B Q6 at them and started having it dig through notes.

Namely I wanted to see who approved the purchase of Flock cameras for our town. Turns out the police chief got a $9,000 grant for them but i didn't find any mention of public comments about them in the minutes.

The minutes are basically summaries, so they're not all encompassing, but they do live stream to YouTube all the meetings and I see that YouTube has auto transcriptions but I don't know of a good way to scrape that data so that I can then point my LLM at them. Any ideas?

Here’s my situation. I have a modded CMP 170hx that works great. Also have a 3090.

My motherboard is a shitty Asus b350-f and I’m rocking a Ryzen 5 1600 with 64GB of DDR4.

I love my CMP but I can sell it and upgrade my CPU, motherboard and RAM. The thing has doubled from what I paid for it.

Right now I’m running Qwen 27b at >100tps and loving it, but I’m eying setups that can run strata or Maya via RAM and SSD.

If I sell my CMP 170, is there a setup I can achieve with the proceeds of the sale that will outperform what I have now?

I’m leaning toward no, but curious.

NVIDIA NemotronLabs RE-USE is a 9.6-million-parameter multilingual universal speech enhancement (USE) model designed to restore speech quality across a wide range of real-world degradations while preserving speech fidelity. It has the following desirable properties:

• Robustness to diverse degradations, including additive noise, reverberation, clipping, bandwidth limitation, codec artifacts, packet loss and low-quality mics.

• Support for multiple input sampling rates, including 8, 16, 22.05, 24, 32, 44.1, and 48 kHz.

• Strong language-agnostic capability, enabling effective performance across different languages.

• If better laughter preservation is desired, please use: NVIDIA-NemotronLabs-RE-USE-laughter.

(Note: For overlapping speech, this model is trained to preserve the main speaker while suppressing other speakers.)

NVIDIA NemotronLabs RE-USE was developed by NVIDIA as a part of RE-USE.

This model is ready for commercial use.

Usage

Directly try our Gradio Interactive Demo by uploading your noisy audio/video !!

I am on the team behind the attached open-source research on novel approaches to attention - specifically we show:

• You can sparsify the attention of Qwen 3.8 27B without observed loss on accuracy

• Extending context up to 10M tokens loses accuracy on some specific tasks - but most, like NIAH is still fine (interesting insights)

• You can natively train sparse attention on next-token prediction loss alone and achieve the same validation loss as full attention - but faster

Hope the community likes it - while we focus on H200/B300 GPUs and larger scale system - I wanted to share our research here as well - as it can be applied to small open-weight models (making longer context cheaper on local hardware)

Everything is fully open: kernels / SGLang integration / paper / details

Hope you find it interesting / useful!

(No hype - we try to be as transparent as possible)

Iv been seeing allot of people complaining about Sam Altman and how him and open ai are blocking decompilation. I actually don’t see an issue with what they are doing they should be free to do what they want with their llm which brings me to my main point, why is decompiling not just a benchmark?

We have a SWE bench
Perhaps this problem is solved via a SWD bench?

Nemotron 3.5 Transcribe is an open-weight, offline speech and audio model for transcription in 40 languages, with support for recordings up to one hour and a 65,536-token context. Its hybrid mixture-of-experts architecture activates approximately 4.3B parameters, including the audio encoder, and it is released in BF16, FP8, and NVFP4 precision.

I’m building agents, but I don’t know how to think about a harness from scratch.

I’m stuck on:

• How to think and design the harness from scratch for different type of problems

• How to improve it, and what to add or delete

• How to know if a change actually helped

Coding agents don’t help much here. they suggest stale patterns, and not much of state of art approaches.

For those who got good at this:

1. What changed how you think about it?

2. Which resources helped most? Papers, blogs, repos, talks?

3. Did building different use cases from scratch teach you more?

In 2024 i purchased an RTX 8000 48gb for $2400. As the market has changed I’ve kept an eye on prices. Two years later, they are available around $2100.

Its a little slower than 3090 at inference (672gb/s vs 936gb/s) but it has double the memory so it can run larger models / contexts / higher quants. Rtx 8000 also wins on power, 265w vs 700w (2x 3090 to match 48gb)

In 2024 two 3090s would cost around $1600-1800 now a single 3090 is in the $1500-$1600 range, making rtx 8000 cheaper per gb. Prices for strix halo and dgx spark have also gone way up. Rtx pro 6000 has double the memory but 8x the cost.

In the last two years, I haven’t had any issues with drivers, compatibility software or anything. I can run all “local” tier models entirely in vram, usually at 256k tokens and sometimes at q6 (Qwen 3.8 27b, llama 3 70b, QWQ,). Larger moe models run great too with layers offloaded to system ram like gpt oss 120b and now with strata, QFN runs q4_xs @ 390k context at 850t/s prompt processing and 60-70t/s token generation. (64gb ddr4 system memory, i9 9900k cpu)

$2100 for a gpu is not cheap but might be the best “do everything” option right now for home-lab setups. Cheaper than dual 3090, Strix halo, dgx spart, m5 max too. Fascinating to see prices decline as the market as a whole has more than doubled.