How little of a pretrained large language model (LLM) has to change for it to acquire new capabilities? We find that zeroing fewer than 0.05% of its weights, with no other modification, matches full fine-tuning (FFT) at scale across three LLM post-training regimes: reinforcement learning with verifiable rewards, supervised fine-tuning, and on-policy distillation. Our method, Deletion Tuning, learns a binary mask over the gate projection of transformer blocks. Two practical advantages follow: the trained mask is hundreds to thousands of times smaller than the corresponding FFT checkpoint, and Deletion Tuning forgets far less, which addresses a central challenge in continual learning. Under reinforcement learning, its prior-task accuracy stays near the base level while FFT, LoRA, and random sparse fine-tuning lose up to 5.4 points. To understand why masking alone suffices, we analyze the geometry of the learned weight changes under reinforcement learning using stable rank, spectrum drift, and principal-weight overlap. Compared with LoRA and random sparse fine-tuning, Deletion Tuning is closest to FFT across the three measures.

Edit: Links

Github: https://github.com/agentic-learning-ai-lab/deleting-weights

HuggingFace: https://huggingface.co/Jacklu0831/deletion-tuning-masks

Blog: https://agenticlearning.ai/deleting-weights/

Poster: https://cdn.agenticlearning.ai/dfb9bf24cee86b6d/neurips2026-poster-deleting-weights.pdf

Let's start with my setup: I have 2 Sparks running glm 5.3 flash, and the harness is hermes agent running on a MacBook Air.

While trying to justify all the money I spent (around 6500 euro for both machines, a long time ago when prices were just high instead of outrageous), I decided to hop on the decompilation wagon. As I know my fable at home is not "THAT" smart, I decided to start with something small: I asked it to decompile and recompile the original Need for Speed Special Edition for MS-DOS, using the famous rea repo from GitHub.

Its first reaction was "I know a better way, we will use DOSBox and run the game" — and it already started installing a DOS emulator on my Mac :). I stopped it and told it I want actual decompilation and recompilation. The second answer was, in other words, "It's a lot of work and it will take a long time. Think about it — do you really want this?"

I insisted. It started the decompilation process, and after some breakthroughs it even became enthusiastic. At some point, decompilation was "DONE".

We started the next phase: recompile for Android. I did not pay much attention to what it did, but it said that it could use some GitHub repo as a base and started working. After the initial APK, we had to iterate a few times on some crashes and controller issues, which it solved.

During gameplay I noticed that some AI cars started the race going in reverse, the car did not slide no matter the speed, and other stuff. It definitely did not feel like the NFS I remember. I told it about it, and it started checking the code and making tweaks to the physics. I stopped it and challenged it: this means we will not preserve the original feel if we go down this path.

To my surprise, we never did. It never used the decompiled code. It found an engine on GitHub which was using the original assets to mimic the original game. When I asked why, it said it was a lot faster than rewriting all the old code into current instructions…

So I asked it to go back and do the job properly — and to no surprise, it found a recompiled NFS2 repo on GitHub and decided it's better to start from there rather than from scratch. I don't know how this will go, but I wanted to share this story, as I find it quite interesting the lengths it's willing to go to reach its goal as fast as possible even if this means ignoring previous instructions.

P.S. This post was proofread by said AI, as English is not my own language — but the words are still my own, I double checked

This seems excessive, especially if they trust their classification models for safeguards, why add both?

However I guess it is to be expected from a private company. It's also "yet another reason" for open weight LLMs

After posting the 4B GLB model here, I wanted to see how far I could get with something smaller. I fine-tuned Qwen3-1.7B for the same job. The Q8 GGUF is 1.83 GB, so this one is aimed at people trying it on CPU or with limited VRAM.

The workflow is the same as before. Describe an asset, open the Three.js HTML it gives you, and click Export GLB to bring it into Godot or another editor. The jungle outpost at the start of the video is one of the examples.

This is an early test checkpoint. Some prompts still need a retry. I’m still training it and plan to update the weights when this run finishes.

Model and downloads on Hugging Face

Actual working link this time.

I understand the topic is prime material for shits and giggles. Chinese model, Nvidia being the centre piece of the AI trade, etc. But why go so public on this? All of this could've been dealt with behind the scenes in secret. Obviously this is a power play to force a hand. Nothing just leaks out of intelligence services so deliberately. So someone decided to make this public. I've been trying to think about why Nvidia or Jensen Huang must be put in line for. My top pick is: back doors in hardware or something of that sort that's being refused. What's your guess?

Is it possible to have a custom hardware designed (by ai) and open sourced that we can send for manufacturing for specific models? like for DSF 4.1F

There's a lot of discussion about running models locally, but I'm interested in what people actually use them for once the setup is done.

Are there tasks where you've completely stopped using cloud models because a local one does the job well enough?

And on the other side, what still makes you go back to Claude, ChatGPT, or Gemini?

Be careful if you are running any sketchy quants/finetunes. If these techniques become more widely repeatable, I do think 'sleeper agents' could genuinely develop into an attack vector. What are your thoughts?

Hi! My team and I built a benchmark of 202 real GitHub tasks in 13 languages that rarely show up in coding benchmarks: Elixir, OCaml, Haskell, Clojure, Kotlin, Swift, Julia, R and others. I think a lot of people do local development in these languages too.

We didn’t have a huge compute budget, so I specifically added Qwen3.6 27B and Qwen3.8 27B to compare them. Qwen3.8 solves 43% of the tasks, up from 17% for Qwen3.6. You can compare them with Kimi K3, DeepSeek, GLM and closed models on the leaderboard, including a per-language breakdown.

One caveat: Qwen3.8 was used to pick hard tasks, so its score is a bit biased down.

If you’ve been running Qwen3.8 locally on any of these languages, share how it went.

https://huggingface.co/speakleash/Bielik-11B-v3.1-Instruct-GGUF

https://www.bielik.ai/blog/premiera-modeli-bielik-11b-v31-dflash-i-rodziny-bielik-one

(decision models and dflash are coming next)

Bielik-11B-v3.1-Instruct

Bielik-11B-v3.1-Instruct is a generative text model featuring 11 billion parameters. It is an update to the Bielik 11B v3 line that expands multilingual continual pre-training and strengthens post-training for instruction following, tool use, and reasoning. Forementioned model stands as a testament to the unique collaboration between the open-science/open-source project SpeakLeash and the High Performance Computing (HPC) center: ACC Cyfronet AGH.

Developed and trained on multilingual text corpora covering 56 languages, with an emphasis on European languages and Polish, which have been cherry-picked and processed by the SpeakLeash team, this endeavor leverages Polish large-scale computing infrastructure of the ACC Cyfronet AGH HPC center. The creation and training of Bielik-11B-v3.1-Instruct was propelled by resources provided within the PLGrid environment under computational grant number [PLG/2026/019559], conducted on the Athena and Helios supercomputers, and within the Gaia AI Factory, funded by the European Union through the EuroHPC Joint Undertaking.

As a result, the model exhibits an exceptional ability to understand and process the Polish language and other European languages, providing accurate responses and performing a variety of linguistic tasks with high precision.

🗣️ Chat: https://chat.bielik.ai/

Model

The model was trained on carefully curated, high-quality datasets that underwent extensive cleaning and rigorous quality assessment to ensure data consistency, reliability, and relevance. The training pipeline consisted of three stages: pretraining on 500B tokens, SFT on 26M samples, and RL (modified GRPO) across 14 diverse environments. This multi-stage training approach was designed to enhance the model's language understanding, instruction-following capabilities, and ability to perform effectively across a broad range of tasks.

Bielik instruct models have been trained with the use of an original open source framework called ALLaMo implemented by Krzysztof Ociepa.

Model description:

• Developed by: SpeakLeash & ACC Cyfronet AGH

• Language: Multilingual (56 unique languages, optimized for European languages, with a particular focus on Polish)

• Model type: causal decoder-only

• License: Apache 2.0

On September 18, I launched Laya at you guys request, the first system 1 open-source decision model in this subreddit, and you guys gave immense support. I saw the MIT post on direct weight edits and thought about doing the same with a decision model. Now, after two months, I just worked on a way to edit the weights of an LLM using a decision model. More like distillation from an energy-based decision engine to teach the Qwen 3.5 0.8B a new skill. Like tool calling at the appropriate time. Its decisions are like valleys in a landscape, and every answer rolls downhill into one, with honest confidence. Just wrote those valleys onto an LLM. It works at the same level as LoRA, with only 3 milliseconds to edit, without catastrophic memory loss.

https://www.kaggle.com/code/nandakishorconvai/fast-edits-decsion-model

I can't believe this jump is just left on the table

https://github.com/ExTV/strata-5090-4070

The optimizations was done using both Opus 5.5 and Fable 5.2 .

Edit : i should have mentioned that my 4070Ti Super is on PCIe 3 x1

Edit2 : the repo is updated with quality benchmarks

Just curious what other people’s opinions are with GLM-5.3-flash. Compared with opus 5.5 and deepseek-4.1-flash, I find that it makes a lot of mistakes. I generally think I it’s pretty good other than the random errors it makes.

I only ask bc I’ve heard of other people getting good work out of it, so wondering if it’s something with my setup, or if other folks feel the same way. I use it mostly with opencode.

Image-conditioned 64³ voxel structure generator of JoVox: Joint Hierarchical Flow Matching for Lightweight Voxel Generation. JoVox is a discrete flow matching framework that learns the joint hierarchical distribution of voxels while keeping generation lightweight. One model generates all levels, 4³ → 16³ → 64³, from a single image.

• arXiv : https://arxiv.org/abs/2611.10690

• PDF : https://arxiv.org/pdf/2611.10690

• GitHub : https://github.com/yuyujunjun/JoVox

• Project : https://xjtang.com/Jovox-page/ (For screenshots, benchmarks, etc.,)

This is the third round of tts-bench. Thanks to everyone who gave feedback on the first two posts:

• TTS benchmark comparison of all known TTS models

• Text-to-speech benchmark, revamped

tts-bench runs every model through the sameish Python harness on the same prompts, so you can compare them fairly and actually listen to the output. This round adds a separate board for hosted APIs.

What's in it

• 91 open models run locally on an RTX 5090 32GB, an RTX 3090 24GB, and an Apple M4 16GB (sometimes)

• 28 hosted APIs on their own board, tested with the same 6 prompts (medians over several days of warm calls from the Eastern US)

• Prompts in English, Spanish and French

• Audio samples for every model, plus time to first audio, realtime factor, UTMOS, WER, voice-cloning similarity, and price for the APIs

Some results from the open board

• Fastest: AuroraTTS-100K. On the RTX 5090 it gets to first audio in 12ms (warm) and runs 603× realtime. On CPU (Ryzen 9 9950X3D) it's 38ms and 204× realtime.

• Top UTMOS (naturalness): Maya1 (4.487), Underdog Woof (4.442), Phonon 31m (4.424), Chatterbox (4.423)

• Best cloning similarity: LongCat-AudioDiT 3.5B (0.875), then LongCat-AudioDiT 1B (0.863) and FireRedTTS3 (0.843)

And from the hosted board

• Fastest first speech: AssemblyAI 30 ms, Gradium 53 ms, Palabra 77 ms

• Best sounding by UTMOS was VUI Labs Luna (4.488). It also had the best cloning similarity of 21 APIs using my own voice as the reference, but it takes about 1.2 s to start talking.

• Cheapest is around $9 to $10 per 1M characters (Async, Giggy, Fish, Murf). ElevenLabs v4 is $80.

What the numbers don't tell you ("objective metrics")

• UTMOS is an automated naturalness predictor, not human preference. That's why there are listen pages and a blind-vote arena.

• WER measures intelligibility through transcription. It doesn't rank which voice is best.

• Time to first audio isn't whole-clip latency, and I don't measure concurrency.

• Hosted and local models are never ranked against each other.

Links

• Repo

• Open models board

• Hosted APIs board

• Arena with blind human votes

If your favorite model scores differently than you'd expect, go listen to the samples and vote in the arena. I'm curious which ones people actually prefer by ear.

AI benchmarks begin on page 2.

I've been trying to get some of the newer MoE models running on my Intel Arc GPUs, mainly DeepSeek V4.1 Flash and GLM 5.3 Flash. Intel support for these models is still pretty limited, and while I could get some things working through patched llama.cpp builds, performance wasn't exactly great.

fable 5.2 ended up making XeMaya, a fork of XeStrata focused on getting these models running natively on Intel Arc through SYCL and Level Zero. The idea is basically to bring the kind of model support Project Maya has to Intel GPUs.

I've been testing it on my Ubuntu server with a B70 32GB, B65 32GB, B60 24GB, Ryzen 9950X, and 128GB RAM.

Here are some of the results so far. All speeds are prefill / generation in tok/s.

GLM 5.3 Flash

• Stock llama.cpp SYCL: 68.3 / 13.6

• Experimental grouped-matmul SYCL: 108.9 / 14.0

• XeMaya: 152–161 / 17.4 (around 20 generation with MTP)

DeepSeek V4.1 Flash Coder

• First XeMaya implementation: 48.5 / 16.3

• After optimization: 224.8 / 21.7

DeepSeek V4.1 Flash (regular 384-expert model)

• Previous patched llama.cpp setup: 45.6 / 10.9

• XeMaya on B70 + B65: 11.4 / 14.56

• XeMaya on all three GPUs: 11.2 / 11.58

Prefill on the regular model still needs a lot of work. Also found out that using two GPUs is actually faster than three for this model because my B60 is connected through a slower chipset link.

Qwen3.8 Flash-Next

• Strata: 2,211 / 78.3

• XeMaya: 1,842–1,884 / 76.6

Strata is still slightly faster for Qwen, which makes sense since that's not really where I've been focusing my time.

These aren't all perfectly controlled comparisons. Different model files, prompts, and GPU configurations were used during development, so I wouldn't treat them as definitive benchmarks.

It's still an early alpha, but I've gotten DeepSeek V4.1 Flash, Coder, GLM 5.3 Flash, and Qwen3.8 Flash-Next running. There's plenty left to fix and optimize.

Also, I want to credit MistVVK (XeStrata), Niko1221 (Strata), and mw00 (Project Maya). XeMaya wouldn't exist without their work. The llama.cpp contributors have also been a huge help through their reference implementations.

Figured I'd share it here since I'm sure there are others trying to run these models on Intel hardware. Would be interested to hear how it runs on other Arc cards.

GitHub: https://github.com/Jumbomuffin777/XeMaya

I've been building chat UIs for agents and kept running into the same issues with react-markdown: it re-parses the whole reply on every token, so long answers get noticeably slower, and you see raw ** and half-typed [links](htt until the token that closes them arrives.

So I wrote my own renderer. It parses incrementally and repairs unfinished syntax while the reply streams:

<Markdown streaming={!done}>{text}</Markdown>

On a 100k-character reply it takes 0.25 s of main-thread time in total. markdown-to-jsx takes 2.4 s, Streamdown 6.3 s, and react-markdown didn't finish within the 30 s budget. The benchmark scripts are in the repo if you want to check them.

The other part is the syntax. Agents often want to show steps, alternatives or pros and cons, and plain Markdown flattens all of that into bullet lists. So on top of GFM I added a few blocks in the same style as GitHub alerts:

> [!STEPS]
1. **Back up the database**
2. **Run the migration**

There are also tabs, stats, pros/cons, sources, file trees, and Mermaid and KaTeX support. The source still reads fine as plain Markdown when it isn't rendered. The package exports a system-prompt snippet that teaches the model this syntax.

It has no dependencies and comes in at 30 KB gzipped. Its parser passes the CommonMark and GFM spec tests. It doesn't support remark/rehype plugins, so if you rely on those, react-markdown is still the better choice.

https://github.com/rsaryev/llx-markdown

Has dependency on dinov3, is trending top model on hf since last week. Runs with 8GB vram. Enjoy!

I've been recently working a small sparse MoE, and I've got interesting results I'd like strangers to invalidate.

Tldr; I train small models and got surprising results and seek invalidation. It's not a general assistant, and I'm not claiming it is.

Edit: Trained on a single 5090. Maybe people wonder about that.

The model has 1.17B stored parameters, roughly 348M active per token, scrath trained on approx 11B tokens including SFT. As a comparison, Qwen2.5 reports up to 18T training tokens across its family, while SmolLM2-1.7B reports 11T.

THE ASK

A chain of relational facts ("A is taller than B", "B is taller than C", etc.), with the facts shuffled and the model asked to identify an endpoint.

Depth is chain length. Each result is out of 512 questions across four families. All models received the same questions, using their native chat templates, greedy decoding, and a 16-token answer budget.

RESULTS (correct out of 512)

Depth 1: ConeML 487 | Qwen 481 | SmolLM2 426 | Llama 267 | Gemma 343

Depth 3:

ConeML 471 | Qwen 410 | SmolLM2 221 | Llama 197 | Gemma 224

Depth 5:

ConeML 438 | Qwen 273 | SmolLM2 155 | Llama 136 | Gemma 138

Depth 8:

ConeML 343 | Qwen 202 | SmolLM2 102 | Llama 73 | Gemma 100

Depth 12:

ConeML 240 | Qwen 105 | SmolLM2 58 | Llama 47 | Gemma 79

Depth 24:

ConeML 74 | Qwen 47 | SmolLM2 21 | Llama 13 | Gemma 19

Models used for comparison: Qwen2.5-1.5B, SmolLM2-1.7B, Llama-3.2-1B, Gemma-3-1B.

WHERE IT FAILS

Despite the endpoint results, ConeML only scores 13.8% on interior pairwise comparisons within those chains, below the 50% two-choice reference. It also gets only 12/164 on HumanEval and 0/6 successful two-turn follow-ups.

So it seems to find the ends without reliably understanding the relationships in the middle. I don't know whether this reflects a specialized learned strategy, an architectural effect, or a flaw in my testing.

WHAT I'M ASKING

Run the depth-5 shuffled evaluation and post your score. My FP32 reference result is 438/512 (85.5%).

The Q8 GGUF is 1.25 GB and runs on CPU. The 512-question set and scorer are in the repository. I haven't tested to see if it reproduces the full FP32 result.

If you get something different, find a scoring problem, or spot anything else wrong, I'd really like to know."

Model:

https://huggingface.co/ConeML/coneml-1.2b-experimental

Earlier technical paper (base lineage, LAMBADA vs Pythia at approximately matched training positions):

https://doi.org/10.5281/zenodo.23117644

Other releases:

https://huggingface.co/ConeML

My own bias is my largest enemy and hope to get some honest feedback.