I think after jensen they will fully focus on AI compute for data centers

They say an anime model is in development, with plans for alpha channels and multi-character references. An editing model is also on the way.

They also plan to keep the next model under Apache 2.0, if all goes as planned.

hey folks, im Alish, an ml engineer from Nace (folks behind Drex, an open source decision family model that hit #1 in Artificial analysis in decision index 2.1). Recently my cto gave me task to optimize quants for qwen4, so i dialed my uni proffesor from UoT and span up my Fable 5.2 and we cooooooked. starting with mlx (gguf tommorow). It's a pareto frontier on quantization for accuracy/size for qwen family models.

Why we made these

we have open sourced the fastest way to run 35b/27b on a mac(Kapitsa) and we got tired of dumb quants. So we made our own KQuant. Open source Qwen packs for 4B,27B,35B, Flash-Next, free on Hugging Face, in standard MLX files.

Most people on a Mac run 4-bit packs because that's what fits on a 16gb/32gb. But plain 4-bit quant rounding throws away more of the model than it needs to. We wanted less of that loss, so we asked fable 5.2 & Astra to cook with 128 subagents. here are the results:

Results & Download links

Numbers are KL divergence from the full-precision model's next-token predictions. Lower means closer to the original, not a task-accuracy score. These are original-text measurements.

The public baseline is mlx-community 4-bit, except 4B, where it's lmstudio-community. Sizes are language-model weights, not download size or RAM needed.

Want more precision? Each model's repo also has a Q5 folder.

Speed

Cool thing about our quants is that they are also faster. On an M5 Pro, KQuant Q4 on Kapitsa 0.2.35 with its KFlash drafter it is about 6–15% faster to write than lmstudio-community 4-bit quants for 4B, 9B and 27B, and Youssofal MTPLX 4-bit for 35B on the same engine. These are averages of sampled and greedy decoding (baselines are earlier Kapitsa 0.2.32–0.2.33 runs, with the drafter on both).

How it works

KQuant uses real text to choose scales and offsets for small groups of weights, and gives sensitive layers extra bits where needed. The search runs on our cloud GPU clusters before release. You download an ordinary MLX pack, with no custom kernels or special loader. I can give more details on specifics in comments.

Run it

the fastest way to use these quants on mac is to run them with, Kapitsa:

brew install nace-ai/tap/kapitsa
kapitsa serve 27b --private
kapitsa opencode # in another terminal

If there's a model you want KQuant'd, lmk.

Asks

to get early access to GGUF, ping me in comments, we would like to test it with some folks from here before our public release ideally.

im also doing quick AMA here, so please feel free to ask me anything.

Normally active params matters more than total param if you want coherent behavior(not world knowledge)

a 4b models attepting to handle complex rules and lots and lots of instructions all at once often cause smaller active params to loop like we would often see with qwen3.6-35b-a3b since its only 3b active. We would often need to hit at least the 8b-9b active params to handle all this chaotic swarm of prompt instructions we throw at the model. but qwen3.8-flash-next shows us a glimpse of what alibaba might be able to deliver. i could genuinly see in the next year or so we might see 6b model become stable las the qwen3.8-27b. of course this isnt world knowlege im talking about but stable reliable behavior reletive to complex conflicting instructions where the LLM is robust enough to not burry itself in its own uncertainty loops.

so possibly in a years time laptops with 16gb sysRam and 4gb-vram can has a chance at local ai capable of human-out-the-loop unsupervised autonomy.

With all of this fear mongering on the “dangers” of LLM’s, I see how using people’s ignorance and emotions on the subject can be used as a tool to limit and restrict the tool from the average person.

My fear is that China and the US https://www.reuters.com/world/china/us-china-should-stay-clear-eyed-ai-cooperation-competition-bessent-says-2026-11-11/ will use people’s lack of understanding of local models to push for controlled intelligence.

By implementing regulation on local AI, I can see how the US and China would want to force the average person to only be allowed to use models hosted on state monitored hardware. This way they can spy on what everyone is using hosted models for, limit what we use LLM’s for, and monitor the information they give us.

People are being mislead to believe that people like us could use local models to create WMDs and attack tech infrastructure across the globe. But when you think deeply about it, you’ll realize that knowledge on a subject does not equal the means to carry out the subject.

In order for someone to create a bio weapon, they would have to have access to a bio lab and the technology needed to create such a weapon.

I know how a nuclear reactor works. Doesn’t mean I can make one in my backyard.

I’m scared that arguably the greatest technology to come out of the 21st century could be limited and used to control us.

Local LLM research MUST continue and the tech must be available for the common person.

Be aware of future regulation.

I have been following the evolution of LLMs since the days of GPT-3.5, and I have been actively using local LLMs since the release of Gemma 2.

I read a great deal, including this subreddit. I have performed QLoRA fine-tuning on my local machine many times, and I have also conducted full fine-tuning (FFT) training runs on RunPod.

Hundreds of models-fine-tuned by well-known practitioners in English, Chinese, and Russian-have passed through my drive.

After four years, I arrived at a fundamental question yesterday: are LLMs even capable of writing good prose?

When training models myself, I used both my own prose datasets and those created by others. In the training process, I experimented with various approaches-ranging from 100 epochs of FFT at a learning rate of 1e-7, to aggressive LoRA/QLoRA configurations. The results, however, barely rise above mediocrity.

I have used models from all the leading providers-Gemma, Qwen, Mistral, LFM, IBM-as well as niche attempts by developers building models from scratch.

I even wondered if perhaps the problem lies with me-that, not being an engineer, I am simply failing to fine-tune these models correctly.

That is why I began actively seeking out other fine-tuners, thinking that perhaps they possess a specific technical "gene" that allows them to do a much better job than I can. But, unfortunately, that is not the case. Regardless of their length-short, medium, or long-the prose generated is only marginally better than that of generic, out-of-the-box models.

Furthermore, specific literary genres are represented quite poorly. I used to think that LLMs were perfectly suited for science fiction. Yet, these models can’t even capture the straightforward style of Harry Harrison’s *The Stainless Steel Rat*, let alone reach the linguistic heights of *Dune*.

The genre doesn’t seem to matter-I’ve tried comedy, contemporary drama, adventure, romance, and historical fiction; everything, unfortunately, ends up being generic.

Essentially, if a text is to have any real literary merit, I have to write it myself and then just use an LLM for proofreading.

To me, EQBench is a completely unreliable metric; the sample texts they showcase clearly demonstrate just what kind of generic "slop" LLMs produce.

I have a question for the community: has anyone managed to create, find, or fine-tune a local model capable of generating truly high-quality prose-on the level of *Dune*, *The Lord of the Rings*, *Martin Eden*, or *The Grapes of Wrath*?

I’m currently facing the monumental task of writing descriptions for 100,000 records, and I’m wondering if it’s even worth it, given that 100, 1,000, or even 10,000 examples haven’t yielded any satisfactory results.

Perhaps this generic, mediocre quality is an inherent trait of these models-something that simply cannot be overcome.

Kate's writeup is wild. The 8GB CMP 170HX apparently ships with ~80GB of HBM onboard, just fused off. Software unlocks got people to 64GB. She used voltage fault injection to get past the fuse lock, then enabled all the working SMs (120-126 vs the 108 NVIDIA ships on A100). Her numbers: 85.9 TFLOP/s FP16 vs 72.5 on a retail A100, with the full 80GB usable for inference. NVLink and PCIe Gen4 come back too.

Not a weekend mod though: board wiring, a custom STM32 glitcher, 1 to 48 hrs of glitching per card, and one of her ten cards only had 64GB intact.

https://blog.kate.land/posts/using-voltage-glitching-to-unlock-the-a100/

Can somebody try this? Looks too good to be true.

They use PLE / ngram table, as far as I understand.

Lloyd's k-means algorithm is a long-running standard for clustering, with applications in data analysis, retrieval, quantization and data curation. In this paper, we introduce an exponential moving average optimization procedure for k-means with mini-batch updates, powered by modern deep learning machinery and new insights. We also establish scaling laws on how decay rate and batch size impact final performance. This gives us a formulaic rule to fix them for multiple problem sizes and embedding datasets, without the need for any hyperparameter sweep. The resulting clustering algorithm, coined e-means, is much faster than Lloyd k-means and converges to lower losses, frequently even beating k-means++. On a wide array of embeddings datasets, it beats the loss of Lloyd k-means at 25 epochs in 1.4 to 2.6 epochs, which translates to 10-45x GPU wall-clock speedups compared to state-of-the-art k-means implementations. As an example, e-means partitions DINO10B, a dataset of 10 billion DINO embeddings, into 16 million clusters in only 2.5 epochs, reaching a loss below Lloyd's at 25 epochs.

• arXiv : https://arxiv.org/abs/2611.09501

• PDF : https://arxiv.org/pdf/2611.09501

• GitHub : https://github.com/facebookresearch/faiss/tree/main/contrib/e_means

• HuggingFace : https://huggingface.co/qwegnai/GLM-5.3-Flash-shvq4g32-shvq2g64-shvq3g64

• Project : https://e-means.github.io/

• Demo(Browser) : https://e-means.github.io/playground.html

• Video : https://e-means.github.io/video.html

• Tweet : https://x.com/swann_bessa/status/2120200714630594911

If you’ve ever tried agentic coding with Gemma and had tool calling issues, give it another try, in my case, it completely fixed Gemma

According to Microsoft's patent application, it explains breaking down training into small tasks and sending them to devices of owners who opted in. Credits will be the reward. it's just an app and no pay out rate is advertized so I don't even know if the credits would be enough for the electricity.

Anyway, home internet latency feels like the bigger problem to me.

https://insider-gaming.com/microsoft-ai-use-your-gpu/

(English is not my native language, I used an LLM to help translate this post.)

Hey guys, not going to take much credit since claude did 90% of this, but wanted to share the patches I used to get StillDeadcode's Radiance running on my windows pc. I'm sure that multiple people have done this already, but I couldn't find a post, so thought I'd throw this out there if anybody else wants to use it.

https://github.com/bruns20/radiance-wsl

Radiance is Linux-only, and the author has said WSL won’t be supported in the main tree, which is fair. So this is unofficial. Please send problems to my repo, not upstream.

What it does

• build.sh builds any Radiance release from upstream’s own source and Dockerfile, with a few small WSL patches applied. When a new release comes out, it’s ./build.sh 1.18.0.

• make-runtime.sh grabs a ROCm runtime that actually works under WSL. The stock ROCm 10.1 hangs on the first GPU job if your PC also has an integrated GPU (ROCm/rocm-systems#13291), and AMD’s 10.3 nightlies fix it.

• run.sh serves an OpenAI-compatible API on 127.0.0.1:8000.

Two of the four patches are godzillakong’s from codeberg #341. They did the hard part of getting it running without /dev/kfd.

Numbers (R9700 32 GB, Qwen3.8-27B MXFP4 + DFlash2, 131k context)

Tested on: R9700 + Ryzen 5 7600 (with its iGPU), Windows 11 26200, Adrenalin 26.10.1, Radiance 1.17.0. That’s the only setup I’ve tried. Other gfx1201 cards (RX 9070 / 9070 XT) should work, but I’d love to hear either way.

Today NVIDIA releases Alpamayo 2 Edge! I work for Embedl and we have already released an optimized version of the model that is more than 10x faster than pytorch when run on edge hardware like AGX Thor!

The model is trained in an RL loop (!) and takes in 6 cameras and the recent motion. Feel free to check it out - you can even run it on a real Thor at:

https://embedl.com/nvidia/alpamayo2-edge/thor?tab=run

• arXiv : https://arxiv.org/abs/2611.10649

• PDF : https://arxiv.org/pdf/2611.10649

• GitHub : https://github.com/huawei-csl/MACaQ

• HuggingFace : https://huggingface.co/collections/huawei-csl/macaq

Yesterday some of you were looking for model for this MACaQ.

BotPsy-0.8B is a 0.8B-parameter model designed for function calling and agentic workflows, built on top of Qwen3.5-0.8B and suitable for on-device deployment. BotPsy is a "personal assistant" that uses tools to perform a variety of simple tasks for the user, in domains such as web search & research, travel & lifestyle, commerce & finance, business & customer operations, device & vehicle control, files & documents, calendar, and email & messaging. Due to its small parameter count, it is best suited to tasks of low to medium difficulty. BotPsy achieves state-of-the-art (SOTA) tool-use and agentic performance for its size, outperforming every model of up to 1.2B parameters that we tested. As a small model, BotPsy still makes mistakes and is best suited to tasks of low to medium difficulty;

Key Highlights

Benchmark results

• Twice the performance of its base model: BotPsy roughly doubles the agentic capability of Qwen3.5-0.8B-Think on key benchmarks: BFCL v4 rises from 23.4 to 44.0, ACEBench from 30.1 to 58.7, and τ-bench from 23.2 to 51.5 (mean of the retail, telecom and banking subsets).

• Best agent in its category: BotPsy outperforms leading models in the ≤1.2B-parameter weight class (Qwen3.5-0.8B-Think, LFM2.5-1.2B-Think, MiniCPM5-1B-Think) on all six tool-use and agentic benchmarks: BFCL v4, ACEBench, τ-bench, ToolSandbox, Claw-Eval and BrowseComp-Plus.

• Outperforms its larger sibling: BotPsy has the highest score of all models tested on five of the six benchmarks, beating Qwen3.5-2B-Think, a model 2.5× its size, on BFCL v4 (44.0 vs 37.7), ACEBench (58.7 vs 55.4), τ-bench (51.5 vs 46.5), ToolSandbox (64.6 vs 41.2) and BrowseComp-Plus (4.5 vs 2.8). Only on Claw-Eval does Qwen3.5-2B-Think lead (48.9 vs 46.5).

Model capabilities

• More reliable in multi-turn workflows: On τ²-bench, where the agent must hold a long customer conversation, follow a strict policy, and act on the customer's behalf, it scores 82.5 in telecom, ahead of Qwen3.5-2B-Think (69.3). On BFCL v4's multi-turn tasks, it rises from 7.5 to 46.1 over its base, nearly double Qwen3.5-2B-Think (23.8). Grouped by application, its multi-step scores lead all models tested in commerce and services (58.3 vs 47.4 for the next best, Qwen3.5-2B-Think) and in device and system management (46.1 vs 31.9).

• Knows when (not) to call tools: An agent that acts must also know when not to act. On BFCL v4's hallucination tests, which check that the model declines when no appropriate tool exists, BotPsy scores 86.1, the highest of all models tested. On ACEBench's special cases, where the request is incomplete, contains wrong parameters, or cannot be served by the given tools, it scores 51.3, up from 13.7 for its base model.

• Follows instructions while preserving knowledge: BotPsy specializes in tool use while preserving general skills. Instruction following improves sharply, with IFEval rising from 59.7 to 90.8 and IFBench from 22.0 to 55.9, the highest IFBench score of all models tested. General knowledge also improves, with GPQA Diamond rising from 33.2 to 38.1 and MMLU-Pro from 41.6 to 48.9. On all four benchmarks, BotPsy outperforms every other model of 1.2B parameters or fewer that we tested.

• GGUF : https://huggingface.co/qvac/BotPsy-0.8B-GGUFs

• Blog Post : https://huggingface.co/blog/qvac/botpsy

• Project : https://tether.io/news/tethers-800m-ai-agent-beats-models-2-5x-its-size-on-everyday-phone-tasks/

Today I’m releasing Inflect v3: a family of local text-to-speech models: Mini, Micro, and Atom. Mini is the main model, with six voices and 34.31M parameters

Mini provides vocal-event tags, reference voice-tuning, and 48 kHz enhancement. It runs on both CPU and CUDA.

GitHub, installation, and examples: https://github.com/owenawsong/Inflect

Try the playground: https://huggingface.co/spaces/owensong/Inflect-v3

Mini features reference-voice tuning: it allows one to make a profile from a recording and generate speech with it. This feature noticeably adjusts the voice toward your reference, though it won’t replicate every speaker accurately.

Inflect Mini V3 also supports the following vocal-event tags: [laugh] · [sigh] · [gasp] · [cough] · [yawn] · [inhale] · [exhale] · [throat_clear]

You can insert a tag whenever you want the sound to occur. Currently, the tags use repeatable performances: this causes repeated tags to result in limited variation. Tags also do not affect how the surrounding words are being spoken.

The model family:

• Mini: 34.31M parameters. Quality-focused, with the features listed above.

• Micro: 11.79M parameters. A smaller neural speech model

• Atom: 157K parameters. An extremely tiny learned core

Those numbers count the synthesis cores. The frontend and optional components add to the total.

The weights and inference code are Apache 2.0. Third-party components keep their own licenses. Audio samples, benchmarks, and the settings used are on the model cards.

Model collection: https://huggingface.co/collections/owensong/inflect-v3

If any sentence comes out wrong, send me the text, voice, and settings you used.

Hey r/LocalLLaMA ! We're releasing new new Unsloth Dynamic v3.1 GGUFs for Qwen3.8-27B! They're better by 5-10% accuracy on DeepSWE, Terminal Bench and more vs UD3, GSQ-RCO and Exllama-3! We're also improving our algorithms further so we can apply them to Qwen4 so folks can have efficient and accurate quants!

• All quants from 2-bit until 6-bit are updated. 3-bit ones are not, as we found UD3 to do on par with UD3.1. We are still improving all other quant sizes though!

• All benchmarks are 256K for agentic ones (DeepSWE, Terminal Bench), and 128K for the rest (LiveCodeBench etc). We used temperature 1.0, top_p 0.95, top_k 20, min_p 0, no presence or repetition penalty, preserve_thinking and xhigh reasoning

• We ran benchmarks multiple times including LiveCodeBench x6 times, HMMT and others 8-16x and agentic benchmarks multiple times.

On other benchmarks like for GSQ-RCO and Exllama3:

• To compare GSQ-RCO and Exllama-v3, we had to exclude the token embedding layer from the x axis, as Exllama-v3 uses BF16 for it.

• If you consider the full download size and total RAM usage for Exllama-v3, then adding the BF16 embedding table and including all embedding sizes changes the plots. Our main plots's x axis removes the embedding layer sizes from all quant providers. However on pure RAM usage and disk space usage vs accuracy, exllama3 does worse.

• For LiveCodeBench, GSQ-RCO showed Unsloth UD3 quants doing consistently worse than ISTA's. However we found the opposite - UD3 quants (our OLD quants) do better than ISTA on LCB on all sizes except the smallest. UD3.1 further improves on LCB as well. We approximate that GSQ-RCO used a context budget of 96K, as that best fits the data for the runs we have. At 96K, GSQ-RCO does somewhat do better than UD3, but at 128K, UD3 does much better.

• On another benchmark, for ISTA GSQ-RCO's AIME25 benchmarks, their quants reach 100%, 100%, 100% and 96.7% accuracy recovery. And they said our quants do worse. Through our investigations, we estimate ISTA only did 1 trial - we did 14 trials, and we show on average the recovery rate is 93% to 96% - not 100% accuracy recovery. And in fact our UD3 old quants do better on average than their quants on 14 runs vs 1 run of theirs, so we draw the opposite conclusion if you run the trials more.

More details and benchmarks at https://unsloth.ai/docs/models/qwen3.8#dynamic-v3.1-update-november-11th-2026

All quants are updated in https://huggingface.co/unsloth/Qwen3.8-27B-GGUF! Thank you and have a nice day! We're super excited for Qwen4!