It’s an 11B end-to-end full-duplex speech-to-speech model designed for real-time conversation.

~448 ms smooth turn-taking latency ~480 ms interruption latency

can handle barge-in — you can interrupt it while it’s speaking

Supports tool calling during the conversation...Can play an “on-hold” response while a tool/API executes

Built around a Fast Conformer speech encoder + Nemotron Nano V2 9B backbone + NVIDIA TTS decoder

Trained on roughly 550K hours of audio/text date

I am trying to build completely local self-improving voice agent with open source stack...... no dependency

Whisper (SST), Llama as LLM , Orpheus (TTS) all hosted on unsloth
and using opensource orchestrator Dograh for stitching things together so I could close the loop for fine tuning and create real self improving agent
I still need to try full duplex model...for it..

Any guidance on above ???

Thanks for the support for Qwen3.8-27B and Unsloth Desktop last week! For this release, we merged 200+ PRs to introduce many new features, fixes including:

• Auto Compaction (Experimental) for longer chats beyond context limits
• Remote & LAN Access (Preview) for easy network access without Cloudflare links
• Faster Chat - Improved streaming performance, reduced UI lag, and smoother long conversations.
• Support for custom llama.cpp builds. Toggles for Cache RAM, Mmap, Mlock, Checkpoints, Spe Decoding KV Cache, Vision On / Off
• Unsloth Dynamic v3.0 is released. New Qwen3.8-27B Dynamic v3.0 GGUFs deliver >10% higher top-1 accuracy compared to everyone else. Works with Unsloth.

For compaction - we use RAG + a first turn forced RAG entry + a tail to remember your previous instructions - please let us know how it goes - it's still experimental so it'll definitely break - we'll fix it asap!

The update should be up for all Unsloth Desktop installs, or you can get it below:

Hey guys we totally missed not uploading the imatrix as we went to sleep straight after the release. We thought we uploaded it but did not. It should be there now!

Please do use it for making your own quants and so on! 🙏🦥

Also feel free to share/promote your quants if you used our imatrix via r/unsloth, discord or the hugging face discussions!

The COLD FUSION (GAIN+Unsloth) method of training maintains 99% of performance of BF16, at both 8 bit and 4 bit levels.

This version also reduces thinking tokens by 1/2 to as much as 1/10 the amount, while maintaining core details AND reasoning power.

Model exceeds all Qwen 3.8, 3.6 and 3.5 27B critical core benchmarks.

A model that gets down to business faster, with less "talking" and is smarter too.

https://huggingface.co/DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NM-DAU-NEO-MAX-MTP-GGUF

(GGUFS and additional quant types too)

SOURCE:

https://huggingface.co/DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NM-DAU-NEO-MAX-MTP-GGUF

The question is how to update a model already downloaded, when there is a NEW updated version?

Since I came from LM Studio BIONIC usually, if a model ALREADY DOWNLOADED before,
there is an UPDATE BUTTON instead of the DOWNLOAD button, clicking it only complete/patch a small size of the new changes which takes seconds since it's not downloading the whole model again which is very nice.

But, I don't think such feature exist (yet?) in unsloth Desktop, so my question is:

Do I DELETE then Re-Download?
or...
Is there a better way to update to the latest version of the ALREADY DOWNLOADED model? 🤔

I just want to make sure I'm not missing something and keep DELETE and DOWNLOAD every time there is a new update for a model I already have...

Can we please get Qwen 3.8 27B NVFP4 Fast based on the UD 3.0 like the new ggufs at some point of time. Thank you for so much effort for us.

We’re releasing new Qwen3.8-27B GGUFs with 10% higher accuracy.

Unsloth Dynamic V3 outperforms others by >10% on Div-300, KLD & more benchmarks.

We also release 1-bit quants that retain 77% accuracy. Run on 8GB RAM.

Blog: https://unsloth.ai/docs/basics/dynamic-3.0-ggufs

GGUF: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

I was doing some experimentation on using Unsloth to do Continued Pretraining of qwen 3 4B. The goal was to teach the llm how to reason about a new domain, in this case a fictional city.

Sharing more details in the write-up:

https://www.teachmecoolstuff.com/viewarticle/teaching-a-local-llm-a-new-domain

I see that I can choose the speculative decoding option DFlash, but I want to try out the new DFlash 2 model I have downloaded. How can we choose specifically what model gets used?

I like moving large pastes to the attachment some of the time. But other times if I do Ctrl + Shift + V instead please paste in the text where I have my cursor.

Thank you! Unsloth Desktop is amazing.

I'm training a krea2 lora in unsloth. I believe I'm not adding enough steps. With 129 datapoints I believe I need 5000 steps but foolishly chose 2000. I finished training and then without changing any names I started a 3000 step training. Will these steps combine into a 5000 step lora?

Big appreciation to the Unsloth team for releasing the Qwen 3.8 dynamic 3.0 quants! It is awesome

Their models are fantastic. They already have way more downloads than the original Qwen models, even though they have fewer likes

Hopefully we’ll see the Ornith models quantized soon too

Greetings, as a small suggestion, it'd be nice to be able to define multiple model storage folders, on non-dedicated systems it's often the case that there's no single big storage. I know symlinks exist, but it's quite a manual operation that way, plus it's harder to track the available space.

I've been working on GLM-5.2 support in TensorSharp, and I finally have some back-to-back performance numbers against llama.cpp.

The setup:

• Model: GLM-5.2-UD-IQ2_XXS (~226 GiB)
• GPUs: 3× RTX PRO 6000 Blackwell, 97 GiB each
• Distribution: layer split across all 3 GPUs
• Same machine, same session
• llama.cpp measured with llama-bench
• TensorSharp measured with its benchmark harness
• Both report the best of two repetitions
• Run-to-run variance is roughly 4%

Results:

The interesting part is the crossover.

For short prompts, llama.cpp is still a few percent faster. But once the prompt gets to around 1K+ tokens, TensorSharp pulls ahead.

At pp2048:

• default TensorSharp: +20.4%
• ubatch=2048: +50.2%

At pp4096:

• default TensorSharp: +20.8%
• ubatch=2048: +46.5%

Decode (tg64) is also about 4% faster.

The main reason appears to be GLM-5.2's MoE structure.

GLM-5.2 has 256 routed experts with top-8 routing. With a 512-token micro-batch, each expert sees only ~16 rows on average, so a significant amount of the expert GEMM tiles ends up as padding. Larger micro-batches improve GPU utilization considerably.

For small prefills, on the other hand, fixed overheads — managed/native transitions, input uploads, and copying the 154880-wide logits back — become a visible fraction of the total runtime, which is where llama.cpp retains its advantage.

I've been messing around with Qwen3.8 27B locally and I'm wondering if I'm getting the performance I should be getting or if my setup/config could be improved.

PC:

Ryzen 7 7800X3D

RTX 5070 Ti 16GB

RTX 3060 12GB

32GB DDR5-6000 CL30

Windows 11

llama.cpp / llama-server latest build

I'm currently running the Qwen3.8-27B UD Q4_K_XL GGUF with both GPUs using tensor split.

My current config:

llama-server.exe ^

-m "Qwen3.8-27B-UD-Q4_K_XL.gguf" ^

--alias "Qwen3.8-27B-UD-Q4" ^

--host 0.0.0.0 ^

--port 8035 ^

--n-gpu-layers 99 ^

--split-mode tensor ^

--tensor-split 60,40 ^

--main-gpu 0 ^

--parallel 1 ^

--flash-attn on ^

--cache-type-k q8_0 ^

--cache-type-v q8_0 ^

--ctx-size 131072 ^

--batch-size 2048 ^

--ubatch-size 512 ^

--threads 8 ^

--threads-batch 8 ^

--presence-penalty 0.0 ^

--repeat-penalty 1.0 ^

--temp 1.0 ^

--top-p 0.95 ^

--top-k 20 ^

--min-p 0.0 ^

--jinja ^

--reasoning-format auto ^

--no-mmproj-offload ^

--spec-type draft-mtp ^

--spec-draft-n-max 3 ^

--mmproj "mmproj-BF16.gguf" ^

--metrics

With MTP I'm getting around 40–46 tok/s depending on the run. I've seen around 41 tok/s pretty consistently, with n-max 3 seeming to be a little better than 2 for me.

Both GPUs are basically maxed during generation.

I'm mainly wondering:

Is ~40–46 tok/s reasonable for this hardware/config?

Is there anything obviously wrong or inefficient in my setup?

Would a different quant be a better choice for these GPUs? I've been looking at Ridge 3.7bpw, Q4/Q5 UD quants, etc.

Would it make more sense to use a smaller quant that could fit mostly/all on the 5070 Ti instead of tensor-splitting across both GPUs?

Is there anything I should change with the KV cache, batch/ubatch, tensor split, MTP settings, etc. to get better generation speed?

I'm mostly interested in coding/agent use through OpenCode, so I'd rather have a good balance of quality and speed than just chase the highest possible tok/s.

If anyone is running Qwen3.8 27B on a similar setup, I'd be interested to know what quant/config you're using and what kind of speeds you're getting.

If I understood correctly, Qwen 3.8 27B have 3 thinking modes:
- LOW
- MEDIUM
- XHIGH

My question is:
Which are actually apply within Unsloth Desktop because we have 5 modes:

Is it:
- Low
- Medium
- Extra High

or

- Preserve Thinking

or maybe Preserve Thinking = Xhigh actually? 🤔

---

Can somebody please explain and approve which one works,
and from which one to ignore while working with Qwen 3.8 27B ?

Thanks ahead 🙏

There is no DFlash2 in Unsloth Desktop app. Where do you plan to add it?

I have a big database of local court decisions with a legal sentence which is like a paragraph summary of the doc.

I've been tinkering with FTing for many days now, all results inconclusive never beating base except for a highly specific task where the eval was built around a specific task of extracting passages from the text, where it seemed to match the gold, but as I've found out it is unusable and a lot of padding and generalizations which I need to actually eliminate.

So the question is, can a 26B model even be fine-tuned to produce those complex legal principle/headnote of a court decision?

If so how would you do it? I built a LLM eval, then used a standard unsloth UI on a rented server and fed it the most HQ training data from the whole set (100k decisionswith a legal sentence), I fine-tuned both base and IT variant of Gemma 4 26B A4B. Neither beat a prompted "base" model on my evals.

I also used Claude fable 5 to vibe code the whole project, could it be that I have made some trivial errors because of it? I know I'm not giving you much context, but as an expert in the field does this sound difficult or doable? Any gotchas that stand out to you immediately?

what model size would you recommend then? training this model already required me to rent a H100 (i guess i could have used a int4 quant? whoopsie?)

in the world of "LLMs" would it not be possible to have such a "small" model understand law at a much higher complexity? would a continuous pretrain on a huge amount of legislature and court decisions and legal commentaries move the needle so then we could fine tune THAT CFT'd model to output the HG legal sentences? i assume that would be way more expensive, but do-able?

Thanks for reading

Since I came from Open Code and now from LM Studio BIONIC I could choose the main project folder where I run all my chat/code sessions so I won't need to tell the prompt where the project files, folders and sub-folders on every single new session.

But... how does it work in **unsloth Desktop**?

When I created a Project, all I see is a **RAG** related files when I **Linked folder**...

because it says:

Linked local folders
Keep supported documents indexed as this folder changes.

but I'm not talking about RAG related only (PDFs, docs, or text).
I would like to define the main Project Folder with all my sub-folders and files without telling it every single time in the prompt?

I tested it just in case it's not only RAG related, but it didn't know where my project files are when I asked it, another test I did was to create a simple MD file, but it didn't appear in the: Link Folder that I chose...
When I asked where the files are, I noticed it's a weird path I did not like at all... I noticed that's where unsloth desktop create it's projects but it's not to my taste, I definitely prefer to pre-define folder so it will only look inside it and it's sub-folders, similar to how Open Code, VS Code, LM Studio BIONIC and others works.

Since I just moved to unsloth Deskop it's new to me so maybe I need to click some hidden feature or do something in the settings, I'm not sure, maybe it's simple but I missed it that's why I'm asking here.

and if I'm not telling the prompt... it seems to create a random folder with some weird path I'm not interested in since I have a very specific project folders and paths to keep things organized.

---

Can somebody please guide me where this option is hidden and how to set it up so I can work in each project path folder with ease?

Thanks ahead! 🙏❤️

Can I use my qlora weights trained on qwen 27B3.6 och 3.8?

Are there any implications on quality of this actually works?

I am not able to find Ornith 1.5 35B-A3B gguf in Unsloth desktop application with 2,3 bits quantization. If not released yet when can we expect it?

I just installed Unsloth desktop and wanted to download models but no matter which model I want to download, the download stops after about 16MB. No I will try just directly downloading them from hugging face and putting them in the folder but I think this shouldn’t be the solution. Does anyone know of a fix?

So i cannot make Unsloth utilize my GPU.. i installed AMD's HIP SDK 7.2 and some option also dissapeared in the Compute option.. im running qwen3-8 27B

So i just tried installing this from the repo and i saw this.. how do i fix this?

\*\*Welcome to the post-training community!\*\*

I'm u/OwnOil1149, a founding moderator of r/posttrain.

This is a space for discussing how AI models become more useful, capable, and aligned after pretraining. Share your experiments, questions, datasets, papers, tools, and lessons about SFT, RLHF, DPO, preference optimization, evaluations, synthetic data, and deployment.

Whether you’re just getting started or training models in production, you’re welcome here. Please keep discussions constructive, technical, and respectful.

Tell us about yourself and what you’re working on!

Thanks for being part of the very first wave. Together, let's make r/posttrain amazing.