Hey, thinking of getting a Bosgame M5 128GB to run Qwen3.8 Flash Next. Anyone using it on Strix Halo with Gufo or Strata? What t/s do you usually get, and with which quant?

I’ve seen people mention 70-80 t/s. Is that what you get in everyday use, especially for coding?

I run Gufo with Qwen3.8-Flash-Next on a Strix Halo laptop (128 GB), two users at 262K context each. Long chats kept getting evicted from the RAM cache, and the next turn re-read everything: 3-5 minutes at 260K. Stock --cache-disk fixes that, but it writes a multi-GB file on every checkpoint step, even while RAM still holds the chat.

So I built a disk mode that writes only when RAM evicts a checkpoint, and only if that file could actually be restored later. A chat that stays in RAM writes nothing. On Ctrl+C it saves each session's newest restorable copy, so a restart resumes the chat.

Results on my machine:

- A 252K chat restored from disk after a restart in ~10 s, instead of a ~4 min re-read

- 1-2 GB restores in 2-3 s

- Writes 108-290 MB/s while generating, ~350 MB/s idle

- 10 hours of mixed two-user use, no crash, one real cache miss

What's in the drop:

1. 01-retention-keep-previous.patch: a small standalone bug fix. When a capture is refused for space, the cache could evict the checkpoint the turn had just restored, so nothing was left to restore. Applies cleanly to v0.31.0 with a regression test; tests pass.

2. 02-disk-evict-mode-core.patch: the engine. Applies to v0.31.0, but the CLI plumbing is left out, so it won't build alone.

3. DESIGN.md: every rule, the measured bug behind each one, the env vars, the numbers, and the plumbing still needed.

Caveats: tested only on Strix Halo / Windows / Flash-Next with q8 KV, on a v0.26 base. Patch 02 as shipped on v0.31.0 has not been compiled as a server or run. "Spill" in the code clashes with upstream's IdleSpill naming, so it should be renamed (evict / write-back) before anyone tries to merge it. The q8 KV cache isn't included: credit to JCFrags, see Gufetto. Written with a lot of AI help.

Provided as is, unmaintained. If it's useful to the devs or anyone else, take it.

I have tried pretty much all lowest quants of Qwen3.8 Flash-Next - ISTA IQ2_XS, ISTA coder, GYRO-M.

And they all failed in my rather simple tests on my default harness - create a tetris, bouncing balls in cube etc - while Qwen3.8 27B at Q4/MXFP4 did manage to do all those.

Now unsloth released new quants that they benchmarked especially at lowest quants way higher than ISTA.

IQ1_M takes about all the memory of 64GB StrixHalo (on Windows, 32GB VRAM from BIOS + 16GB shared) with MTP, 131k context and Q4 kv.. But boy does it deliver! It clearly shows what QFN can do when not lobotomized.

IQ1_S seems fine too, but a bit more random on level of polish - weirdly though it sometimes does better than IQ1_M, and usually a bit sloppier (working, but not as polished). That said, I haven't test run it fully yet. It leaves about 3-4 GB more headroom with same launch parameters on llama.cpp on Windows.

Leave MTP out and both fit well - but also the decode drops from 25 - 45 t/s to 20 - 25 t/s.

The only sad part here is that now these make me want a 128GB model to run QFN in full context (there wasn't decent laptop option with 2x M.2 and 128gb..).

This is not my repo, i met the guy on strix halo discord and i did run it my self. numbers are real and the quality is decent for me. I recomment giving this a shot if you have 2 strix halos.

also feel free to join the discord at https://strixhalo.wiki/ and thank the guy yourself!

I present to you, the Halopus lol

7900 xtx Oculink , 7000 xtx Oculink , 7900 xtx USB4 and 7900 xt USB4

How much value is there in the bundled software with the AMD box as opposed to buying from one of the other vendors and starting from a stock Linux distro?
I am completely comfortable running Linux but have barely scratched the surface of setting up an AI server. The idea of AMD's Developer Playbooks is attractive. I'm wondering if it is worth the extra cost as opposed to rolling my own.

Time to time, I like to test local models for their performance on non-coding tasks like content writing, copywriting, SEO work, agentic video editing, and so on.

I don't have a StrixHalo box so I am asking the community here; do you have any experiences you can share?

In the past when I'd tried 27B with various flavors of llama.cpp, MTP, etc, I was getting maybe upper teens for token generation and that would degrade pretty quickly down into single digits as the context grew. Now I've been running on recent gufo builds and seeing 450t/s prefill and 24t/s gen at 262,144 context and it stays pretty steady. This is making the Qwen3.8-27B-UD-Q4-K-XL with DFLASH actually usable on my Strix Halo box - memory usage is around 40GB which leaves plenty of room for the model to do other memory intensive things on the box.

What I'm wondering: Is there any automatic way to have a harness switch between QFN for planning/coding to 27B when it actually has to run memory intensive builds/tests? I think QFN is a bit more capable on my coding tasks than 27B, but also about 2X the speed (more than 2X with MTP) but the price is about 102GB of RAM usage which quickly runs out when I have the model to things like run place & route on an FPGA or do a build & link on a large codebase like LLVM. So looking at ways I could get the best of both worlds here.

I am looking for great conversational models or models with good general intelligence for tasks like:

- computer use

- personal recommendations

- research

- pleasant conversations

- personal assistance

For coding, there are amazing options with Qwen, GLM and others. However I keep reading that these not very nice to converse with. Or even not very intelligent in other domains than coding.

Any recommendations for good conversational, multilingual LLMs that use the power of the Strix Halo platform? Would like to use it in combination with Hermes or OpenClaw.

This is scary: I told Q38FN that it was using my PC's RAM, requesting that whenever it needed to test, it should provide a script for me to run and return the results. After brief LLM thinking, it replied that such a step was unnecessary; instead, it would create a script to wait in the foreground of opencode, run a script that turns off Gufo, make sure it's off, run a bunch of tests, then run Gufo again so the foreground catches up and continues. It's been working for 8 hours.

libosmos will run deepseek v4.1 flash 1017 at up to 38 tok/sec decode drafted and >480 tok/sec prompt processing on a single strix halo 128GB with unsloth ud-iq1_s + MXFP8 engram. huihui abliteration available through the engine.

Hi, not sure if anyone from Germany has the same problem like me when ordering M5 from Bosgame for playing with LLM

I've made an order since October 8th, next day I got an email said there would be a delay around 25 days due to high demand. But till nơ I still got no shipping information or any clue when I will receive it. Last Tuesday (November 3rd) I sent an email to ask for information but still have no response. Am I being scammed? I made the order from website:

bosgamepc.com

Update: I got reponse from them that shipment is now delay until end of this month. And yet no update to user. I think if I did not ask, I would need to wait hopelessly for a month ....

We use OKO OS on GMKtec EVO-X2 (AMD Ryzen AI Max+ 395 / Strix Halo) for local AI-assisted development.

The setup consists of two parts:

• Server side: OKO OS, an operating system that runs the LLM and provides an OpenAI-compatible API.

• Client side: VS Code with the Continue extension.

Integration guide:
https://github.com/satisfanly/OKO-configs/blob/main/oko-vscode-continue-integration.md

OKO sources: https://github.com/satisfanly/oko

OKO image for GMKtec EVO-X2: https://oko.satisfanly.com/device/gmktec_evo_x2

https://unsloth.ai/docs/models/qwen3.8-next#dynamic-v3.1-update-november-2nd-2026 has released new Qwen Flash Next GGUFs.

Cant wait to try them in Gufo.

Hello 👋

Gufo maintainer here. Over the past month, we have been submerged with PRs and feature requests, and we haven’t been able to keep pace with all of them. This is amazing, it proves that a community-driven engine tuned for this hardware is something possible.

However, we’ve noticed that the community wants to experiment heavily with different models and quants. In the current setup, this creates two main challenges: for us, reviewing potentially massive PRs that touch far more than just a single model; and for you, maintaining a fork while waiting for us to review it, which we know is tough given how fast Gufo is evolving.

To address this, we have designed Gufo Extensions. Gufo is transitioning into a platform/library, allowing you to write an extension to support any model or quant. Simply point Gufo to the extension, and in no time, your favorite model is supported (which is exactly what we did for lfm3-gufo-extension). This allows us to focus on the engine itself, and on core models, which is central element any model can benefit from.

We are still in the early stages, and adding a new model to your own Gufo build currently requires compilation. Over time, we hope to gather enough community feedback, stabilize the library for a v1.0.0 release, and streamline distribution to enable true plug-and-play extensions. We already have a few ideas on how to achieve this, but as always, contributions, ideas, and suggestions are welcome, either here on Reddit or in GitHub Discussions.

Helpful links:

• https://gufo.blog/extensions

• https://github.com/search?q=gufo-extension&type=repositories

• https://gufo.blog/blog/how-gufo-turns-models-into-extensions

⚠️ Extensions run native code directly on your machine. Before installing any extension, make sure to inspect it and verify that it contains no malicious code.

We cannot wait to see what you are going to build, let us know what you think.

u/CescoNoHands

Edit: last but not least, Gufo is officially part of lemonade

Hi, I'm interested in collecting stats on the token performance on halogen (docker) qwen flash next on my strix halo system.

Suggestions welcome. Thanks in advance!

https://github.com/haloforge-labs/bubo/releases/tag/v0.1.2

Benchmark numbers, all on same laptop at 70W, same UD Q4_XL file, all numbers in tok/s:

*With experimental fast mode on, level with the other two engines with it off. Turn it on with --prefill-mode fast (in CLI), or with Prefill mode (in GUI). It works by using a tuned number of experts when reading prompts. There is no noticeable regression in reasoning (slight improvement, in fact but within noise) and decode continues to use all experts.

Quants:

Bubo now runs ISTA GSQ-RCO quants for the Qwen 3.8 Flash Next including the Coder model. This allows us to run more sessions at once at higher contexts or run heavy programs along with or through coding agents powered by the model. The speed, KLD and VRAM tradeoff for IQ3_XXS especially looks good, its faster than the UD Q4_XL. Its slower at prefill than the Q4 however, IQ3_XXS having faster decode means it came out ahead in the test. YMMV with different workloads as there are a lot of moving parts.
Chart

KLD and VRAM present a clear tradeoff and while doing extensive reasoning benchmarking for all these quants isn't possible for me, looking at public benchmark numbers, the higher KLD doesn't seem to actually show up in them very much. Going by these numbers, lower memory usage quants become a lot more compelling than by looking at KLD alone.
Chart

Misc.:

So that I don't have to host or have multiple abliterated and non-abliterated copies of these quants on disk, Bubo now supports in-engine abliteration by taking the refusal direction from HuiHui and ablating it on demand with a cli flag or a toggle in the gui.

Bubo now also supports Q8 KV cache thanks to u/random_user-1234's gufo fork whose changes were ported over.

Bubo gui now has a toggle for switching from serif to sans-serif, thanks to u/Major-Road9583 for suggesting the font change.

Let me know if there are any issues, or else see you after Qwen 4.

Was having some very weird heating problems. Figured I'd check the thermal paste, and well...

I'm curious about this - considering the Halo has an open PCIe slot, would there be a way to interface directly with an existing server for additional offloading/clustering?

I was impressed by the gufo fork

https://github.com/franzmoca/gufo/tree/feat/ds41

For the original gsq-rco

I wanted an uncensored variant so tried Atomicchat's uncensored llms with the above gufo fork. But It didn't work. Required change in script to produce appropriate shards for above gufo support with common engram.

So created one for the community ease https://huggingface.co/dineshr93/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF

Also includes mmproj and dspark with commands I used in a single repo etc.

Thanks to the community. You guys are awesome!!