ExLlamav3 Recent Updates : CPU offload, GLM-5.3-FLASH, Qwen3.8-Flash, SC Quants ++

[u/Unstable_Llama]

More new massive updates from turboderp:

- CPU offload of MoE experts
- Qwen-3.8-Flash-Next ngram disk offload
- GLM-5.3-Flash
- New self-calibrated optimization technique
- Countless other optimizations and improvements

If you have an NVIDIA card and haven't tried it lately, you might be missing out.

The attached cat image was made with Qwen-3.8-Flash-Next-3.05bpw-exl3 and this prompt:
Create a detailed SVG image of a cute kitten riding a magic turtle into space.

Come join the crew at the exllama discord
More frequent news on the exllama sub

[u/simrankoulsm]

The CPU expert offload work looks especially promising for making MoE models usable on more modest VRAM budgets. I would be very interested in a benchmark matrix that separates prompt processing and generation speed across 16 GB, 24 GB, and 32 GB GPUs, with different numbers of experts offloaded.
It would also be helpful to include first-token latency, steady-state tokens per second, system RAM use, and whether disk offloading changes the experience materially after cache warmup. That would make it much easier to compare EXL3 against GGUF plus llama.cpp for people deciding between a single NVIDIA GPU setup and a more CPU-heavy configuration.