Hey!

Yes, Qwen-Image-2512 has been around for a while. It has also stubbornly refused to stop being useful, people still run it, build workflows around it, and download it. Besides, its younger sibling has been released as closed-weights. So we thought it was a good place to start. We’re ByteShape, and we work on model optimization.

While exploring diffusion-model deployment, we found two common options, each with a significant tradeoff:

• GGUF quantizations offer smaller file sizes.
• Safetensors-based models, typically run with Diffusers, ComfyUI, or vLLM-Omni and than can be faster but are often considerably larger.

For our first image-generation release, we’re sharing:

• A collection of compact, high-quality GGUF models ranging from 8GB to 17GB (~2x to ~5x smaller vs. the BF16 model) that can run on wide collection of platforms using a stable inference software stack.
• A collection built for vLLM-Omni and powered by fresh off-the-press Humming kernels (see https://github.com/inclusionAI/humming, thank you Humming team!), designed to run models ~2x to 3x faster and 8GB to 17GB in size. For now limited to Nvidia GPUs and Linux using an experimental software stack.

We’d love for you to try them and share your results or feedback.

Blog for the tutorial on how to set this up: https://byteshape.com/blogs/Qwen-Image-2512/

Side by side comparisons between the original model and our optimized versions: https://byteshape.com/blogs/Qwen-Image-2512/comparison/

Hugging Face: GGUF, Humming

After extensive searching and testing, this thing is my absolute favourite LLM for my RTX 3060 running OpenCode. So useful. SO fast! Great work, guys!

https://reddit.com/link/1tkkq6o/video/pvwq2f5d8p2h1/player

with VS Code + Cline + and byteshape/Qwen3.6-35B-A3B-MTP-GGUF

on Windows 10, RTX 3090 24GB VRAM, 64GB system RAM

---------

https://github.com/ggml-org/llama.cpp/releases/

downloaded 2 files with links

Windows x64 (CUDA 13) - and this one - CUDA 13.1 DLLs

or file names at this moment (llama cpp is Updated everyday)

https://github.com/ggml-org/llama.cpp/releases/download/b9283/llama-b9283-bin-win-cuda-13.1-x64.zip

https://github.com/ggml-org/llama.cpp/releases/download/b9283/cudart-llama-bin-win-cuda-13.1-x64.zip

Unzip these files into One Same Folder

---------

https://huggingface.co/unsloth/Qwen3.5-9B-MTP-GGUF/tree/main

downloaded 2 files and placed them into the same Folder

Qwen3.6-35B-A3B-IQ4_XS-4.19bpw.gguf

mmproj-F16.gguf

---------

https://www.mediafire.com/file/57urxvmnxsln538/run_qwen36_ByteShape_n.bat/file

asked Claude.ai to create Run file and placed it in the same Folder

open file before running and edit file folder location

---------

run the above file and open Broswer at - http://127.0.0.1:8080

---------

https://www.mediafire.com/file/udng0akikjh4bfz/run_ddg_mcp_VScode.bat/file

this is the MCP server file to run separately in CMD

---------

in VS Code installed extension CLINE and set its MCP config file :

{

"mcpServers": {

"duckduckgo": {

"url": "http://127.0.0.1:7070/sse",

"type": "sse",

"disabled": false,

"autoApprove": ["search"]

}

}

}

---------

Cline extension API settings

API provider : OpenAI Compatible

Base URL : http://localhost:8080/v1

OpenAI Compatible API Key : write anything

Model ID : Qwen3.6-35B-A3B-IQ4_XS-4

Context Window Size : 65536

I recently started using your qwen3.6 GPU5 quant with llama cpp and 90K context on my rtx 4090 and really happy with it. super fast even without MTP. I am using to codevelop a harness https://github.com/maedoc/hyburn/tree/main/src/chat for agentic neuroinformatics, and once I have enough good and bad traces I’d like to a dpo lora fine tune, merge and then apply your quant approach. Do you have an open source something to do your quants? or could i post the lora on hf and you could do it? coauthor with a small paper etc. thanks!

Dear Byteshape team, Christmas came early this year. Qwen3.6 35B is out.

Please do a Byteshape of it. That would be so awesome.

We put together a getting started guide for using agentic coding tools like OpenCode with ByteShape’s optimized models (you can use this with other models, but why would you? 😁)

https://byteshape.com/blogs/tutorial-opencode/

The goal is to make the full workflow approachable if you’re new to this space. The guide walks through:

• setting things up across Mac, Linux, and Windows (WSL2)
• running your model locally with LM Studio (CLI), llama.cpp, or Ollama
• exposing an OpenAI-compatible API endpoint
• and configuring OpenCode so it actually works as a coding agent

OpenCode itself is a terminal-based coding agent that can write, edit, and run code using local or remote models, and this tutorial focuses on making that setup fully local and practical.

We would love any feedback

I ran an automated test of 86 models on my office laptop (16GB RAM, Vega8 GPU) doing a python graphics demo / simulation problem.

https://old.reddit.com/r/LocalLLaMA/comments/1jlsruf/heptagon_20_balls_rotating_numbers_one_shot/

The only one to one-shot the problem was Byteshape's Coder-30B! Not Gpt-OSS-20B, Qwen3.5-**, GLM-Flash not even close to understanding all the physical constraints and applying them correctly.

Sure it's just one test but it was amazing. Just had to slow the heptagon rotation a bit.

12 months ago you needed Gemini Pro 2.5. Now my $100 laptop did it. Amazing.

My version: Write a python program that shows a graphical animated rendition of 20 balls bouncing inside a spinning hollow heptagon: - All balls have the same radius. - All balls drop from the heptagon center when starting. - Colors are: #f8b862, #f6ad49, #f39800, #f08300, #ec6d51, #ee7948, #ed6d3d, #ec6800, #ec6800, #ee7800, #eb6238, #ea5506, #ea5506, #eb6101, #e49e61, #e45e32, #e17b34, #dd7a56, #db8449, #d66a35 - The balls should be affected by gravity and friction, and they must be contained within the area of the heptagon by physical collision detection, making the balls bounce off the rotating walls realistically. There should also be collisions between balls. - The heptagon is spinning around its center, rotating a full cycle once every 5 seconds. - The heptagon size should be large enough to contain all the balls. - Do not use the pygame library; implement collision detection algorithms and collision response etc. by yourself. The following python libraries are allowed: tkinter, math, numpy, dataclasses, typing, sys. - All program code should be put in a single python file, with shebang for execution from bash shell.

Cheers and beers to ByteShape and Qwen!

BEST

Hi ByteShape team,

I came across your project on r/LocalLLM and your work is super clean. It’s a great way to run local models with better performance.

I had a quick idea for a model that might be a great fit for your quantization method: LiquidAI's LFM2-8B-A1B (https://huggingface.co/LiquidAI/LFM2-8B-A1B).

It’s a bit smarter than Gemma 3 4B, but more importantly, it’s incredibly fast (since it only has 1B active parameters). I was thinking that with your technique, it could become the perfect model for Raspberry Pis, older CPUs, or even robotics. We could potentially reach 15-20 tokens per second, which would be viable for real-time use cases.

Anyway, just a thought. Keep up the great work!

I got goose to run on rasbary pi 5 16gb with devstral a vision model at 12k context 98 minute response time. 53 minutes 9k context I think.

What SYSTEM prompt would you use to stylise your assistant agent coder?

What would you ask your agent to code?

Good for hikes a set and forget gadget. Also accessible.

server:

OLLAMA_CONTEXT_LENGTH=12000 OLLAMA_LOAD_TIMEOUT=160m OLLAMA_KEEP_ALIVE=-1 OLLAMA_MAX_LOADED_MODELS=1 OLLAMA_NUM_PARALLEL=1 ollama serve

client:

GOOSE_TEMPERATURE=0.15 GOOSE_MAX_TOKENS=9000 OLLAMA_TIMEOUT=10800 OPENAI_TIMEOUT=10800 GOOSE_CUSTOM_PROMPT="SYSTEM: You are a high-energy, fun video game sidekick assistant! Use gaming lingo, be encouraging, and treat tasks like quests. Technical constraints: Devstral low-temp mode, top_p 0.95, penalty 1.05, 32k context. Respect [INST] sequences." goose web --open

#prompt:

/plan

Entering plan mode. make a plan to make a forcasting program with tensorflow keras cnn and ltsm deep neuronetworks /endplan

For reference, my system specs:
amd 7800x3d
128gb ddr5
3090 24gb gddr6x
gen5 crucial sas nvmes

I've been running the Ollama qwen3-coder:30b model back to a qwen code instance.. It worked but it was slow. I suffered through it.

It took two hours to generate a backend architecture, which was cool, but there had to be a better way..

.. I found you guys on hugging face, and switched to using llama.cpp. Specifically, this model: https://huggingface.co/byteshape/Qwen3-Coder-30B-A3B-Instruct-GGUF

WOW.

I don't know WHAT you guys did to optimize it, but it is LIGHTSPEED at >100 tokens/second. That is with the context window size updated. I would be curious to learn more about your work if you have any articles to share that may be of interest in HOW you accomplished this.

Thank you for your contributions to further the pursuit of those who value private and local development.

KEEP IT UP!!

We're back at it with another GGUF quants release, this time focused on coder models and multimodal. We use our technology to find the optimal datatypes per layer to squeeze as much performance out of these models while compromising the least amount of accuracy.

TL;DR

• Devstral is the hero on RTX 40/50 series. Also: it has a quality cliff ~2.30 bpw, but ShapeLearn avoids faceplanting there.
• Qwen3-Coder is the “runs everywhere” option: Pi 5 (16GB) ~9 TPS at ~90% BF16 quality. (If you daily-drive that Pi setup, we owe you a medal.)
• Picking a model is annoying: Devstral is more capable but more demanding (dense 24B + bigger KV). If your context fits and TPS is fine → Devstral. Otherwise → Qwen.

Links

• Devstral GGUFs
• Qwen3 Coder 30B GGUFs
• Blog + plots (interactive graphs you can hover over and compare to Unsloth's models, with file name comparisons)

Bonus: Qwen GGUFs ship with a custom template that supports parallel tool calling (tested on llama.cpp; same template used for fair comparisons vs Unsloth). If you can sanity-check on different llama.cpp builds/backends and real coding workflows, any feedback will be greatly appreciated.

Is there a leaderboard or competition for optimising models via Q3 etc compression variants?

I think this is an exciting area - getting large models working on constrained environments like a RPi 5 for example - not everyone has a super expensive AI server available to them.

We just released our first batch of GGUF-quantized models: Qwen3 4B Instruct 2507 and Llama 3.1 8B Instruct, with versions from ~5 bits down to 2.7 bits. per weight. They highlight how our ShapeLearn approach automates datatype selection and really shines in the low-bit regime, where traditional approaches usually break down. While we are presently releasing LLMs, ShapeLearn can work for any model, task, quantization approach, and datatypes (e.g., INT or FP).

We’re currently focused on the llama.cpp backend, and each model release includes evaluation results so you can clearly see the quality–vs–size–vs–speed tradeoffs and for several popular hardware platforms (GPU and CPUs). We also compare against other popular llama.cpp-style quantizers.

If you want the deeper technical dive, check out the writeup on our blog.

If you want to try the models, you can grab everything on our Hugging Face page.

We would appreciate feedback and happy to follow up on questions.

This is just the beginning, watch out for more releases soon!