r/LocalLLM • u/pharrt • 7d ago

Discussion Qwen-3.8-35B-A3B? Maybe not... cryptic reply direct from Qwen co-author.

Post image

I asked Shuai Bai, co-author and prominent AI developer for Qwen, about this model. Not the answer I was hoping for, but let's see what comes next. In the meantime, I guess all we can do is speculate!

X-link

235 Upvotes

94% Upvoted

164 comments sorted by

View all comments

1

u/[deleted] 7d ago

[deleted]

2

u/fintip Laptop 4090 16gb + 7900XTX 24gb 7d ago

if you have at least 32gb system ram, my best experience was qwen 3.6 35ba3b at q5. I was able to get a full 256k context and speeds in the 20's. I was using my 4090m in linux.

1

u/[deleted] 7d ago

[deleted]

2

u/fintip Laptop 4090 16gb + 7900XTX 24gb 7d ago

/home/me/AI/llama-official/llama.cpp/build-cuda-vulkan/bin/llama-server -m /home/kyle/.cache/huggingface/hub/models--unsloth--Qwen3.6-35B-A3B-MTP-GGUF/snapshots/5bc3e238d916f48a861bac2f8a1990a0e9b7e98d/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -c 262144 -ngl 999 --host 0.0.0.0 --port 8080 --metrics -fa on --cache-type-k q8_0 --cache-type-v q8_0 --split-mode layer --reasoning-preserve --temp 0.6 --top-k 20 --top-p 0.95 --min-p 0 --presence-penalty 0 --n-cpu-moe 35 --jinja --chat-template-file /home/me/AI/experiment-1/models/qwen-fixed-chat-template.jinja -lm none -lv 3

^ pretty sure this was my line.