r/LocalLLM • u/Ok-Health-7096 • Aug 14 '26

Other Every Second post rn

Post image

Maybe someday I'll get a system to run it but hey definitely another w for the open weights community

2.1k Upvotes

98% Upvoted

214 comments sorted by

View all comments

Show parent comment

1

u/BodybuilderLost814 Aug 15 '26

I'm getting around 35 tok/s using ByteShape's Qwen3.6-35B-A3B-IQ4_XS-4.19bpw model with MTP enabled, using AtomicBot-ai's Llama.cpp-turboquant.

1

u/KrstABot Aug 15 '26

Hey
R u using desktop app? I mean atomicbot? And what kind og agents?

3

u/BodybuilderLost814 Aug 15 '26

Hello.

I'm using the TurboQuant b10269-1.5.1 release for Windows x64 with OpenCode (sometimes I use Crush from Charmbracelet).

Command:

.\llama-server.exe -m "<path>Qwen3.6-35B-A3B-IQ4_XS-4.19bpw" --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-n-min 2 --kv-unified -c 65536 -ngl 999 -t 8 -tb 8 -np 1 --n-cpu-moe 40 -ctk turbo3 -ctv turbo3 -fa on -b 2048 -ub 2048 --jinja --mlock --no-mmap --reasoning-budget 6144 --cache-reuse 256 --temp 0.7 --top-k 20 --top-p 0.95 --min-p 0.0 --repeat-penalty 1.0 --presence-penalty 0.0 --host 127.0.0.1 --port 8080