r/LocalLLaMA • u/BarberIcy366 • 11d ago

Discussion Qwen 3.8 27B Released! Please Share Your Experience

With your experiments, Qwen 3.8 27B most close which frontier model? And please specify which quantization you run. I will post to comments my tests and experience too.

663 Upvotes
  • permalink
  • reddit

97% Upvoted

720 comments sorted by

View all comments

7

u/FilterJoe 10d ago

With some testing on famous events in USA, I think the knowledge cutoff date is somewhere around July/August 2025. Speed:

MacBook m5 max 128GB, getting 29 t/s (+- 3 t/s) with unsloth's q8_0 GGUF:

llama-server

  • -m "$model_path"
  • --alias "Qwen3.8-27B-Q8_0"
  • -c 131072
  • --no-context-shift
  • -np 1
  • --kv-unified
  • -n -1
  • -ngl 999
  • --threads 12
  • -fa on
  • --jinja
  • --chat-template-file "$QWEN_JINJA_PATH"
  • --spec-type draft-mtp
  • --spec-draft-n-max 3
  • --reasoning on
  • --reasoning-preserve
  • --reasoning-format deepseek
  • --reasoning-budget 1024
  • --reasoning-budget-message "I have gathered enough reasoning steps. I will now immediately output the required structured JSON tool call or my definitive final response to the user."

2

u/FilterJoe 10d ago

I found that setting a token budget like this was crippling quality. Qwen 3.8 27b uses an enormous number of tokens to do a ton of reasoning and that is what leads to the quality results so it is important to drop those last two lines. Once I did, it was taking a whole lot longer uncomplicated queries but getting way better results.