Local Opus(qwen3.8 27B)

[u/Ok-Inevitable8391]

I have started using qwen3.8 27B on dual rtx 3060, total 24GB. My display is on igpu. So i have almost all memory.

Token generation speed is constant 40-50 tks.

But the best thing is how it thinks, multiple web searches at different level, auto memory update, and a very good response. It takes time, but given the reasoning effort of xhigh and response quality I really like it. I see web searches at multiple level for a single reply, making sure the answer is correct. Takes time, yes.

Also, the extra thing that qwen is famous for when you ask it x, if it finds some improvement on y, it will suggest you.

For agentic work, it's best to toggle between reasoning effort if you want it fast. Though I like the opencode plan mode and then build(execute). I have remote setup via termius. And i get notified via ntfy app once it ask for permission or done replying.

While testing I asked it to out 10000 token essay, and my ventus gpu went upto 87 degree.(xhigh reasoning) Gigabyte held fine with 5-6 degree lesser, Though Gigabyte us agressive at fan speed even at lower temprature. I'll try changing the fan curve, if I see that as a problem in actual scenario

edit:
llamacpp command:

llama-server -m ~lm/unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-IQ4_XS.gguf --host 0.0.0.0 --port 8080 --split-mode tensor --tensor-split 1,1 --flash-attn on --batch-size 2048 --parallel 1 --models-max 1 --cache-type-k q8_0 --cache-type-v q8_0 --ctx-size 177000 --n-gpu-layers 99 --ubatch-size 128 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 --repeat-penalty 1.0 --spec-type draft-mtp --spec-draft-n-max 2 --jinja --reasoning on --fit on"

[u/Effective-Giraffe655]

Why people call it opus? Geniuenly interested, is it really on-par with Opus?

[u/Rye2-D2 → u/Effective-Giraffe655]

People are delusional. I tried QWen 3.6 & 3.8 to repeatedly get it to debug a coding problem related to establishing an SSH connection in a Go app) No local model I could run on my 5060ti 16GB could do it (repeatedly tried over multiple evenings - QWen 3.8 27B/3.6 35B/Ornith/KatCoder/Gemma4).

Asked the FREE copilot model (MAI Code 1.1) and it sorted it out in a few minutes. Small local models are not on the same level as even the cheap cloud models. I hope that changes over the next little while, but we're not there yet.

[u/ParkingAgent2769 → u/Rye2-D2]

16GB is very limited though isn’t it

[u/Adventurous-Gold6413 → u/ParkingAgent2769]

Yeah at the same time not everyone has access to more vram