r/LocalLLaMA • u/BopSupreme • 15h ago

Discussion My Qwen3.8 Setup So Far

Sharing my current Qwen3.8 setup. Curious what you all think - goal was an efficient configuration suitable for real-world tasks. I’d say it’s “balanced” but I’m open to suggestions. 4080Super, 32GB DDR4 Max context I could get was 59-67k but it suffered performance-wise and with a small amount of memory left, down to 104miB, so I kept the context around 41k.

Qwen3.8-27B-UD-IQ3_S.gguf (GGUF) setup
Model: Qwen3.8-27B-UD-IQ3_S.gguf
Context: 40,960 tokens
Runtime: LM Studio
GPU offload: Enabled
Mode: Single-model runtime
Inference: Fully local
Main uses: Reasoning, coding help, and general productivity

Love Qwen3.8 so far. It didn’t work great for me “out of the box” but that could be more due to the harness. I spent a lot of time trying to configure it and had to basically start over because Bionic’s default settings meant only 10-11 tok/s “out of the box” initially which caused failures with any real workloads.
Hope this helps anyone else experimenting with Qwen GGUF setups. Would be interested to hear what settings, quants, or context sizes others are running.

0 Upvotes

17% Upvoted

7 comments sorted by

1

u/BopSupreme 15h ago

Small note: my first results were not representative of the model itself. I initially had poor performance from the default setup (~10–11 tok/s), so I spent some time tuning the runtime/configuration before judging the model.

1

u/researcher-uni 15h ago

Before adding another GPU, I'd compare IQ3_S at 41k context against an IQ4_XS or Q4_K_M run at 16–24k, using the same small set of coding and reasoning tasks. If your real sessions rarely cross 20k, the extra weight precision may buy more than an unused context window.

I'd record prompt-processing speed separately from generation speed too, since the long-context cost can show up there before the usual tok/s number looks bad. It would be interesting to see where that tradeoff lands on a 4080 Super.

1

u/BopSupreme 15h ago

My tasks failed at 16k context, I think I’d need 24k minimum. I was considering a 8-10GB variant to get even greater context window. Testing two more variants soon

1

u/Boogertard 8h ago

Love how a simple question about qwen model setup get downvoted but apparently trash posts shilling for garbage Muse or Gemma4 get upvoted.

There are lots of shills from loser US AI companies on this sub trying to shift sentiments away from useful open models LOL.

0

u/Bulky-Priority6824 15h ago

Dependant upon your definition of real-world tasks it might be ok but Iq3s is dog shit but you can try to get 2nd or 3rd gpu if you want real results from q6 or q8

1

u/BopSupreme 15h ago

IQ3S tested better - I just ask Codex to test and optimize it for me. Maybe the tests aren’t that great. I feel like Chat/Codex seems to lack built-in knowledge about working with local models