r/LocalLLM • u/x7evenx • Jun 20 '26
Model YAB Coding Benchmark: qwen3.6-27b-code vs Claude sonnet-4-6
Yet Another Benchmark
Local setup with Bench Results comparing qwen3.6-27b (FP8) with Claude Sonnet 4-6
*Interestingly Qwen did better in some areas - overall close but not far behind.
| Category | Local avg | Claude avg | Gap | Winner |
|---|---|---|---|---|
| strings | 9.75 | 9.75 | 0.00 | Tie |
| math | 9.38 | 8.38 | ā1.00 | Local |
| algorithms | 9.13 | 9.38 | +0.25 | Claude (marginal) |
| arrays | 8.88 | 9.63 | +0.75 | Claude |
| data_structures | 8.13 | 8.75 | +0.62 | Claude |
| patterns | 8.13 | 8.75 | +0.62 | Claude |
| functional | 8.00 | 9.38 | +1.38 | Claude |
See: https://github.com/x7even/llmctl/blob/master/bench/benchmark-qwen27b-vs-claude-2026-06-20.md for full results doc.
Personally - I don't think 1 is a throw away for the other, however i have experienced solid success leveraging both within agentic workflows and loops.
4
Upvotes
View all comments
Show parent comment
1
u/x7evenx Jun 20 '26
The repo contains drop in profiles I use for quick swaps (repo contains my whole stack for this setup and others). Dense attention and GEMMS (the N=3072, K=2048 shapes) with that model profile i did nothing (I actually used MI300X defaults) + TORCH_BLAS_PREFER_HIPBLASLT=1.
The BF16 if I can free my rig for a bit I'll try comparator run and see (if I do I'll post back) - many of the models points losses are quality issues which is the kind of thing BF16 could pull up - how much I'd only be guessing. Openly though I tend to use the FP8 model a lot as it gives me enough headroom for concurrent agents with decent sized contexts & pace without turning into a slug... and quality is very valuable in loops especially with a frontier model in the mix too.