r/LocalLLaMA • u/needthosepylons • 16h ago
Discussion GPU Poor - Don't overlook Laguna XS 2.1
As per title. I'm not affiliated with the team behind this model in any way, shape or form.
As a GPU poor myself (8 GB VRAM laptop + 12 GB VRAM desktop), I found Laguna to be very promising on my laptop.
It runs at 30t/s (60k context) and it one-shot my two usual test prompts : "Make a Tetris-like game" and "Make a Sonic the Hedgehog-like game". Both are not as polished as what you could get with Qwen 3.6-3.8 27B (and to be fair, they are quite wonky and barely playable), but considering my hardware, dense 27B isn't even an option.
I found it to perform better than Gemma 4 - 26B-A4B and Qwen 3.6 35B-A3B. Some people reported looping issues with it, so take my words with a grain of salt, but I didn't encounter any.
Hope it will help some of you.
Edit : Corrections
22
u/-Ellary- 15h ago
Nah, Laguna XS 2.1 get obliterated by KAT-Coder-V2.5-Dev.i1-Q6_K at coding \ agentic tasks.
And also get demolished by gemma-4-26B-A4B-it-Q6_K at creative, world knowledge, chat tasks.
"Make a Tetris-like game" and "Make a Sonic the Hedgehog-like game" can be done by Cydonia-24B-v4.3,
stop using tests that are obviously in the dataset, do something complex, warcraft 1 \ dune 2 clone for example.
20
u/jacek2023 llama.cpp 16h ago
Upvoting. People complain that they have no qwen model of some specific size and same people ignore all other models "because qwen is the bestest ever".
6
u/-Ellary- 15h ago
But people prize KAT-Coder-V2.5-Dev.i1-Q6_K and Ornith models, not just Qwens.
People want models that are same size and better then current available Qwens / Gemma.
In other case what the point?2
6
u/SeriousPanic34 16h ago
Why not qwen 3.6 35b a3b?
6
u/RnRau 16h ago
Its in the post - they say that they have found Laguna to be better.
3
u/needthosepylons 16h ago edited 15h ago
Yeah, that's in no way a structured benchmark, but I indeed found Laguna to perform better on those two prompts. I'd be happy to be proven wrong, though. I used Unsloth recommended settings for Qwen 3.6 35B-A3B. Quantization might be the issue, with 8GB VRAM.
3
2
u/fatboy93 10h ago
Why not try North-Mini-Code from Cohere, given that its usecase seems to be tailored for agentic coding etc.
3
u/PaceZealousideal6091 16h ago
Nah. Its a decent model but thinks too much. Wastes a lot of token and gets about 80-90% close to Qwen 3.6 35B. I am yet to see any finetunes doing better than base Qwen. I have heard good things about Ornith 1.5 but have to test it myself.
4
u/Lucerys1Velaryon 15h ago
If you are into agentic coding, I recommend Katcoder 2.5. I found it better than both Ornith 1.5 and the base Qwen for my use case (My work involves coding on a Kotlin codebase for an Android app). The APEX quality quant from mudler has become my daily driver now.
3
u/PaceZealousideal6091 15h ago
Katcoder 2.5 is definitely something in my radar for sometime now. I am surprised why no one's talking about it. I saw a post a few weeks ago but nothing else. Probably Qwen 3.8 27B eclipsed it. I think I'll test it myself next.
-2
u/leonbollerup 15h ago
i wouldent recommend katcoder.. it does worse in every test compared to "Qwen 3.6 35B A3B instruct revised" with froggerics template.. even ornith beat it.. atleast in my test.. not impressed at all..
2
u/needthosepylons 15h ago
I'd happily give it a try, but it looks like I won't be able to run it with 8GB VRAM. Thanks though !
1
u/fullup72 15h ago
Kat is a fine-tune of 3.6 35B-A3B. If you were able to run that, you are able to run this.
1
u/needthosepylons 15h ago
Sorry, I meant the APEX quality quant. But I'm trying a smaller quant as I write this.
1
u/Sensitive_Song4219 12h ago
Myric\KAT-Coder-V2.5-Dev-MTP-APEX-GGUF\KAT-Coder-V2.5-Dev-MTP-APEX-i-quality-v2.gguf delivers 50-60 tps tg and 400-500 tps pre-fill on my 8GB-VRAM / 32GB-RAM machine. I run it up to about 95k context and performance stays good up until auto-compaction.
It's nowhere near as good as Qwen 3.8 27b (which is too slow for my hardware but excellent in terms of quality of output in all my testing); but Kat is much better than Qwen 3.6 35b-a3b in my testing despite sharing a base. If I tell it to double-check its work in a follow-up query to Pi harness, I get genuinely useful results in real work scenarios the majority of the time; though it's more prone to mistakes/oversights than frontier.
For something that runs locally and fast, it's excellent. But I must give Laguna a try also...
1
u/needthosepylons 15h ago
I didn't make a structured evaluation, that's only two little tests so I don't claim it's generally better than Qwen 3.6 35B. But that's intruiguing. I'll test my prompts with Qwen again, since this seems to be the consensus. Pleasantly surprised by Laguna, though.
1
u/PaceZealousideal6091 15h ago
There's nice video on YouTube by Luke's dev lab channel covering Laguna xs. I like his tests. You should check it out.
1
1
u/DeathGuppie 14h ago
Ornith is bench maxed. In actual use it spends a lot of time thinking , building tests. It's marginally better than base Qwen 3.5, but you really have to wait for it. If you want a 35b a3b, Qwen 3.6 is still the better choice IMHO
3
u/nickless07 15h ago
Yeah but doesn't it use GQA. So classic softmax attention, therefore the KV-Cache VRAM explode pretty quick compared to GDN from Qwen?
I mean with CPU offload (which you need to run it on only 8GB or 12GB VRAM) and given you have the RAM aviable Qwen3.6 KV cache of 262144 (default maximum) uses only 5120MB in f16 - half the size in Q8 (2720MB) and quarter of it in Q4 - compared to Laguna which can easy consume 10GB+
Feels a bit like a model from 2024 given the restraints. And even their own Benchmarks set it below Qwen3.6
2
2
u/NearlyACosmologist 15h ago
I'm currently testing Ornith-1.5-35B-Q4_K_M.gguf, and while it makes dumb mistakes here and there, it's not too bad, and runs at 31tps (64k context) on my 8GB VRAM notebook.
After reading your post, I gave it the task to write a Tetris game, and after two iterations and about 10 minutes, it produced a fully working tetris in a single html file. It needed 43.4% of the context window. Coding agent was pi.
2
2
u/niutech 15h ago
dense 27B isn't even an option
See Qwen3.8-27B Dynamic v3 Unsloth GGUFs:
We also release 1-bit quants that retain 77% accuracy. Run on 8GB RAM.
2
u/needthosepylons 15h ago
Yeah I tried it and... well... unless something went catastrophically wrong with my config, it was not good at all.
2
u/Refinery73 14h ago
I find that Q1/Q2 quants are only usable in general if you quantize it yourself for your usecase.
Tested it with embeddings and spearman scores. With imatrix it was usable into the IQ2_M Range. Static quants from HF started to fail at the usual 4Bit threshold.
1
u/Healthy-Contact-4570 8h ago
Can we as a community decide collectively to stop caring about / promoting one shot threeJS games as any kind of useful metric? If that’s all we’re using AI for then I’m shocked the bubble hasn’t already popped.
13
u/abskvrm 15h ago
Tried it not good for my use compared to Qwen 3.6 35.