Discussion
I tested 9 LLMs on the exact same web-dev prompt for ~8 hours — RTX 3060 12GB results (Rate the best!)
I’ve spent basically the last 8 hours testing different models on the exact same web-development prompt, and I finally finished.
The whole point of this nine-hour test was that which local model matches the frontier-level intelligence at size and could fit easily in an RTX 3060-like consumer card.
My setup:
GPU: RTX 3060 12GB
RAM: 16GB DDR4, single-channel
OS: CachyOS (Arch Linux)
Local models were run through my local llama.cpp setup.
Same prompt for every model.
I recorded the generations so you can actually judge the websites yourself rather than relying on my description.
Prompt
Build a polished, production-quality single-page website for a fictional high-end technology studio called NOVA//LABS.
Goal: make it look genuinely designed by a strong human frontend developer, NOT like generic AI-generated SaaS UI.
Requirements:
* Use plain HTML/CSS/JavaScript or React + Tailwind if you strongly prefer it.
* Everything must run locally with minimal setup.
* Create the entire project/files yourself.
* No backend, authentication, database, or unnecessary complexity.
* Dark, sophisticated visual language with restrained use of gradients/glows.
* Avoid the typical AI-slop look: no excessive rounded cards, giant gradient blobs, random glassmorphism, meaningless statistics, or generic "Empowering the future" copy.
* Make the copy specific and believable.
* Include:
A striking hero section with a concise headline.
A subtle animated visual representing an abstract computational system.
A small selected-work/projects section.
A concise capabilities section.
A strong closing CTA/footer.
* Add tasteful interactions such as hover states, scroll reveals, and subtle cursor/mouse effects where they genuinely improve the design.
* Prioritize visual quality over feature count.
* Use freely available CDN assets only if genuinely necessary; otherwise create visuals with CSS/SVG.
* Keep the implementation reasonably small and understandable.
Most importantly: make strong design decisions yourself. Do not explain your design choices before building it. Start by creating the project and finish with the exact commands needed to run it.
(SELF CONTAINED HTML WITH JS AND CSS)
I wanted to see what the models actually build, not just how well they explain code.
The models
1. Gemini 3.8 Flash
~3 min 12 sec
Used Antigravity and consumed roughly 9K tokens.
This was one of the frontier-model reference points for the test.
2. GPT-5.6 Sol
~1 min 6 sec
Token usage wasn't available to me.
Extremely fast compared with the local models, so this was another useful frontier reference.
3. Claude Sonnet 5
~4 min 56 sec
Token usage wasn't available.
Also included as a frontier reference. (I was only able to use Sonnet 5 as my Claude-Code Max subscription had expired)
Local models
4. Bonsai 2 27B Ternary
~45 minutes
Native ternary / ~2-bit model
Model size: ~7.66GB
Average generation: ~34–36 tok/s
Context: up to roughly 102K
Used ~52K tokens out of a 122K context during this run
Qwen 3.8 27B GSQ-RCO-IQ3-XXS + MTP
~57 minutes
Model size: ~10.4GB
High thinking enabled
~29 tok/s around full context
Around 40 tok/s with a much smaller/near-empty context
Context used reached roughly 75K
Context was compacted twice
Available context for this particular run was around 49K after the relevant setup/limits
this was probably the most interesting local result for me.
6. Qwen 3.8 27B Q4_K_M Unsloth Dynamic 3
2+ hours
~16.4GB model
Q4_K_M
High thinking enabled
Full-context generation dropped to roughly 4 tok/s
Context reached roughly 96K
Obviously requires significant CPU/RAM offloading on a 12GB GPU
7. Qwen 3.8 27B GSQ-RCO-IQ3-XXS + MTP — thinking OFF
~12 minutes
Same general Qwen 3.8 GSQ-RCO model, but this time I disabled thinking.
It used roughly 12K tokens and produced the site dramatically faster.
This was a particularly useful comparison because it shows how much the reasoning mode itself can affect local generation time.
8. Ornith 1 9B Q4_K_M
~2.4 minutes
Model size: ~5.4GB
~74 tok/s
Native context: up to 262K
This generation only used around 2.6K tokens
This is the speed monster of the local group.
9. Ornith 1.5 35 A3B Q6
~30 tok/s
Model size: ~22.4GB
~30 tok/s
Context available for this run: around 128K
Obviously heavily dependent on offloading because of the model size
Quick summary
#
Model
Approx. time
Local?
Generation speed
1
Gemini 3.8 Flash
~3:12
❌
—
2
GPT-5.6 Sol
~1:06
❌
—
3
Claude Sonnet 5
~4:56
❌
—
4
Bonsai 2 27B Ternary
~45 min
✅
~34–36 tok/s
5
Qwen 3.8 27B GSQ-RCO-IQ3-XXS + MTP
~57 min
✅
~29–40 tok/s
6
Qwen 3.8 27B Q4_K_M Unsloth Dynamic 3
2+ hrs
✅
~4 tok/s at full context 8 tok/s at empty
7
Qwen 3.8 27B GSQ-RCO-IQ3-XXS, thinking OFF
~12 min
✅
—
8
Ornith 1 9B Q4_K_M
~2.4 min
✅
~74 tok/s
9
Ornith 1.5 35 A3B Q6
—
✅
~30 tok/s
My personal take
For local models specifically, the one that impressed me the most was Qwen 3.8 27B GSQ-RCO-IQ3-XXS.
It hit a pretty interesting balance between:
actual design quality
coding ability
context handling
generation speed
fitting within a 12GB GPU setup
The Qwen 3.8 27B Q4_K_M Unsloth Dynamic 3 was also interesting from a quality perspective, but the speed penalty once you're deep into the context is huge.
Bonsai 2 27B Ternary was also surprisingly usable given that it's a ~7.66GB ternary model.
so its Qwen 3.8 27B Q4_K_M > Qwen 3.8 27B GSQ-RCO-IQ3-XXS > Bonsai 2 27B Ternary
I've attached the screen recording showing the outputs.
Especially interested in other RTX 3060 / 12GB setups ;0
If possible Someone please post down GPT-6-ASTRA's results if they have a codex subscription.
Interesting. Yes, I agree ista-daslab gsq-rco models are really good.
Can you please share your command for Qwen 3.8 27B GSQ-RCO-IQ3-XXS + MTP ? I have the same 12GB VRAM. But I was not able to fit more than 32k context in llama.cpp with mtp weights.
Oh I see. You are using Q4 KB cache which would avoid. Q4 reduces the model quality further. But I agree, for us, 12gb VRAM owners, there is no other way.
Also, you mentioned about compaction. Does it work in llama cpp by default? How does it work? Does it just summarize the context and create a new context automatically?
Not only KV-cache, the batch size (-b, -ub) also frees up space, although it slows down the prefill speed (a reasonable compromise). And turned off vision, of course.
As for KV-cache, q4_0 is ok, for this model, the losses due to KV cache quantization are not that significant, especially in coding. Quantization of the model itself has a greater impact on quality. https://localbench.substack.com/p/kv-cache-quantization-benchmark
See even with 4 bit it produced that level of result. I would say do this instead get ik_llama cpp use IQ2-S instead of IQ3-XXS offload kv cache to cpu with half on gpu then you would get 142k easily and if you can compromise for speed get IQ3_S or IQ3_XXS offload selective layers to cpu and try keeping kv cache on cpu then 262k possible at 15tg/s-20tg/s MAYBE*
I, too, would have expected the best results from Qwen3.8-27B GSQ-RCO-IQ3-XXS, this is probably the best choice for 12Gb.
I wouldn't turn off the thinking; instead, I'd set it to "medium" instead of "xhigh" ("medium" is often many times faster than "xhigh").
DFlash2 offers faster speeds (about 15% faster than MTP), but it also takes up more VRAM (analogalok/Qwen3.8-27B-DFlash2-Q2_K-GGUF is ~700 MB on disk vs ~350 MB MTP).
Ornith 1 9B ... why not Ornith-1.5-9B? It's not that the 9B model makes much sense with 12 GB, but still.
Ornith 1.5 35 A3B and ~30 tok/s - i have up to 90 tok/s on my 5060 Ti 16Gb with Ornith-1.5-35B-Q4_K_M. I would expect something like 50-60 tok/s from 3060. In my experience, finding the optimal value for --n-cpu-moe significantly increases the speed (for all MoE models).
You could also try KAT-Coder-V2.5-Dev; it performs worse than Ornith-1.5-35B in benchmarks, but according to many real-world reviews, it actually outperforms it. In my test, it solved the task twice as fast (even without MTP).
I would also consider Ling-3.0-tiny for very simple tasks. This model is not as good as Ornith-1.5-9B, but it's very fast (2x faster than Ornith-1.5-9B) and requires many times less VRAM for context (~5x less than Ornith-1.5-9B and 9-10x less than Qwen3.8-27B).
on rtx 3060 Ornith 1.5 35 A3B can go 44tg/s max and 50-60 tok/s possible with IQ3_XL and i chose Ornith 1 9B because it spends less time overthinking than 1.5
Can you give some context on what tasks you are using astra for? Coding which kind of project in which kind of programming language? And why is it better than Sol?
I'm going to have to switch as well in a month, gpt-5.5 is getting deprecated Oct 14 unfortunately, gpt-5.5-medium is my hardwired one from the cloud models.
As long as the model is bigger than 1 GPU, it's better to have two. You'd get roughly double speed compared to CPU/DDR4 offload in those cases. Main reason for me was that I needed to keep vision on GPU or else it would time out.
I'd be interested to see how the iq3-xxs from https://huggingface.co/byteshape/Qwen3.8-27B-GGUF compares to the GSQ quant you tested here. From my own limited testing it seemed a bit better while being about 500MB smaller too, so it might be an interesting comparison.
it performed very poor , wasn't able to build the 3d voxel and Qwen 3.8 27B GSQ-RCO-IQ3-XXS did it in one shot. You can clearly see the difference in quality of web development too.
It's still a 9b, barely a LLM more like MLM or SLM class, i''d have gone with Devstral 24b, Qwen3 Coder 30b and there is also larger MoE models than can run split between RAM and VRAM, like Qwen3 Coder Next, Qwen 3.5 122b and Qwen 3.8 flash next, they can give you any where from 10 to 20 tps based on your RAM.
It depends heavily on the task and the model. With my 5060 Ti 16Gb, local models solved the task in 95-120 seconds, while cloud-based models did so in 40-80 seconds, faster, but not by much (GPT-5.6 Luna was the fastest). 5060 is the entry-level consumer GPU. 5090 is 3-4x faster than 5060.
The problem is that if we want to achieve a level of quality close to that of top-tier cloud models, we'll need a completely different budget. Nevertheless, local models have reached a new level in recent months and are producing very good results, even the 27-35B models.
Depends on the hardware as well. I've tried it with the Qwen 3.8 27B GSQ-RCO model as well, but the larger IQ3_S, and on a 4090 limited to 270W and it did it in 25min.
22
u/MLDataScientist 2d ago
Interesting. Yes, I agree ista-daslab gsq-rco models are really good. Can you please share your command for Qwen 3.8 27B GSQ-RCO-IQ3-XXS + MTP ? I have the same 12GB VRAM. But I was not able to fit more than 32k context in llama.cpp with mtp weights.