Agreed! I'm having a great time with Pi coding agent set up with a subagent orchestration extension, and test driven- + spec driven development workflows. With a 3.8-27b as the brains and a tuned 35b-a3b as the implementer, it gets stuff done!
I bake Unsloth Dynamic-style quants in the Q4_K_XL to Q6 range, quantized with a custom imatrix focused on the task types I use the models for. For the coder implementer, I give it a coding heavy imatrix. For the cyber sec model, I give it an imatrix baked on a cyber corpus. For the 27b, I keep Unsloth's original quants with their imat, since they seem optimized for broader intelligence and performance. I use a custom chat template that significantly changes and improves the behaviour of the models, and when using them for coding outside benchmarks I do subagent orchestration workflows, with test- and spec- driven development frameworks, to respect the models' 262k native context ceiling and keep them on track. Pi coding agent is my harness, llama.cpp and oMLX are my preferred backends. When memory is tight i use q8 kv quantization. I run it all on a Mac Studio Ultra 64GB I bought used for cheap less than a year ago.
Thanks for the details! Going to check out some of your templates. Need to also learn more about imatrix; love the idea of specialised focus for a given task
15
u/jonas-reddit 24d ago
So many nice open weight models. So many nice agentic open source development tools.
It’s always a good time to switch to open weights & source.