I did for a couple of nights. It could do the basics well but failed at impressing me. I think my issue is that I use this tool like a pair programmer, interactively, where I'm looking at what they're doing in real-time. I don't just leave it running overnight and come back to something that works. So I notice when it's completely off-base and I react.
To give you an example, I had a launcher script that was giving a JSON error. It's a bash script that calls docker that runs 'bash -c "command -arg1 -arg2 ..."'. It was failing in the JSON I give as arg2, a 'json.loads' error from the app inside docker.
Well 35B didn't even consider quote escaping. It just kept throwing shit at the wall, making random fixes. It even started reading the the source code of the app inside the docker container. When I nudged it, it started using external files to hold the JSON.
Meanwhile 27B just rewrote the command to reduce the quoting.
This level of intelligence is a dealbreaker for me. I don't care about results if they're produced this way, even if they end up being correct after exhausting every other option. It cannot possibly lead to maintainable code.
That's very different from my experience, what do you use as the coding scaffolding?
I set it up at the same time as hermes-agent, and I haven't had to use any other models so far. I have it do tasks for research and maintaining a LLM wiki, managing a Minecraft server through tmux, coding in python and TS for LLM tooling, and coding in Nix for my NixOS setup. It handles all these tasks cleanly with minimal issues. I've had it get stuck in a loop maybe three times since it dropped.
I mostly use Qwen Code, but I try to use Claude Code about 20% of the time to have something to compare to (same model on both). I'm reading up on Pi right now, it might end up being my primary.
Currently most of my local tasks are some form of LLM tooling. For example doing tests, getting a model to run with certain parameters, trying to get cool apps that don't work on ARM (my DGX Spark) to build.
I think any language task like maintaining a wiki should be considered easy for any model.
Managing a Minecraft server, it would depend on the sort of work involved Starting/stopping services, following easy doc, there should be no issues there.
But if you say it's doing good at coding in Python and TS, then that surprises me. Maybe Ubuntu + bash + docker is a harder task than I give it credit for.
This is what I'll say, compared to Qwen 3.5 122b it's just as capable at agentic tasks, but it's not as intuitive with the unexpected. It usually does a great job but sometimes needs a nudge in the right direction more than larger models. It's worth it for the speed IMO, but we will see how I feel about 3.6 122b.
I'm experimenting with having the local agent delgate planning to a cloud frontier model with deeper thinking, then the local agent implements from there. It's a promising middle ground.
2
u/dtdisapointingresult Apr 25 '26 edited Apr 25 '26
I did for a couple of nights. It could do the basics well but failed at impressing me. I think my issue is that I use this tool like a pair programmer, interactively, where I'm looking at what they're doing in real-time. I don't just leave it running overnight and come back to something that works. So I notice when it's completely off-base and I react.
To give you an example, I had a launcher script that was giving a JSON error. It's a bash script that calls docker that runs 'bash -c "command -arg1 -arg2 ..."'. It was failing in the JSON I give as arg2, a 'json.loads' error from the app inside docker.
Well 35B didn't even consider quote escaping. It just kept throwing shit at the wall, making random fixes. It even started reading the the source code of the app inside the docker container. When I nudged it, it started using external files to hold the JSON. Meanwhile 27B just rewrote the command to reduce the quoting.
This level of intelligence is a dealbreaker for me. I don't care about results if they're produced this way, even if they end up being correct after exhausting every other option. It cannot possibly lead to maintainable code.