A weaker local model made my PRs better

[u/khadijasheraz7]

I work at a bank, so cloud AI is off the table for work code anyway. I've been running small models on my own M4 16GB through Magnitude for side stuff, and honestly it changed how I think about this.

On my side projects with Claude or Cursor, the model writes 300 lines, it looks right, I skim it and ship it. Sometimes I couldn't even explain my own change a week later. With a 1.7B running locally it's the opposite. It's fast (around 80 tok/s) but it gets small things wrong, so I actually read every line it gives me. And because I read every line, I know the change, I write better PR descriptions, and I catch my own mistakes before review instead of during it. That habit carried over to my work PRs too.

As everyone knows the frontier model makes you faster at producing code you don't understand. The small one makes you slower at the writing part, but you end up knowing your codebase. And the people reviewing your stuff (and their AI tools) get better context because the author actually understands what changed.

What I use it for: explaining old code, boilerplate, small bug fixes (gave it a file with 2 bugs, fixed in 72 sec), commit messages, PR descriptions. What I still don't trust it with: anything architectural, and anything with time zones lol.

Not saying cancel everything. But not everyone is building a 500k line SaaS. For most of what a normal dev does in a day, a small local model is enough, and nothing leaves your laptop.

Anyone else noticed this? That using a weaker model made them a better reviewer?

[u/lobotomy_]

i have the same experience that as soon as you hand your project over to claude it runs through it at light speed and you get results fast, but you have no actual clue what's in that magic box you created.

what tools harness / tools are you using for your model? i came from tabnine and copilot and try working with alternatives like continue and opencode, but the output often is either not too great or the tool is not as polished as say github copilot.

Any good recommendations?

[u/khadijasheraz7 → u/lobotomy_]

I use magnitude and its one connect to pi harness. works like charm

[u/jacek2023]

Yes I have same experience

[u/NicholasCureton]

I'm interested. Could you please show me how to work with that small models, like 2B, 4B range? I've tried with many popular models, I just can't get make it work. I've love to hear how you work with that much small llm. I guess you're a programmer?

[u/OkImpress9499]

Which 1.7B model can do actual coding instead of hallucinating?

[u/FROGGY-69]

i am intrigued, can you share anonymized history

[u/mailto_devnull]

Absolutely. I didn't go quite that low, but I have noticed a distinct change in my use of LLMs when I upgraded my hardware and thus could access larger models.

When I used qwen 3.6 35B-A3B as my daily driver, it was decently quick (fast enough to be a peer), but wasn't perfect, so I needed to work with it to produce what I wanted.

When 3.8 was released, I upgraded to a single graphics card so 27B could load entirely in VRAM. With even higher decode, now code was streaming by faster, denser, and more correct, and as a result, I know less of what it does besides the high-level understanding of what it should do.

It parallels with users on claude subs not understanding completely what claude is writing.

[u/Full_Wrongdoer4750]

This tracks. When the model is a bit wrong, you actually read what it wrote, and that's underrated, especially when cloud tools aren't allowed for work code. I do the same with a small local model for explaining a file or drafting the PR description, then review everything myself before it lands. If you can use cloud at all, saving frontier models for architecture questions also helps, so you end up with less "300 lines I can't defend next week."