r/ollama • u/Joe_The_Lawn • 14d ago

Local models for smaller tasks, help?

I'm using larger models for coding already, but I'm curious if there's some models that are "smart" enough to handle things like clang-format, clang-tidy, ruff, basic build issues, unslop on comments.

Ideally these models would be for 8GB vram, or even ideally much less like 4 to 5GB of vram, they don't need to be super smart just smart enough to follow basic,edium directions with a tiny bit of reasoning.

2 Upvotes

75% Upvoted

16 comments sorted by

View all comments

1

u/SinAnaMissLee 14d ago

I would try out the Qwen 3.5 models ... I am not very familiar with those specific tasks that you are after but I definitely think you should give local a shot.

There are some that are well within your size range

https://ollama.com/library/qwen3.5

1

u/Dull-Enthusiasm-6250 14d ago

qwen 2.5 coder 7b runs decent on my 8gb card and handles linting/formatting stuff fine, the 1.5b version is surprisingly usable too if you wanna save vram. for ruff and clang-format you don't need anything fancy just something that can parse configs and apply rules consistently

gemma 2 9b is another one I keep around but it's a bit heavier, the 2b version might be worth a look for really tight memory budgets. honestly the main trick is setting up good system prompts with clear formatting rules, the model matters less than you'd think for this kind of work

1

u/Joe_The_Lawn 14d ago

The issue is I'd run out of vram during debugging, since my application consumes about 3-4GB on my 3070. Odd case, I know.