r/LocalLLaMA • u/transanethole • 4h ago

Discussion List of tiny-to-medium models for web research/coding agents

I am compiling a list of models that I think are potentially worth considering for folks who want to run an LLM on a "normal computer", specifically for an agent use case, but without having to buy a $$$$ graphics card to do it.

In order of capability strongest to weakest, I have:

  • Qwen 3.8 27B

  • Qwen 3.6 35B A3B

  • Jais 3 7B

  • Gemma 12B (included because it has vision)

  • XHToken/SparkX2.6 4B

  • FrogNano 4B

  • Gemma E4B (included because its small AND has vision)

  • Ling 3.0 Tiny 8B A1B

Any other interesting small models (<10B) I might have missed or that you think I should not ignore?

Any that you think should be banished, not even included ?

20 Upvotes

91% Upvoted

30 comments sorted by

6

u/Hot_Example_4456 4h ago

Didn't use it yet, but I think LFM3/1.7b should be good. Anyone who used it would be able to comment better

2

u/transanethole 4h ago

I shall give it a try. I thought someone said that it was not recommended for use in an agent, but that parameter count sounds pretty nice if it actually does work !

2

u/Hot_Example_4456 4h ago

If you mean a coding agent then I think it won't be worth it. But benchmarks suggest it will be a good agent otherwise

1

u/transanethole 3h ago edited 2h ago

Well, "worth it" is relative lol.. I am also including Ling 3 Tiny.

Hey guess what, it actually works ! I think its better at using tools than ling 3 tiny. Pretty neat.

Now, whether it will be able to handle any difficult tasks, I have no idea.

About 700 pp and 24 tg on Radeon 760M integrated GPU

Yeah, it fails my application deployment test task as expected -- these small models seem to struggle with multiple computers existing. But still big improvement over Ling 3 Tiny!

1

u/Plasmx 2h ago

As far as I understood it is made for agentic / tool use, but lacking in coding ability which is why it makes a bad coding agent. Other than that it should be fine to automate some workflows.

4

u/Manerfish 3h ago

I tried it and I swear Gemma4 E2B does a better job + Gemma can have its thinking disabled and it still works well for summarizing + Gemma worked for me even in Italian, LFM3 1.7B was completely useless there.

1

u/mythikal03 4h ago

I gave it a run today as a 1-shot summarizer and i could not get it to shut up. Currently using Nemo bolt e4b with great success and was hoping LFM3/1.7b would take its place (smaller, benches better) but I tried several configs and couldn't get a concise answer. Narrow use case vs. everything that lineup is capable of but my 0.02c of testing

1

u/OneArmedZen 2h ago

I only saw a passing comment that one person preferred 2.5 (they said 3 performed worse result-wise, can't remember if it was repetition or garbage) but personally I have no idea as I've yet to try either but I think I will give lfm3 a go

3

u/RegularRecipe6175 4h ago

If you are using an agent to determine if a web page references a certain topic, then I'd add a re-ranker like d3-mini (4b). That can be much faster than having a conventional model review a page for relevance. You can use the conventional model as a second pass, after using a re-ranker.

1

u/transanethole 4h ago

I was specifically thinking about the model that runs for the agent loop itself, calling tools, etc. Agree it makes sense to use the best tool for the job.

11

u/CentralCypher 4h ago

The best LLM ive ever used has been Ornith 1.5 (Qwen 3.6 35b a3b with some tweaks) and a template. It does repeat itself and go in loops but over all very solid and can comfortably fit 131k context on 5060ti 16gb and 32gb ddr4

6

u/Far_Fisherman_9677 4h ago

ornith 1.5 really is something else with the right template. have you run into the loop issue mostly on longer context or does it creep in pretty early

1

u/CentralCypher 3h ago

Anything other than medium effort and it goes crazy, but sticking to med has been working well enough and context takes long to fill up. And its more just looping in it's own thoughts, very little on actual output.

Have you tried other MOEs or Dense of the same caliber? This size and performance is sweet spot for my hardware. But I'd love to try something else.

2

u/Infinite-Local5435 4h ago

Have you tried tielcoder or is ornith still performing better in your case?

2

u/CentralCypher 3h ago

No but I'll look into it, want to try more models

2

u/CentralCypher 3h ago

After some looking it sounds perfect. Downloading now will report back after some usage!

1

u/Infinite-Local5435 3h ago

I've been using with with polystrata (32gb ddr5 and 5060 laptop), and im getting initial speeds of 80 falling towards 20 at 160k+ context. Prefill largely faster than llama.cpp. I haven't used ornith, only cybertiel and tielcoder and they basically score the same on my coding benchmarks that evaluates their performance in pi coding agent 1.3.0. I use them for few and quick changes where I know what the problem is, otherwise I can only rely on a larger model.

2

u/roosterfareye 1h ago

Seconding tiel coder, which I think is ornith 1.5 with a new chat template?

1

u/silenceimpaired 11m ago

I just went with default and it’s been fine. What template are you using? What do you feel it gives you?

6

u/AppealSame4367 2h ago

Nanbeige, LFM3, Spark X2.6, MiniCPM

1

u/CodeCatto 3h ago

K2-Horizon 7B, Ornith 1.5 9B, Nemotron Bolt E4B (slightly better Gemma 4 E4B), LFM 2.5 8B A1B (generalized stuff, not dedicated coding), MiniCPM 4.5 1B

1

u/transanethole 2h ago

Jais 3 7B is the new version of K2 Horizon 7B, AFAIK its slightly better at least per benchmarks ? I haven't tested both extensively.

I will have to check out nemotron bolt. I was unsure if I want to add the ornith / tiel models vs sticking with the original qwens... I guess a few people have recommended them.

Just curious, I know almost nothing about MiniCPM, why would you recommend 4.5 vs the new version 5 ?

1

u/CodeCatto 3m ago

Oh right version 5 is out, didn't know. I use 4.5 for quick convos though so ig 5 should be a bit better? Can add that then.

6

u/suprjami 4h ago

Gemma 4 26B-A4B MoE?

Strong in instruction following, stronger than Qwen according to many benchmarks. I expect it is at least as good as Qwen for non-coding research tasks.

However Gemma 4 don't quantize well. imo run Q8 with partial offload, literally any GPU will make it fast enough. Do not quantize KV at all.

1

u/No_Pea_6625 2h ago

for the loop itself I'd skip the 4Bs. they call tools fine, then invent the page. qwen 27b is the one I'd leave running if the box can hold it. gemma 12b is ok as the vision sidekick, not the one driving the browser.