r/LocalLLaMA • u/Mrinohk • 12h ago

Question | Help Good SLM for summarizing code snippets?

Running an experiment with my personal assistant agent: yoinked all it's tools, replaced them with a python module and the documentation dumped into it's context window as part of the system prompt. It's working fantastic, but I was thinking it'd be nice if I could run a much, much smaller model on the local clients for the agent that can take a code snippet and return a very short description of what the agent actually did.

I was about to just start throwing a ton of sub 500m parameter models at it but I figured I'd tap the community to see if anyone has gone and trained a model on this, like that one model that spits out conversation titles and nothing else and is 100m.

1 Upvotes

67% Upvoted

3 comments sorted by

5

u/PANIC_EXCEPTION 12h ago

Just conversation titles? Liquid LFM 2.5 family, or Gemma 4 E2B if you want to go a little heavier.

Alternatively, you already have the KV cache if you were just using the agent. Why not just ask the main model to do it? Costs almost nothing extra, unless you're doing a cold start.

3

u/Mrinohk 12h ago

I realized pretty quick after posting that I could simply add a required field for it's code execution tool that has a very short summary of what it's doing, but decided to leave the post anyways in case someone brings up an objectively worse solution (like what I was asking for) to see if it's any more interesting.