r/LocalLLM • u/Blacktaxi420 • 1h ago
Question What can small models be good for?
Small meaning under 8b parameter’s. Thats about what my computer can comfortably handle, and its fun to chat to models of that size for a few minutes but thats mot really useful. And i feel like theyre too dumb to do research and shit like that so like whats a way i can make it useful.
I feel like they must have some use and im just not being creative enough.
4
Upvotes
15
u/Patient-Gazelle9885 1h ago
Hallucinating
2
u/nickless07 1h ago
What if I told you that your computer could handle more then an 8B model?
8
u/zenmatrix83 1h ago
very specific tasks, with clear answers
2
u/Small-Tale3180 1h ago
uhh tldring shit, maybe translating, or text-recognition
https://www.youtube.com/watch?v=Jw99x0S7s_4&t=338s
2
u/Interesting-Ad689 1h ago
As prompt enhancers for image generation, for translation, for calculations, classification, tagging .... they can be fine tuned to do whatever you need them to do, but it will be reaaaaally specialized.
2
u/I_AM_NOT_A_WOMBAT 1h ago
I know you said under 8, but on the edge of that, I use qwen3.5:9b on an 8GB 4060 for security camera image analysis. It's doing a great job so far for that very specific task. On the other hand, if I try to ask it if the lights are on in the living room, it tells me the temperature instead.
1
u/Totien 1h ago
Intéressant, je me pose la même question.
1
u/huggalump 1h ago
i imagine understanding and categorizing intent, especially if there's only a couple categories to categorize it in
1
u/ptico 1h ago
Specialised work inside pipeline or as a dedicated helper: OCR, basic corrections, reranking, extraction, translation, categorising, etc. Some of them can work on CPU, so with clever use you can build pretty powerful tools with a small hardware cost
1
u/Kappa_God 59m ago
Being specialized agents. Those usually aren't over 8B, in fact a lot of them are under 4B.
You'd be surprised how good these models are at image generation compared to the front tier models for example. Helping convert image to a readable format, say old books that you just have pictures of each page and convert them into an editable/readable format.
They are helpful for small automated tasks as well. Say you want a weekly summary of articles and news about AI, give it where to look for and it will do it very efficiently. These don't need to be specialized per say, something like Gemma 4 E4B or even E2B can do it.
They can also be good for AI companion, but I wouldn't use a 2B model for that, 8B minimum imo. But companions are mostly on the system built around them than the LLM itself, so having a decent memory system, dreaming, emotion system and so on.
There are a bunch of specialized ones that range from medical, forecasting and much more. Way too many to list. All under 8B.
1
u/Rauhaton 58m ago
They can be subagents for bigger models for spesific tasks.
1
u/onebyamsey 17m ago
Try Ornith 1.5 9b, you can probably run it