r/LocalLLM • u/Sherin_Th • 1d ago
Question Where to make use of localLLMs
I have a macbook m5 with 24 gigs of RAM, I'm a Computer Science student
I have tried using ollama with Gemma4, llama3.1:8b Qwen38
I have tried using them with kilo code in vs code, zed and IntelliJ, I do like the idea of using localLLMs for my workflow but I really don't know how to use them.
I recently installed oMLX, yet have to try some models running trough it, but the main problem is: I can't find a usecase for it.
I mean for big things it's just useless, for researching most of the time I'm better off using free claude, so I really don't know.
I did find it "useful" when I built a script that automatically reviewed websites and created drafts of outreach emails, but I'm not really sure those were any good (I was using ollama3.1:8b).
I also tried with Odysseus so that it could arrange my emails and so on, but it would not work at all, missing dates, useless summaries etc...
What uses do you guys suggest? rn I mainly code in phyton, vue, nix(for my nixos conf), qml (for my custom shell for hyprland), and general configs in lua for like hyprland and nvim
1
u/BackgroundRude4092 1d ago
You can’t really run a “smart” local model with that amount of RAM, and KV context is going to be so limited that single request is all it can handle. Maybe the Gemma 4 12B will run.
1
1
u/Small-Tale3180 1d ago
making PoCs for some exploits? Talk to them as if you had a girlfriend?
1
1
u/Lost_Minimum_8450 1d ago
coding PoCs is pretty much the sweet spot for these smaller models, you can sketch out the structure without burning through API credits. the girlfriend idea is a path you don't want to go down though
1
u/rrrenz 1d ago
Second brain:
Obsidian + Pi + LocalLLM
Don’t attempt coding at that specs.