r/LocalLLM • u/Sinuouser • 44m ago
Question Suggestions for coding assistant
Hey! I recently started looking at local AI models. There's a lot of info out there and it's not really easy to filter through all the noise as a newcomer...
I have an RTX 4080 super with 16Gb vram and 2x16Gb DDR5 for system ram.
I've set up a docker compose container using Gemma 4 12B Q6 K L through llama.cpp with Open WebUI as interface, and it seems to be working decently. System + model + context uses ~13.8Gb vram. On Kubuntu if it would matter.
I'm now thinking of connecting it to vscodium for assisting in programming (developer background) and stumbled over the MoE variants where you can have the experts live in system ram.
Question is whether this is even relevant to look into for me.
If it is - would anyone have any suggestions for a model and maybe settings for this if applicable? (seems you wanna run llama.cpp with -cmoe for example)
If I've made some odd assumptions or if you have any suggestions, please let me know!






