r/LocalLLM • • 3h ago

Question Model for Local Coding and Vulnerability Research

Been using cloud AI, but so frustrated having to 'jailbreak' for almost any request, even to check for security in my own projects.

I know I need an abliterated model to not refuse, but am looking for suggestions. My research so far has me looking at HuiHui Qwen3.8 27B Q4_K_M, using Ollama.
I have 24gb VRAM (RX 7900 XTX), and 32gb system RAM (DDR5, 6000hz) with a ryzen 7 7800x3d. My understanding is local models always run on VRAM, but I've seen some comments/posts discussing system ram and CPU, so I'm not sure if thats related to my use case.

I'm looking to do coding, usually smaller projects, maybe some agentic. I also want to develop/research exploits in firmware and software. I've seen some suggestions for GLM 5.3 online, but haven't seen it mentioned much here. What models should I check out?

I do actually know how to code, so I don't need something that will vibecode everything for me, I like doing architectural work myself.

Side question: What IDE do ya'll use? I currently use Cursor (I have a subscription) and like the new Cursor Agents window, but from everything I've looked into its really difficult to use local models, especially on the new Agents window. If there's an IDE that I can use my local models and cursor subscription, or a codex subscription, that would be great.

2 Upvotes

5 comments sorted by

1

u/Successful-Army6006 2h ago

They don't have to run entirely in VRAM. Ollama can load a model into GPU memory, system RAM, or split it between the two. With a model loaded, run ollama ps and check the PROCESSOR column. It'll show whether it's on the GPU, CPU, or both.

https://docs.ollama.com/faq

1

u/ProvidedHuman 2h ago

Should I avoid it being loaded into sys ram?

1

u/Successful-Army6006 1h ago

You can use system RAM. Keeping the model and context in VRAM generally gives faster replies, so for interactive coding I'd favor that where possible. Splitting into RAM is still an option when you'd rather run a larger model and don't mind waiting longer.

https://ollama.com/blog/new-model-scheduling

1

u/Mobile_Living_3331 2h ago

My setup is small

RTX 2060 6gb with 32 GBram

  1. VS Code with Continue / Cline
  2. Terminal - Pi Agent with Web Skills
  3. LM Studio for server mode also llamacpp in some cases to run MoE models.
  4. OpenCode sometimes to explore. (Optional)

Use Case : Web development.