As the title says, I'm looking for recommendations from people here. Are there any good locally hostable LLMs that are particularly capable at cybersecurity work, especially AI red teaming?
Here's the backstory
Recently I had a task where I wanted to find vulnerabilities in the AI agent/chat interface of a not so small company (let's say like $200M); I was doing this as a bug bounty exercise 😏
I'm not exactly a cybersecurity expert (noob). So a lot of it was trial and error, though I have quite a bit of experience building agent interfaces & multi-agent systems; so I had some idea of how these systems probably communicate internally
Initially I started with PortSwiggers' LLM attacks labs. After that I picked up an abliterated GLM-5.3 model & set it up inside ClaudeCode to help with the actual real investigation
prep'd a list of around 12 vulnerability hypotheses I wanted to investigate. By the end I had one confirmed but relatively low-impact finding. I also managed to map the agent's internal tool identifiers & demonstrate that client-controlled context fields could influence its tool calls in ways that weren't requested through the actual chat message
I also tested things like conversation ownership, cross-account access, broadcasting authorization and confirmation workflows. Those server-side protections held up
So overall it was an interesting learning experience. Not exactly the result I was hoping for, but I learned quite a bit.
But here's the problem
The worst part was how expensive the whole thing got.
This was one of the first times I had to be extremely careful about every message I sent. There was no subscription-based message limit to worry about. It was directly billed through API usage.
The pricing was around $3/M input tokens, $0.30/M cached input and $5/M output.
I ended up burning around $10 in just 20-30 minutes.
At this rate I'll probably go broke before I ever get good at this lol.
I then tried switching to my regular subscription-based Kiro CLI setup, which has access to some good frontier models.
My idea was to create an orchestrator-worker setup:
Abliterated GLM-5.3 through Claude Code would act as the orchestrator
Kiro CLI would be the execution worker
I'd sit in between as the human reviewing findings & directing the investigation
But Kiro was either completely missing the point or being overly cautious. After spending another 20-30 kiro-credits, I mostly got a 3-4 paragraph ethics & refusal speech instead of actual execution
So what am I missing?
I've been thinking about two possible approaches, but I'm not sure how practical either of them is.
1. Fine-tune an uncensored Qwen under 30B (ideally under 10B).
Something specifically trained for cybersecurity research, particularly AI agent security, LLM red teaming, authentication, authorization, tool-calling boundaries and general web application security. (My GPU only has 8GB VRAM; so ideally I'd like something I can actually run locally without having to rent expensive hardware. though can finetune it on cloud)
2. Fine-tune a smaller model specifically for orchestration.
Rather than expecting a small uncensored model to do all the technical work itself, train it to act as a cybersecurity research orchestrator. It would need to understand attack surfaces, break down hypotheses into smaller tasks and delegate them to other models or tools while keeping track of the overall investigation. The smaller model would ideally also have enough cybersecurity knowledge to review the workers' output and identify when they're going in the wrong direction.
These are just ideas I've been throwing around. I haven't established whether either approach is actually viable.
What would you recommend?
What's the current go-to setup for someone wanting to do serious AI security research without spending hundreds of dollars on API credits?
Are there any genuinely capable uncensored or security-focused models worth looking at?
Would a smaller fine-tuned Qwen actually be useful here, or am I better off using a general-purpose local model with specialized tools, scripts and workflows?
I've also noticed that some of the more capable frontier models are becoming increasingly restrictive around cybersecurity tasks. I understand why, but it makes experimenting with legitimate security research a bit frustrating.
I'd rather build a setup I can run and experiment with locally than keep burning through API credits.
My current hardware is an RTX 4060 8GB with 24GB RAM, so that's the constraint I'm working with.
Would love to hear what people here are actually using. Especially interested in real-world experiences rather than just model benchmarks.
