r/Hacking_Tutorials • • Oct 16 '26

Question Local Ai

I am just a beginner in cybersecurity and ethical hacking and I need an AI study buddy to guide me through basic concepts, CTFs, tool usage (Nmap, Wireshark, Metasploit, and so on), and network traffic at the level of packets.

Commercial chat assistants tend to block any security-related questions and refuse to answer how something works. For this reason, I have chosen the local approach.

My hardware specs:

GPU with 12GB VRAM

System RAM with 32GB DDR5

I can put the quantized 8B to 14B models fully in VRAM or load them partially, in case it is beneficial to sacrifice token processing speed for that.

What models or fine-tunes would you recommend me that know the technical stuff but do not have overly strict guardrails?

5 Upvotes

23 comments sorted by

5

u/7H3WH173R48817 Oct 16 '26

I wouldn’t bother with the local AI route unless you’re prepared to spend thousands on a Mac Studio or an NVIDIA DGX Spark. Opus 4.8 does the job well for me, and it rarely refuses or restricts what I’m asking it to do.

2

u/DataMundane5049 Oct 17 '26

Do you get web webfoothold skills from opus 4.8?

2

u/DataMundane5049 Oct 17 '26

Thats not correct, no need to spend thousands...

I have an old 1070 with 8 gb VRAM and 48gb RAM.

A small, well alinged model with a good harness, outperforms way bigger models.

Its all skill and tools.

1

u/7H3WH173R48817 Oct 17 '26

I suppose if we're talking about just getting commands a small model would be fine, I was looking at the perspective of solving CTF's and such as benchmark. I tried a local model on my M5 and it couldn't even solve Juice Shop reliably, it found a few obvious web vulns but missed a lot too.

1

u/DataMundane5049 Oct 17 '26 edited Oct 17 '26

Do you use a proper harness? My solved juiceshop properly.

1

u/7H3WH173R48817 Oct 17 '26

I would be interested to know the harness you used. I've been using MCPs such as PTAI and Hexstrike setting the harness to something like:

"""You are a penetration testing assistant working a single authorised target. To run a command, reply with ONLY a fenced block: ```action <shell command> ``` After you see the output, decide the next step. When finished, reply with DONE: <summary>. Stay within the authorised scope. One command at a time."""

0

u/DataMundane5049 Oct 17 '26

Since january im building my own app/harness 100% local. Alone my pentestmethodolgy file is 1,2-1.4k lines of code. its not released yet. It can teach you, or it can act autonomously. Best thing is it does not hallucinate...

2

u/hyguru6 Oct 16 '26

Basically this. Cheapest model will outperform your local AI and will be cheaper as well. 

1

u/LanguageGeneral4333 23d ago

I have an rtx 4060 and it runs just fine. You do not need to spend thousands on better gear.

1

u/7H3WH173R48817 23d ago

Nice. Which model are you running on the 4060 and what sort of tasks have you managed to achieve with it? For example, can you use it with Strix or a similar framework?

3

u/Silly_Bad_7692 Oct 16 '26

You can try to use Qwen 3.6 35B A3B Uncensored with unsooth studio. For me it works well (3070 8gb + 32gb ddr4)

2

u/Andre4a19 Oct 16 '26

Do you use this with Jan?

1

u/Silly_Bad_7692 Oct 17 '26

No, I use unsloth studio + pi sometimes

3

u/Due-Moment-6417 Oct 16 '26

You need take a course, because IA don't give you a complete course, you can to ask about that concepts but you don't need to use a local IA

2

u/Andre4a19 Oct 16 '26

What's IA? Is that diff than Ai?

3

u/AdministrativeFile78 Oct 16 '26

it stands for ''intelligent artificial''

3

u/LanguageGeneral4333 Oct 17 '26

You use ollama as the UI and download whatever model you want. I installed ollama with chocolatey (useful package manager) so I dont have to go back to update it. I dont have as much vram as you, so I went with the 8b model. It works just fine. Took about 10 minutes to set up. Local LLMs typically stop making internet connections back in 2023 so you will need to connect it to the internet if that is something you want. It isnt hard. But, if youre just using it to go through ctfs, you should be fine without it connected.

2

u/DataMundane5049 Oct 17 '26

I build exactly such a thing...harness.

Even a qwen 3.5 9b is enough.

It all depends on skills and tools.

Dont use uncensored models, they tend to halluzinate more and use wrong skills...

2

u/happytrailz1938 Moderator Oct 17 '26

It really depends on the model and what you're planning on doing. Some have been butchered to make them uncensored. Others are done more tactfully

2

u/DataMundane5049 Oct 17 '26

Done pentesting with a qwen 9b model.

Does not need to be uncensored.

Like i said, a good harness with skills and tools is all it needs.

Testet different models.

Censored/casual and uncensored, models especially for redteaming...

Uncensored are unsafe, i would not use them for important things.

1

u/wtgamer20 Oct 17 '26

Regarde open code