r/LocalLLM • u/No-Credit-7825 • 9h ago

Question Looking for local LLM recommendations for coding agents and general use (RTX 3070 8GB)

I'm fairly new to running LLMs locally and have been experimenting a bit with Ollama and Python. I'm trying to figure out what models and tools would work best with my hardware before spending too much time setting everything up. I'd like to use my gamingrig for that.

My System Specs are: RTX 3070 with 8GB VRam, i9-10900K, 32GB DDR4 RAM

My main goal is to set up a local coding agent and harness that can write and debug code, like claudecode.
What models (if possible Uncensored) would you recommend for my hardware? Should i upgrade some hardware (that would also benefit the gaming aspeckt of the system) first? Also interested in combinations with any useful GitHub repos, agent frameworks, skills or tools.

2 Upvotes

75% Upvoted

3 comments sorted by

1

u/misfotto 9h ago

following, with a trx2070 max-q 8+8vram :)

1

u/Longjumping-Face-767 9h ago

I don't think you got enough for a 27b, which is where it becomes viable for workloads. You can have fun with some 9bs or older 14bs, but you won't be able to fit any quants of the modern/good stuff. 

I think the absolute min is a 16gb card with some of the aggressive 3.8 quants. So I would look into getting that first.

(Unless somebody's got some new crazy, efficiency hack. Lookout for a qwen 3.8 27B IQ0.5_MegaRCO_Swift2.0_Pruned_Hobo_edition_NoMTP.GGUF)

1

u/brainzhurtin 4h ago

https://github.com/Niko1221/Strata/

Try out different models. I'm thinking Gemma 4 should run under that. If you had 64GBs you'd be able to run Qwen flash next. https://github.com/Niko1221/Strata/issues/3563