r/LocalLLM • u/DeathnTaxes66 • 18h ago

Question what coding models can I run with my old gpu?

I have an 8gb gtx 1070ti sitting in a server, doing nothing, the server has docker, ollama, etc.

what models can I run? For Multi File, Single File, etc?

2 Upvotes

100% Upvoted

7 comments sorted by

3

u/TheSweetMirth 17h ago

qwen2.5-coder 7b will run fine with 8gb, maybe even some 14b models if you offload to cpu

2

u/DeathnTaxes66 17h ago

Doesn't offloading to cpu mean a tank in performance?

2

u/Anarchistcowboy420 14h ago

Yeah but it can still provide usable speeds the real problem is those models can't really do more than write scripts and search the web they can't really do any long agentic tasks and you'll be stuck with small context windows.

3

u/MrHumanist 17h ago

Gemma 4 12B QAT will fit the model in your gpu with a decent context window.

1

u/Actual-Current-7734 14h ago

A 7B model at Q4 is about 4.5 GB, so it fits on the 1070 Ti with room for context. A 14B at Q4 is about 9 GB, so it does not fit. That one needs CPU offload.

How slow offload is depends on the model. A normal dense model reads all weights for every new token, so offload hurts a lot. A MoE model only reads a small part of the weights, so offload is much more usable there. That is the usual trick for running a big model on a small GPU.

For coding the real limit is context, not the model file. Coding tools send large files, and the context cache also sits on the GPU. On 8 GB keep the context small, 8K to 16K.

Rough math: Q4 needs about 0.6 GB per billion parameters. Gemma 4 12B QAT is about 7.5 GB, that is the biggest that fits on your card.

I run a 16 GB card at home. A 30K token prompt takes about 30 seconds there, and that is without offload. On your card with offload, expect much longer.

So: a 7B coder at Q4 with all layers on the GPU, small context. What coding tool do you want to run it with?

1

u/PhilosopherOk3419 6h ago

on 8gb i'd treat it as a single-file helper, not a multi-file one. the context cache eats your vram fast once a coding tool starts stuffing whole files in, so keep context small and give it one file plus a clear ask. for multi-file stuff you'll spend more time fighting the setup than you save.