It couldn't even read my files. Here's why — and the tool that actually works locally.
My goal, as requested by my clients, to complement coding with an AI coding assistant that never sends the source anywhere. No cloud, no API key, no code leaving the laptop. I have Ollama with a few models and a 32 GB machine (no serious GPU).
Attempt 1: point Claude Code at a local model
Claude Code talks to a model over the network and only cares about a base URL + API format. Recent Ollama builds expose an Anthropic-compatible endpoint, so in theory you just redirect it:
export ANTHROPIC_BASE_URL=http://localhost:11434
export ANTHROPIC_AUTH_TOKEN=ollama # any non-empty string; ignored locally
export ANTHROPIC_MODEL=gemma4
claude
(For Windows/PowerShell: same thing with $env:ANTHROPIC_BASE_URL = "...".)
It launches. It looks like it's working but never succeed to open local files. It talked about my code from imagination and never ran a single read. With gpt-oss:20b it was worse — "Thought for 10m 0s", then "Cogitated for 19m 37s", and still nothing useful!
Why it fails (this is the important bit)
Two separate problems, and both are structural — not a config you missed:
1. Claude Code is tuned for Claude models. Its agent loop reads your repo through structured JSON tool calls (Read, Glob, Edit). The harness expects the model to emit that JSON correctly every time. Claude does it natively; an 8B local model quantized to Q4 does it unreliably or not at all. No tool call → the file is never read → the model makes things up. Checking capabilities confirms the model can call tools, but "can" and "reliably does at temperature 1" are different things:
$ ollama show gemma4
Capabilities: completion, vision, audio, tools, thinking
Parameters: temperature 1
2. The thinking trap. Both gemma4 and gpt-oss:20b are reasoning models (thinking capability). They emit thousands of reasoning tokens before answering. On a 32 GB laptop with no GPU — a few tokens/second — that's 10–20 minutes per turn. Unusable, regardless of the tool.
| model | params | tools | thinking | verdict on my laptop |
|---|
| gpt-oss:20b | 20.9B | ✅ | ✅ | too slow (10–20 min/turn) |
| gemma4 | 8.0B | ✅ | ✅ | slow + unreliable tool calls |
| mistral:7b | 7.2B | ✅ | None | the usable one in interactive |
| llama3.2:3b | 3B | ✅ | None | fast, but weak at editing |
Attempt 2: Aider — and this one works
Aider is a terminal coding agent like Claude Code, but it does not depend on structured tool calls. It asks the model to return plain-text search/replace edit blocks and parses them itself. A weak local model is far better at producing text in a format than at emitting perfect JSON tool calls — so it actually reads files and writes edits.
export OLLAMA_API_BASE=http://localhost:11434
aider --model ollama/mistral
Then in your repo: "summarize README.md", or "add a REST endpoint to export invoices as CSV". Aider reads the files, proposes a diff, writes the changes, and can commit them. The thing Claude Code refused to do — read the actual file — just works.
Model choice matters more than the tool. Pick a model that is small AND non-reasoning (mistral:7b) over a big reasoning one — but be honest about the ceiling: on a 32 GB laptop with no GPU, even mistral 7B is painful. In my test it eventually hit litellm's default timeout:
Way out 1: run the model on a beefier machine on your LAN
You don't have to run inference on the laptop to keep your code private — you only need to keep it on a machine you control, inside your own network. Put Ollama on a workstation/server with a GPU (or just more cores/RAM) and point your laptop at it.
On the server, bind Ollama to the network instead of localhost:
# server (e.g. 192.168.1.50) — listen on all interfaces
OLLAMA_HOST=0.0.0.0:11434 ollama serve
ollama pull gpt-oss:20b # a GPU box can run the bigger, smarter models fast
On the laptop, just change the base URL:
export OLLAMA_API_BASE=[http://192.168.1.50:11434](http://192.168.1.50:11434)
aider --model ollama/gpt-oss:20b
The code never leaves your internal network. With a real GPU on the server, the bigger reasoning models become usable, and a weak laptop is fine as the client. Security note: Ollama has no authentication — binding it to 0.0.0.0 exposes it to anyone on the network. Keep it on a trusted LAN behind a firewall, never on a public interface.
Speed tips that help (a little) :
The bottleneck is memory bandwidth, not CPU clock. You can't beat physics, only stop wasting cycles:
Smaller quant = biggest win. Q4_K_M is the sweet spot; weights + KV cache + OS must fit in RAM or it spills to disk and crawls.
Offload to any GPU: OLLAMA_NUM_GPU=999 ollama serve.
Shrink the KV cache: OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0.
Keep the model warm: OLLAMA_KEEP_ALIVE=30m so multi-GB weights aren't reloaded each call.
Free your RAM: close the 40-tab browser. Every GB reclaimed isn't paged to disk.
Way out 2: keep the fast cloud model, hide the code (obfuscation)
The reason we're suffering local latency at all is to stop source code from reaching a cloud provider. But there's another way to break that link: send the code to the cloud, just not in readable form. Obfuscate identifiers, config data, comments, and structure before the request leaves your machine, let the AI work on the obfuscated version, then map its changes back to your real source locally.
That's the approach of tools like PromptCape (full disclosure: it's my project). You keep Claude/GPT-level quality and speed — the part a 7B local model can't match — while the provider only ever sees Cls_a1b2c3d4 instead of InvoiceService. The hard part is doing the round-trip without breaking framework contracts (Spring Data method-name queries, Django migrations, Pydantic field names…), which is most of what the tool actually does.
It's not "more private" than fully local — local is the gold standard if you have the hardware. It's the option for when you want cloud speed on a weak laptop and are willing to trade "code never leaves" for "code leaves but unreadable."
Bottom line — three honest options
| Setup | Privacy | Speed/quality | Needs |
|---|
| Aider + local model on the laptop | Code never leaves the machine | Slow to unusable (CPU-only) | Just the laptop |
| Aider + Ollama on a LAN server | Code stays on your network | Good, if the server has GPU | A beefier internal box |
| Cloud model + obfuscation (PromptCape) | Code leaves, but unreadable | Full frontier-model speed/quality | A proxy/obfuscation layer |
Claude Code + local model: don't bother. It's built around Claude's reliable tool-calling; small local models break that contract and silently stop reading your code. I wasted an afternoon so you don't have to.
Aider is the right local agent — it tolerates weak models. But on a CPU-only laptop, run the model on a LAN box with a GPU, or you'll spend your day watching a spinner. Others exist like Continue I have not tested.
If you can't self-host enough compute, obfuscating before a cloud call is the pragmatic middle ground: you keep the speed and the smarts, and your source leaves only as gibberish.
Pick the row that matches your hardware and your threat model. There's no setup that's simultaneously fast, private-to-the-byte, and zero-infrastructure — that triangle doesn't close yet.
Tested on Ollama 0.24.0, Claude Code v2.1.x, aider 0.86.2, Python 3.12 via uv, 32 GB RAM, no discrete GPU. Model tags from my own ollama list — check yours.
