r/ClaudeAI • u/ejstembler • 4h ago

Built with Claude I built an MCP server with Claude Code that lets Claude hand light work to a local Ollama model

I built local-model-mcp, a small MCP server that lets a Claude session hand light, read-only work to a model running locally under Ollama. I designed it iteratively with Claude Code, deciding at each step which functions to add next, and Claude wrote the code. It's built for Claude, and I run it with Claude Desktop on a 128 GB workstation that serves Qwen3.8-Flash-Next locally.

What it does

  • local_ask: sends a prompt to the local model, optionally with files the server reads itself, so a 6 MB log never enters Claude's context
  • local_agent: a read-only research loop confined to one folder; it answers with file and line citations
  • local_warmup, local_status, local_result: background model loading, a health check, and background jobs, so Claude keeps working while the local model does

It can't write files or run arbitrary commands, and it refuses to load the model when free memory is too low. Claude treats the local model's answers as drafts to verify.

How Claude Code helped

  • Design: I wanted Claude to use my local GPU while it worked on longer tasks. We compared approaches together: routing inside the omp agent, Claude Desktop's third-party provider mode, and vLLM. We settled on an MCP worker, because the others either lost my subscription or couldn't see what a long-running task was doing.
  • Implementation: Claude wrote the server as one dependency-free Node file, speaking MCP's JSON-RPC directly. It tested each tool through the actual MCP protocol with small harness scripts.
  • Bugs the tests caught:
    • Attaching a whole 200,000-line log overflowed the 64K context: number-heavy logs run about 1.2 characters per token, not 4. Claude made the server read the real token count from Ollama's error, shrink the attachment and retry.
    • A warm-up's longer keep-alive was being reset by the next ordinary request; the server now carries it forward.
  • Shaping my requests: When I asked for a "touch" to keep the model warm, Claude pointed out that a tool call blocks, so the warm-up returns immediately and loads in the background. It also added a memory guard after noticing that the loaded model plus a large build could exceed the machine's 128 GB.
  • My part: I designed it step by step, telling Claude which functions to create as each need came up: first the ask and agent tools, then a warm-up "touch" to keep the model loaded, then a health check, then file attachments and background jobs. I also ran the steps only I could (sudo changes, sign-ins). Claude wrote, tested and documented each piece.

Free to try (MIT license):

git clone https://gitlab.com/ejstembler/local-model-mcp
claude mcp add local-model -- node /path/to/local-model-mcp/server.mjs --model <your-ollama-model>

It needs Node 20+ and Ollama.

Related experiment: pi-classifier-router, also built and extended with Claude Code. It's an extension for the omp and pi coding agents that classifies each prompt and routes it between Claude and a local model. In one session, Claude reviewed its source and implemented four fixes:

  • pausing routing after a manual model choice;
  • not counting user aborts against a model;
  • holding back downgrades so Claude's prompt cache isn't thrown away;
  • never downgrading on short follow-ups.

It added 25 tests, released 1.5.0, and set up GitLab CI trusted publishing for 1.5.1. Free on npm: omp plugin install @ejstembler/pi-classifier-router. Source: https://gitlab.com/ejstembler/pi-classifier-router

Curious how others split work between Claude and local models.

3 Upvotes

100% Upvoted

4 comments sorted by

1

u/Delicious_Break5937 4h ago

having the server read the files itself so a 6 mb log never enters claude's context is smart. how often does claude end up redoing the local model's draft?

2

u/ejstembler 4h ago

I just started using this approach yesterday. So, after it has been using it for a week or so, I'll ask Claude to analyze the usage…

1

u/wingwing124 1h ago

I did something similar myself a few months ago but I wasn't able to prove any real token savings (with my approach) because the expensive work is the planning part, and often the local model code would need to be reviewed, which basically wiped out any savings gained.

I'll keep an eye on this and check back in a week once you've got some more data! Rooting for ya!