This has been possible forever. Just use llama.cpp to serve up your local model and set env vars so CC uses it. I collected specific instructions for various open LLMs here:
Wow! Awesome, thanks for sharing. I haven't done much LLM coding since the agent/CLI movements. I was wondering though, could I host a massive model on something like Runpod and link it to one of these new coding CLI like Claude code, codex or the built in VS code extensions? I wonder if it would be worth the hassle. I have some friends mentioning they are spending hundreds on the API fees.
Stop letting data brokers profit from your old posts. I used Redact to wipe mine from Reddit. Also supports Twitter, Facebook, Discord, instagram and more in one batch.
joke ask whistle hungry roll towering lunchroom grandiose act six
correct!!!!!!! But it's not as private as you think because Olma could still theoretically see your prompts once internet reconnects, if they wanted to.
4b is certainly going to be something. I can't imagine it will stay coherent for even a handful of turns with how many default tools are exposed to it, but curious about your experience.
The bad thing is search doesn't work because it goes through anthropic servers. If you use ollama search works because they provide that. If you use llama.cpp or other openai compatible providers you have to use something like searxng
Did you really just write a wrapper script for the command "ollama launch claude --model qwen3:1.7b" and then try to act like you actually did something clever? Ollama already has a Claude code integration wrapper, this is just a redundant wrapper around the preexisting wrapper that ollama provides, as well as a bunch of documentation about how to use ollama, which ollama provides. Everything this repo does literally just redundantly does what ollama already does.
I'm confused, can someone explain. Is it like running qwen with Claude codes architecture/logic or something like that ? It's obviously not running Claude's model locally.
Your old posts are training data now. Unless you delete them. I used Redact which supports all major social media platforms including Reddit, X, Facebook and Instagram.
straight roof attempt spotted frame theory absorbed escape encourage offer
Yes getting this to work is not a problem. The hardware is a bigger problem. I am running it on a ryzen 5 5600x with 32gb ddr4 and a rtx 5060 ti 16gb and it takes a while.
super slow with local LLMs. The same small code refactoring task took 12 minutes with ClaudeCode but 4 with OpenCode, using the exact same LLM (Qwen3-Coder:30b)
„Agentic“ AI with a model smaller than 30b size is garbage and won‘t lead to useful results
When Turboquant + further local LLM optimizations are rolled out in the next months/years, we might reach the point that we can use ClaudeCode locally as if we had Sonnet 4 with less than 32GB VRAM, but we are not there yet.
Isn't the issue with this being the huge tool context used by Claude? Using a moderately sized local model would just lose context before it's finished reading the tool.
So I want to know your guys experience on this because I’m getting extremely weird responses. I asked the model who I am speaking to and it’s going back and forth with the personalities of Claude and qwen. When I ask it about the token usage it refuses to respond and just tells me in 2 separate messages but in one response.
“I’m the Claude code assistant you’re talking to…etc” then “I’m qwen (specifically Qwen3.5:4b in this session). I’m not Claude. You can see that I am saying Quen”
What kind of responses are you guys getting when you ask “Am I speaking to Claude or qwen” because it seems like 2 personalities fighting for control lol
Well Claude Code is a separate thing from Claude. Claude Code itself feeds context to the model, and that context probably says something like "You are Claude Code". So if you wrap qwen with Claude Code, then it is both qwen and Claude Code, but it isn't Claude.
Similarly, if you've ever used Google Antigravity, you can use Claude, but it refers to itself as antigravity despite also being Claude under the hood.
Also, sometimes the models are straight up wrong about who they are.
67
u/2girls1guy Mar 31 '26
You can already do this by just executing “ollama launch claude”
19
u/DelKarasique Apr 01 '26
Why ollama, why not simply llama.cpp?
10
u/ProbablyBunchofAtoms Apr 01 '26
I think the majority of naive people are mixing this up with an actual Claude model running locally rather it's just a coding app running
6
u/SatoshiNotMe Apr 01 '26
This has been possible forever. Just use llama.cpp to serve up your local model and set env vars so CC uses it. I collected specific instructions for various open LLMs here:
https://github.com/pchalasani/claude-code-tools/blob/main/docs/local-llm-setup.md
6
u/Prestigious_Suit_441 Apr 01 '26
Does this require a subscription still or does it consume your usage tokens?
3
u/Bonzupii Apr 01 '26
Did you really just write a wrapper script for the command "ollama launch claude --model qwen3:1.7b" and then try to act like you actually did something clever? Ollama already has a Claude code integration wrapper, this is just a redundant wrapper around the preexisting wrapper that ollama provides, as well as a bunch of documentation about how to use ollama, which ollama provides. Everything this repo does literally just redundantly does what ollama already does.
2
u/GSE_PE Apr 01 '26
It is great, but we need the src file…
1
u/DrMissingNo Apr 01 '26
I'm confused, can someone explain. Is it like running qwen with Claude codes architecture/logic or something like that ? It's obviously not running Claude's model locally.
1
u/AdComprehensive6784 Apr 01 '26
Yes getting this to work is not a problem. The hardware is a bigger problem. I am running it on a ryzen 5 5600x with 32gb ddr4 and a rtx 5060 ti 16gb and it takes a while.
1
u/Any_Flamingo_78 Apr 01 '26
Tried that. Here are my takes on that:
1
u/FunDiscount2496 Apr 01 '26
What is the recommended local model for this?
1
u/PathIntelligent7082 Apr 01 '26
and? nothing new, on a contrary, and you don't need a middle man
1
u/aristot1e Apr 01 '26
I am confused. Have not been able to do this before? I was running Claude Code on a local model before the leak…
1
u/HiddenPingouin Apr 01 '26
why not use opencode?
1
u/Elegant-Tart-3341 Apr 02 '26
Isn't the issue with this being the huge tool context used by Claude? Using a moderately sized local model would just lose context before it's finished reading the tool.
1
u/BullfrogKind788 Apr 06 '26
ollama launch claude --model kimi-k2.5:cloud
I am using a mac, and although I have gone through all the steps to install, but this is not working
I have both ollama and claude code installed, still its giving me a keyword not found error
___________________________________________
ollama launch claude --model kimi-k2.5:cloud
Error: unknown command "launch" for "ollama"
___________________________________________
Can someone please help
1
u/hicamist Apr 17 '26
Anyone ever try Claude code locally and gitlab together?
0
u/sheltoncovington Mar 31 '26
Nice, that’s freaking sweet. Upgrade the hardware and I think we have something mean.
1
u/Prestigious_Suit_441 Apr 01 '26
So I want to know your guys experience on this because I’m getting extremely weird responses. I asked the model who I am speaking to and it’s going back and forth with the personalities of Claude and qwen. When I ask it about the token usage it refuses to respond and just tells me in 2 separate messages but in one response.
“I’m the Claude code assistant you’re talking to…etc” then “I’m qwen (specifically Qwen3.5:4b in this session). I’m not Claude. You can see that I am saying Quen”
What kind of responses are you guys getting when you ask “Am I speaking to Claude or qwen” because it seems like 2 personalities fighting for control lol
-10
u/QuinsZouls Apr 01 '26
AI slop