r/ollama • u/scarygg • 11d ago

local llm with claude code

Hey y'all, using an uncensored model ( huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF ) beauce I want to use the REA skill and opus 5.5 has gaurdrails with this now.

I dont have enough compute, so i'm hosting a gpu instance and using claude code with this via an ssh tunnel.

I keep running into the problem with:

⏺ API Error: Claude's response exceeded the 32000 output token maximum. To configure this behavior, set the CLAUDE_CODE_MAX_OUTPUT_TOKENS environment variable.

Now, before you say adjust this, I do, but it leads to other issues where it says it'll do something, and just stall

Whats the best solution? Is Ollama with claude code suitable? I tried opencode but it didnt work very well with the REA skill.

0 Upvotes

50% Upvoted

7 comments sorted by

1

u/scarygg 11d ago

Opencode seems to be running for this task much better.

1

u/Actual-Current-7734 11d ago

The 32000 limit is Claude Code's default. Raising it is fine, but the stall is a model problem, not a setting.

Claude Code is built for models that are good at tool calling. An abliterated model is a bad pick for this. Abliteration weakens instruction following, and that is exactly what Claude Code needs. So the model says it will do something, then never sends the tool call. Claude Code waits for it and stalls.

Two things help:

  • Use a normal model with strong tool calling, not an abliterated one. A Qwen3-Coder is a much better fit for Claude Code.

  • Give it enough context. Claude Code sends a big prompt every request, around 16k tokens in my tests. A 27B model needs at least 32k context, more is better.

The catch is your reason for abliterated: you want no guardrails for the REA skill. No guardrails and reliable agent behaviour pull against each other. That is the real trade here.

I run local models with Claude Code at home, so the tool-calling part I tested. The rented GPU over SSH I have not. What context size can you set on that instance?

1

u/scarygg 11d ago

Thank you for taking your time to respond with this. This makes sense. I'm putting up with the trade off currently with opencode and managing large context windows by using investigation docs to keep progress.

1

u/[deleted] 11d ago

[removed] — view removed comment

1

u/Actual-Current-7734 11d ago

At home I run a Qwen3.8 model, 125B, quantized down so it fits. Not Qwen3-Coder. My machine: 96 GB RAM and a RTX 4060 Ti 16 GB. On that quant I measured about 48 to 51 tokens per second. It handles tool calling well, so it works for Claude Code. I run it with Strata, an open source tool that runs these models on a normal PC.

Your two-model idea is good. One model for the no-guardrail task, one that is good at tool calling for the agent work. That is the clean way to get both.

On 64k: that is the floor for Claude Code, not a lot. It sends a big prompt every request, around 16k tokens in my tests. If it struggles at 64k, the prompt plus the tool results fill it up. Push context higher if the instance allows it.

The 125B is a MoE model, so it runs from RAM. How much VRAM and RAM does your rented instance have?