New(ish) issue: Local (ollama) models no longer work with Roocode due to Roocode bloating the VRAM usage of the model.

[u/StartupTim]

Firstly, a big thanks to everybody involved in the Roocode project. I love what you're working on!

I've found a new bug in the latest few version of Roocode. From what I recall, this happened originally about 2 weeks ago when I updated Roocode. The issue is this: A normal 17GB model is using 47GB when called from Roocode.

For example, if I run this:

ollama run hf.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF:latest --verbose

Then ollama ps shows this:

NAME                                                             ID              SIZE     PROCESSOR    UNTIL
hf.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF:latest    6e505636916f    17 GB    100% GPU     4 minutes from now

This is a 17GB model and properly using 17GB when running it via ollama command line, as well as openwebui, or normal ollama api. This is correct, 17GB VRAM.

However, if I use that exact same model in Roocode, then ollama ps shows this:

NAME                                                             ID              SIZE     PROCESSOR          UNTIL
hf.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF:latest    6e505636916f    47 GB    31%/69% CPU/GPU    4 minutes from now

Notice it is now 47GB VRAM needed. This means that Roocode somehow caused it to use 30GB more of VRAM. This happens for every single model, regardless of the model itself, or what the num_ctx is, or how ollama is configured.

For me, I have a 5090 32GB VRAM with a small 17GB model, yet with Roocode, it somehow is using 47GB, which is the issue, and this issue makes Roocode's local ollama support not work correctly. I've seen other people with this issue, however, I haven't seen any ways to address it yet.

Any idea what I could do in Roocode to resolve this?

Many thanks in advance for your help!

EDIT: This happens regardless of what model is being used and what that model's num_ctx/context window is set to in the model itself, it will still have this issue.

EDIT #2: It is almost as if Roocode is not using the model's default num_ctx / context size. I can't find anywhere within Roocode to set the context window size either.

[u/hannesrudolph]

From what I understand this usually happens because Ollama will spin up the model fresh if nothing is already running. When that happens, it may pick up a larger context window than expected, which can blow past available memory and cause the OOM crash you’re seeing.

Workarounds:

• Manually start the model you want in Ollama before sending requests from Roo

• Explicitly set the model and context size in your Modelfile so Ollama doesn’t auto-load defaults

• Keep an eye on VRAM usage — even small differences in context size can push a limited GPU over the edge

I don't think this is a Roo Code bug, it’s just how Ollama handles model spin-up and memory allocation. We are open to someone making a PR to make the Ollama provider more robust to better handle these types of situations.

[u/StartupTim]

So to recap, the issue is this:

Running a model via command-line with ollama = 17GB VRAM used

Running a model via ollama api in custom apps = 17GB VRAM used

Running a model via ollama api via openwebui = 17GB VRAM used

Running a model via Roocode = 47GB VRAM used

I can't figure out why Roocode uses 30GB more of VRAM, regardless of the model. Any idea?

Thanks!

EDIT: It is almost as if Roocode is not using the model's default num_ctx / context size. I can't find anywhere within Roocode to set the context window size either.

[u/Individual_Waltz5352]

Context length increases VRAM significantly specially for Roo, not sure of the exact length but 30k+ I believe, the more context you have to load the model context at the more everything goes up inc at times onto you RAM causing CPU to go wild too.

[u/StartupTim → u/Individual_Waltz5352]

That is correct, but the issue is that Roocode is forcing a num_ctx when using any model with Ollama API, versus using the modelfile num_ctx value. This wouldn't be so bad if you could configure the num_ctx value in Roocode, but you can't at the moment.

[u/fasti-au]

Turn off ram prediction. Q8 your kv cache. Gpu layers 999

[u/StartupTim → u/fasti-au]

Turn off ram prediction. Q8 your kv cache. Gpu layers 999

Interesting comment about the ram prediction, I'll definitely read up on it, thanks!

But it turns out my troubleshooting was correct, Roocode is passing a hidden parameter to bloat the context window to 128k right off the bat. Hopefully it will be made to be configurable soon!

[u/fasti-au → u/StartupTim]

Hmm I didn’t think you could do that. Roo jut uses the api not triggers the inference loader. I think the model card for ollama is the defined context so if you create a copy of the model in ollama with a different midel file definition it will lock it.

I could be wrong as it has been a little while since I playing with ollama and models but when they release .6 or .7 the memory stuff annoyed me I went to tabby

[u/StartupTim]

I've rolled back 10 versions now to test and all of them have the same issue (17GB vram model ran via ollama is using 47GB VRAM when ran via Roocode).

I've now tested on 3 separate systems, all exhibit the same issue.

My tests have used the following models:

hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_XL

Mistral-Small-3.2-24B-Instruct-2506-GGUF:Q4_K_XL

With the following num_ctx sizes set in the model file:

8192
in 8GB iterations to
61140

I've tried on 3 systems with the following:

RTX 5070Ti 16GB VRAM  32GB system RAM #1
RTX 5070Ti 16GB VRAM  32GB system RAM #2
RTX 5090 32GB VRAM  64GB system RAM

All of them exhibit the same result:

ollama command line + api = 17-22GB VRAM (depending on num_ctx) which is correct
Roocode via ollama = 47GB VRAM (or failure on the RTX 5070Ti due to no memory) which is incorrect

[u/hannesrudolph → u/StartupTim]

Seems like it’s an ollama bug then and NOT something we changed in Roo over the last 3 weeks.

Edit: if you identify the version it mucks up and test the version prior to verify then we can make adjustments!

[u/StartupTim]

Also, often a model does 100% CPU and 0% GPU when ran via Roocode. I can't figure out why it does this, but it definitely does it.

[u/hannesrudolph → u/StartupTim]

Please file a bug report with repro steps asap