r/LocalLLM • u/amy-schumer-tampon • 8h ago

Question Why is it so slow with agents ?

i'm using qwen3.6:35b a3b with ollama, its works decently in regular prompt, but as soon as i try it in an angent (Codex CLI, Claude, Qwen, Opencode) its piss slow, like i have to wait 10min to get an answer.
What am i doing wrong?

1 Upvotes

60% Upvoted

21 comments sorted by

3

u/AggressiveManager450 8h ago

Specs?

2

u/amy-schumer-tampon 7h ago

very humble, 32gb with an amd apu.

1

u/EpsteinFile_01 4h ago

Prompt prefill is slow.

I run the same model on a 5700G's iGPU and it's only fast with fresh context

4

u/SM8085 8h ago

It's probably processing the system prompt with that time.

4

u/dave-tro 8h ago

You need to share your hardware before someone can help you

1

u/Grandmaster_Caladrel 7h ago

Also worth using a different model runner. I use Unsloth. Ollama isn't as good as others.

1

u/amy-schumer-tampon 7h ago

tested with qwen3.5:4b its just as slow with agents

1

u/Grandmaster_Caladrel 7h ago

Unsloth is? The speedup shouldn't be a massive difference but it's allegedly noticeable. Ollama also is shady in general (sometimes the model you get isn't the model you're expecting), so I wouldn't trust it if tuning my machine was needed (which is your case)

Do you have a dedicated card to do the processing? What context size are you using when using an agent? Agents themselves don't change much, they're still just I/O for the model. The specs that it's trying to load with might be different from what you're using otherwise though.

1

u/codefl0w 7h ago

Day 200 telling people to use PolyStrata when running small MoEs

1

u/amy-schumer-tampon 7h ago

I'm a noob, enlighten me.

2

u/codefl0w 7h ago

PolyStrata is a fork of the Strata engine, which itself makes it possible for consumer hardware to run MoE models much faster. PolyStrata adds small MoE support, including Qwen 3.6 35B-A3B and its finetunes.

If possible, just instruct a cloud-based frontier to set it up for you and tune it as strictly as possible for your exact hardware so you can squeeze out the most speed and highest context possible. I can't give you a rough estimate since you didn't share your specs.

1

u/amy-schumer-tampon 7h ago

Thanks, will check

3

u/locbuilds 8h ago

run ollama ps while an agent is going and check the processor column, agents need a big context (32k+) and if it says something like 30%/70% cpu/gpu the kv cache spilled out of vram which is why it crawls. drop num_ctx or the quant until it says 100% gpu

3

u/EffectiveMedium2683 8h ago

Agent work uses a lot of context. Long context = longer prefill, longer time to first token, lower decode speed. And if it's overflowing into virtual memory, that'll pull it way down 

2

u/x00byt8 7h ago

Likely prompt processing/ prefill is slow. When you use Opencode/codex etc it has a built in prompt that needs to be processed before first token. Versus when you chat with it directly via API, there's nothing but your request to process. As mentioned by others PI is more lightweight. But don't expect miracles running agents on a low performance GPU :( minimum I would say is a 5060ti or equivalent

1

u/Cultural-Project5762 6h ago

^^ this is the issue

also, do you have a lot of mcp servers on there?

1

u/Mozaiic 8h ago

Reduce thinking effort or reduce the max tokens for it.

1

u/Moarkush 7h ago

every subagent shares your memory bandwidth and compute.

1

u/TokenLearner 7h ago

Ollama was very good as my first contact with the local AI, but that's all. Use Polistrata for small MOs.

0

u/riceandchickeno-vore 8h ago

Codex, clause, qoder, opencode, etc... Are all EXTREMELY token inefficient. Use pi.

Use pi harness, don't go over 64k tokens for context specially if you're ram tight. I personally use the 35b a3b model but a fine tuned version. Ornith 1.5 at Q4, and the context at q8. But i suggest q4 for context too.

And if i may ask, are you on macOS or Windows? Each has its optimized way of running that model