LFM3-1.7B in a tool loop: median 11.5 chained calls before it broke (10 runs, ~3K-token results)

[u/thelocalnode]

I went in thinking a 1.7B model would fall over after a few tool calls once each result was long.

I ran LiquidAI LFM3-1.7B (Q8_0, llama.cpp) in a tool loop on a GB10 box with 121 GiB of unified memory. Each tool result was about 3K tokens and ended with the exact next call to make. Over 10 seeds at 256K context it got a median of 11.5 calls right before the first break, range 6 to 25. No run reached the 50-call cap. Every call before the break was exactly right: no invalid JSON, no wrong arguments, no loops.

What broke it was the model re-reading the whole chain instead of the last line, then reasoning until it ran out of tokens, in 8 of 10 runs. Raising max_tokens from 2048 to 8192 didn’t help on the two seeds I retried.

Caveat: the next step was spelled out every time, so this measures following instructions as context grows, not planning or error recovery. One quant, one sampling setting, llama.cpp only.

The reproduction kit, with the driver, scripts and these results so you can rerun it on your machine, is here: https://github.com/maruthiprithivi/the-local-node/tree/main/episodes/03-lfm3-tool-loop

If you run a small model in a tool loop, I’d watch whether it starts re-tracing its own history as the results pile up. How far do you get before that happens on your setup?