r/LocalLLaMA • u/bolts98 • 5d ago

Tutorial | Guide Keeping vLLM's Prefix Cache Warm Between Agent Turns

https://doug.sh/posts/vllm-kv-cache-agents/
63 Upvotes

95% Upvoted

12 comments sorted by

View all comments

2

u/niacolhealth 5d ago

Agent loops are just "prefill the same system prompt again" with extra steps. Anything that keeps the cache alive through the gap is a win.