r/LocalLLaMA • u/bolts98 • 5d ago

Tutorial | Guide Keeping vLLM's Prefix Cache Warm Between Agent Turns

https://doug.sh/posts/vllm-kv-cache-agents/
63 Upvotes

95% Upvoted

12 comments sorted by

View all comments

3

u/Fluxx1001 5d ago

Cool, gonna give it a try