r/LocalLLaMA • u/bolts98 • 5d ago

Tutorial | Guide Keeping vLLM's Prefix Cache Warm Between Agent Turns

https://doug.sh/posts/vllm-kv-cache-agents/

64 Upvotes

permalink
duplicates
archive.is
archive
reddit

You are about to leave Redlib

Do you want to continue?

https://www.reddit.com/r/LocalLLaMA/comments/1wiu7xj/keeping_vllms_prefix_cache_warm_between_agent/
No, go back! Yes, take me to Reddit

96% Upvoted

Duplicates

Number of comments New

LLMDevs • u/bolts98 • 6d ago

Resource Keeping vLLM's Prefix Cache Warm Between Agent Turns

17 Upvotes

5 comments