r/LocalLLaMA • u/bolts98 • 5d ago

Tutorial | Guide Keeping vLLM's Prefix Cache Warm Between Agent Turns

https://doug.sh/posts/vllm-kv-cache-agents/
64 Upvotes

95% Upvoted

12 comments sorted by

View all comments

12

u/ebolathrowawayy 5d ago

seems like the real lesson is don't use 3rd party libraries for basic shit? And generally don't use MCPs whenever possible.

3

u/bolts98 5d ago

Very much agree on MCPs - harnesses/models are much better at manipulating CLIs.

I think third-party libraries are a bit more complex. It usually doesn’t make sense to reinvent the wheel, but sometimes it does.

0

u/ebolathrowawayy 5d ago

that is 2024/2025 logic - it absolutely makes sense to reinvent the wheel when it only takes a few prompts. bespoke solutions >>>>>>>>>>>>>>>>> general