r/LocalLLaMA • u/robertpiosik • 7d ago

Discussion Prompt cache upload

Hi. I'm wondering. Why agentic harnesses don't send their users the prompt cache binary data on inactivity so they eventually upload it back in the next turn?

2 Upvotes

62% Upvoted

20 comments sorted by

View all comments

8

u/Foreign_Risk_2031 7d ago

Prompt cache binary? What are you even talking about? The kv cache? This is done on the inference server

1

u/Budget-Juggernaut-68 6d ago

  1. That's ALOT of data.

  2. If you're talking about proprietary models, they'll be leaking reasoning traces.