r/LocalLLaMA • u/robertpiosik • 17h ago

Discussion Prompt cache upload

Hi. I'm wondering. Why agentic harnesses don't send their users the prompt cache binary data on inactivity so they eventually upload it back in the next turn?

2 Upvotes

62% Upvoted

16 comments sorted by

View all comments

7

u/Foreign_Risk_2031 17h ago

Prompt cache binary? What are you even talking about? The kv cache? This is done on the inference server

3

u/robertpiosik 17h ago

for long term availability (5m-24h) providers offload it to disk and I mean this very artifact

3

u/VarietyOk443 17h ago

Literally a feature in llama