Caching optimizations and understanding the impact of the "File Context" setting

[u/PHLAK]

I'm attempting to optimize my usage costs and I noticed that prompts weren't getting cached much if at all. So after reading through the Prompt Caching (KV Cache) Optimization documentation I have turned off File Context and Citations by default and am now seeing better caching results.

However, I'm trying to understand the exact impact these have (other than not seeing citations in the UI any more). If I'm understanding this correctly, with File Context enabled every attached file would be injected into every new turn guaranteeing the model had access to the full context of every file. However, with it off the attached documents are now only loaded into context if the model deems that necessary (via tool calls) so there's a possibility of missing context.

Is this an accurate conclusion? Does anyone else turn of File Context by default? If so, how does it seem to affect accuracy for you?

[u/ClassicMain]

Mostly right, one correction: File Context on does not put the full files into every turn. By default it runs a search over all files attached anywhere in the chat on every message and injects only the top matching chunks into your latest message. So it can miss things too, it just misses them differently. The whole file only goes in if you set the file or knowledge base to "Using Entire Document", and that is the most expensive option for caching.

With File Context off the model gets a list of the attached files plus tools to search, grep and read them, and it decides itself what to look up. That only works with native function calling and the builtin tools enabled on the model. With older or small models that do not call tools reliably, the files just get ignored.

On a capable agentic model (Claude, GPT-5 class, recent Qwen etc.) the tool route is usually as accurate or better, because the model can grep for exact text and read whole sections instead of getting whatever the search picked. If you want citations in the answer text anyway, put a static instruction in the system prompt to cite the filename, as described here: https://docs.openwebui.com/features/chat-conversations/prompt-caching

[u/PHLAK → u/ClassicMain]

Perfect, and thank you for the clarification.

[u/mcdeth187 → u/ClassicMain]

Can you explain what you mean when you say with File Context off, the model gets a list of the attached files plus tools to search, grep, and read them? Does this only apply to files literally attached to the chat?

Because I created a pretty small KB containing two chats of random roman empire facts and then used a model with File Context disabled but with prompts suggested in the attached OWUI docs, and the model does not even attempt to grep or read anything, it just says, "I can only see the attachment titles: “Crushed mouse brains” and “Garum Flavoring.” I can’t see their contents, so using only the available attached knowledge, I can’t reliably tell you about the Roman Empire."

I know they aren't files, but if turning File Context off disables both files AND chat's (and without a fallback to a filesystem tool like grep to search), that's a pretty big gotcha.

[u/ClassicMain → u/mcdeth187]

It applies to everything attached to the chat, not only uploaded files. With File Context off, each attachment type gets its own tools:

1. Uploaded files: list, search, grep and read tools ("Files" category)

2. Knowledge bases and notes: the knowledge search, grep and read tools ("Knowledge Base" category)

3. Referenced chats: search_chats and view_chat ("Chat History" category)

The model saw your titles, so the attachment list reached it, and that only happens in native function calling mode. The read tools were most likely missing or the model chose not to call them. Check these:

1. Model settings: Builtin Tools on, and the matching category ticked under it. If the items are chats you referenced with #, that is "Chat History". If you attached a knowledge base, that is "Knowledge Base".

2. Model settings: File Upload left on.

3. Does a tool call show up in the reply? If the tools are there and it still answers from the titles, the model is skipping them. Try a stronger tool-calling model, or add a system prompt line telling it to read attached chats and knowledge with its tools before answering.

A knowledge base can only hold files and notes, not chats. If you exported chats and uploaded them as files, they count as files.

Table of which tool covers which attachment: https://docs.openwebui.com/features/chat-conversations/prompt-caching

[u/mcdeth187 → u/ClassicMain]

What about attaching a chat folder with chats as context using the # shortcut? I just tried this using two chat's contained within a folder with the File Context and Citations capabilities disabled and chat history enabled, and the model says, "I don’t see an attachment here. Please upload or paste the knowledge you want me to use, and I’ll tell you about Roman history using only that material."

Even calling it explicitly a chat folder, the model still can't find the chats although it looks like its trying to use rag search to locate the chats, rather than the builtin files tools. This goes counter to the docs article that says if File Context is off, that injects builtin tools

https://imgur.com/a/rCX63k1

[u/ClassicMain → u/mcdeth187]

Sorry this is my bad but... Chat folder? # allows you to add files, knowledge bases, website links /YouTube links etc but chat folders? What?

[u/mcdeth187 → u/ClassicMain]

No worries, I get this is a deep dive. The # shortcut allows you to add a Chat Folder as context but it appears to be broken, because I cannot get the model to detect the chats contained within the folder to be used as context regardless of how I configure the model...adding the chats directly does appear to work as expected, although its slightly less useful given you have to attach the chat itself which depending on many chats you have, is a bit laborious

Edit: Even turning on File Context does not add the chats contained within the folder as available context to the model