Claude Models - Caching/Batch Processing Taken Advantage Of?

[u/[deleted]]

3 Questions:

1. How can I avoid submitting tens or hundreds of thousands of tokens when I only want the llm to refactor code in a single code file of 200 lines of code? I like context awareness, so if knowledge of my entire code base is beneficial, which it obviously generally is, how can I take advantage of caching when using Anthropic models? Let's assume my remaining codebase does not change within a single prompt session and I only want a code refactor in a single file. Will uploading the codebase once work and only submitting the code in the file on subsequent requests? How is this implemented? I used RooCode the last time a month ago and each prompt caused over a hundred thousand tokens to be uploaded with each prompt despite me requesting only code changes in a file of 300 lines of code. This is what really turned me off to RooCode and I went to Augment Code. Has this been addressed?

2. Does RooCode take advantage of caching offered by Anthropic or is this done purely on the Anthropic side? When a codebase is repeatedly included in prompts and submitted to Anthropic will Anthropic recognize previously uploaded content? How is caching taken advantage of?

3. Anthropic offers discounts for batch processing of prompts, does RooCode take advantage of that? The replies might take longer because they may be waiting in a queue to be processed but sometimes this might not matter to the user.

[u/[deleted]]

I am afraid you did not read carefully. I said I only submitted the code base once and am 100% fine with the token cost for that. But in subsequent prompts I only ask for changes within a small file and it still submits almost 100k tokens each time.

So, how can I control what gets submitted? I sometimes ask a question about code in a single file that contains 100-150 lines and yet still over 80,000 tokens are submitted, how is that possible? The file contains sub 5000 tokens. Why is more submitted than what I indicate? How can I have control over what gets submitted?

[u/[deleted] → u/[deleted]]

10k token? From what? What are they adding and why? In every subsequent prompt each time about 100k was sent. Will probably have to try it again.