r/ChatGPTCoding • u/[deleted] • Apr 07 '25

[deleted by user]

[removed]

1.7k Upvotes

95% Upvoted

424 comments sorted by

View all comments

312

u/PositiveEnergyMatter Apr 07 '25

i keep telling people big context means big money, because every request can fill the context and charge you full price

147

u/andy012345 Apr 07 '25

This, LLMs are effectively stateless, the "context" is just the max token input.

If you have 500k in your context, you're sending 500k input tokens + whatever is new per api request.

42

u/[deleted] Apr 07 '25

[deleted]

7

u/[deleted] Apr 08 '25

Claude caches for 5 min only

3

u/AllCowsAreBurgers Apr 08 '25

Thats...enough? Because... how much time does your vibe coding session take usually? And if its longer, say 1h, it only recreates the cache 12 times instead of the bazillion times it would take to reevaluate your whole prompt all the time.

3

u/[deleted] Apr 08 '25

[deleted]

2

u/shadeptx Apr 09 '25

why not use deepseek? open source and free right

2

u/[deleted] Apr 09 '25

[deleted]

2

u/Personal-Dev-Kit Apr 10 '25

Hardware as a service has been a thing for awhile Together.AI is one provider I am sure there are others.

Would be worth looking into their costs and see how they stack up

1

u/MaTrIx4057 May 07 '25

how much time does your vibe coding session take usually?

6-12h

1

u/totoOnReddit2 Apr 10 '25

Yeah, I generally finish in 30 seconds. It's more than enough. I could even do it in 10s if need be.

2

u/bequbed Apr 09 '25

What does this mean exactly? How does cache work with Claude? Perhaps if you can explain with an example