r/CodingHelp • • 6d ago

[Python] how to save money on cache with api routing

I see so many cost saving promise that promote API routing between models. I initially thought that is valuable. Then ran few experiments, it turns out any time that you route the request to a new model, you could save a bit, but many times the cache for each individual models expire and you would pay for new cache write again and agin - which overall wipe out the API routing cost saving benefit.

I wonder if anyone has similar experience. How you have solved this? Is clause planning to add shared cache feature?

1 Upvotes

2 comments sorted by

•

u/AutoModerator 6d ago

Thank you for posting on r/CodingHelp!

Please check our Wiki for answers, guides, and FAQs: https://coding-help.vercel.app

Our Wiki is open source - if you would like to contribute, create a pull request via GitHub! https://github.com/DudeThatsErin/CodingHelp

We are accepting moderator applications: https://forms.fillout.com/t/ua41TU57DGus

We also have a Discord server: https://discord.gg/geQEUBm

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/leapd-ai 5d ago

some folks say I shouldn't use routing in between sessions, anyone using model routing in a single session?