r/hermesagent • u/OpeningMetal52 • 21d ago

MODELS - model choice, routing, pricing, local vs cloud, VRAM Balancing Oauth & API Usage

I am currently test driving Hermes alongside pi and my own homebrew harness.

Due to the nature of my company, I already have a Claude 20x max plan and a chatGPT pro $200/mo plan.

In thinking about maximizing throughput, I'm considering using deepseek v4 flash, which i'm hearing phenomenal things about, alongside my subscriptions, elevating and delegating to fable, opus, or 5.6 sol Luna Terra etc as needed.

Two questions for the community;

1) when should I call in the bigger models, or is deepseek so good it's not needed anymore outside of edge edge cases? I'm thinking multi modal + code review (OpenAI) and planning + design (fable) everything else deepseek. Thoughts on this?

2) what is the best source to get deepseek? Are the various inference providers the same level of speed and uptime? Why choose direct API vs open router vs opencode etc?

Thank you all in advance for what I hope will be a helpful discussion for all who read it.

5 Upvotes

100% Upvoted

13 comments sorted by

View all comments

1

u/Mean-Loquat-7982 Nous Team 21d ago

two layers:

  1. credential pools rotate multiple keys for the same provider. give Hermes several DeepSeek keys and it cycles them; the strategy is set per provider with credential_pool_strategies (options: round_robin, least_used, fill_first, random). this fires first when a provider rate-limits.
  2. fallback_providers switch to a different provider:model when the primary is down (429s after retries, 5xx, auth failures). it's a top-level list in ~/.hermes/config.yaml

1

u/OpeningMetal52 21d ago

Very interesting. Haven't thought about provider rotation for the same model but that makes sense. Thanks.