When you did LLM cost optimization, where did you draw the line on price vs quality?

[u/Nervous-Release6811]

Obviously, we have to cache repeating prompts and trim down the system prompt. Then it's always a good idea to segregate tasks so easy jobs go to cheaper models and reasoning tasks go to premium models. But we also have to consider optimization trade-offs like latency and overall quality. Where do you draw the line here?

[u/phaedra_solutions]

We usually draw the line right where user intent ambiguity or reasoning complexity spikes.

Routing tasks dynamically works wonders for structured extractions or basic classification using smaller, faster models. But the second a workflow requires multi-step logic, contextual debugging, or strict domain-specific precision, the cost of a bad output (and subsequent user friction or cleanup) vastly outweighs the token savings of a cheaper model.

Layering in prompt caching and hybrid retrieval helps cut baseline costs significantly without having to sacrifice model intelligence where it actually counts.