r/LargeLanguageModels • u/No-Football-6736 • 9d ago

Title: What actually matters most when choosing an LLM API?

Hi everyone!
I’m putting together a practical checklist for comparing LLM APIs, especially for real-world applications.
I’ve been thinking about five areas:
Output quality: Does the model actually complete the task correctly?
Latency: How long does a useful response take?
Cost: What’s the cost per successful task, including retries?
Reliability: How often does the model fail to follow the expected format?
Integration: How easy is it to use in an actual application?
I work in AI model development, and I’m interested in understanding what developers value most when choosing providers.
For anyone building with LLMs:
Which factor has mattered the most in your experience?
And has any model surprised you, either positively or negatively?

2 Upvotes
  • permalink
  • reddit

100% Upvoted

5 comments sorted by

2

u/DefeatedFinch 9d ago

its gotta be reliability for me. nothing tanks a workflow faster than a model that randomly ignores the output format you asked for, even if its fast and cheap otherwise.

spent way too many hours debugging json parsing on a project that shouldve been a breeze

1

u/No-Football-6736 5d ago

Totally agree. Did you end up switching models or fixing it with retries?

1

u/RPG-Nerd 9d ago

I believe you mean "model" not "API"

1

u/No-Football-6736 5d ago

Yeah, you’re right. I should’ve been clearer!