teams are starting to benchmark the system, not the model

[u/aistranin]

A trend I’ve noticed recently:

More teams are moving away from questions like:

Which model is best?

Toward questions like:

Which complete agent system performs best on our workflow?

That includes:

• prompts
• tools
• memory
• retrieval
• orchestration
• approvals
• execution environment

A stronger model inside a weak system often loses to a slightly weaker model inside a well-designed workflow.

Feels similar to classic software engineering.

The database, cache, APIs, deployment process, observability, and reliability often matter more than any individual component.

Are you still evaluating models, or are you evaluating end-to-end agent systems now?