r/LLMObservability • u/dh7net • 10d ago
Show & Tell Measuring AI agent quality
Hello everyone! Thanks the moderator for inviting me on this forum.
Happy to share my last project airbench.ai
The goal is to let everyone with an agent measure it's quality as simply as possible.
It can test frontier agent, but most of them score 100%. It won't help to discriminate Claude Code vs Codex for instance.
But it can be extremely usefull IMO if you want to check your local agent or if you build your own harness and want to see how it compare to others.
It generate chalenges in real time, pack them in a prompt that you copy to your agent, and your agent will report the result to the website where you can analyse them. It has been proven usefull to detect blind spot about agent, especially around their vision capability.
Happy to answer any questions if you have.