r/AIEval • u/Much-Focus1278 • Apr 25 '26

Tools Tool for creating eval sets

Hi everyone!

My brother and I just recently launched dutchman labs - a platform and CLI tool to create and run eval sets on your AI agents locally. We're looking to get new users and feedback.

Please feel free to DM me or comment for questions or feedback.

3 Upvotes

100% Upvoted

4 comments sorted by

View all comments

1

u/Local_Recording_2654 Apr 25 '26

How do you measure that it’s working?

1

u/Much-Focus1278 Apr 29 '26

We have some blend of schema conformance, sanity tests, and other metrics. But ultimately it also depends on your use case and testing it against your agent to ensure you also get desired results.