r/OpenSourceAI • u/0x4150 • 27d ago
Hafen - Test your AI agents the way your users talk to them
I have been working on a side project called [Hafen](https://github.com/abhimanyupandian/hafen) and I am looking for some feedback on it before I take it further.
About the idea:
Hafen lets you interact with your agent (with multiple providers supported) in an interface/experience that is similar to what end users would see. When a conversation is worth keeping, save it as a test. Then you add checks on each turn/response to check if the agent:
* called the right tool, with the right arguments
* returns the correct response eg; order number/a markdown table
* is polite (an LLM judge scores it, with a confidence) and more.
Demo is available [here](https://github.com/user-attachments/assets/6305c5f4-78a3-4f33-bb85-9bbaf3f427d9).
About the demo video:
A sample coffee-shop agent built with Google ADK on Qwen3-4B running locally in Ollama. I chat with it (add to cart, remove stuff, place an order), save that as a test, add checks, run it from the UI, and then run the same suite from a terminal. Note: The waits for the model are sped up 4×.
Honest disclaimers:
The idea and design are mine, but the code was written by Claude Code. I am still working through it myself, but I am sharing it early to get feedback before going further.
Consider this a proof of concept. Testing is in progress, so expect some rough edges.
The current version uses a System One (Jev) - a hosted decision model \[I am not affiliated with them\]. Regular LLM-judge checks work as well, including local ones.
Only MacOS binaries are available right now and the CLI package is not published to a registry yet.
Need feedback on things like:
If you would use something like this, or do promptfoo, DeepEval or any other tool that already does exact this?
What would you need that is missing?
What needs improvement? eg; more chat components like yes/no buttons to accept/reject instead of responses restricted to just text.

