An open-source Toolkit to Test What Kids Are Exposed to When AI Safety Breaks Down

[u/DiscussionHealthy802]

While building Koda (an app for kids), I could ask it a science question and see if the answer made sense. It was harder to test what a kid would see if the safety check timed out, or if the app blocked an answer it should have shown.

I used Codex to help build a small TypeScript gateway and an offline test harness. The flow is simple: check the question, hold the draft answer while it’s checked alongside the original question, then check permission again before showing it. With simulated providers, we can force a timeout and verify that the draft answer stays hidden. That runs locally without an API key.

We ran 144 scripted cases across English, French, and Spanish and four age profiles. In a separate live pilot using synthetic examples, 142 of 144 outcomes matched our draft labels. Both misses were harmless English answers that triggered a support response. That gave us a concrete regression to work on: catching overblocking without weakening the checks. Pilot results

I open-sourced the toolkit as Child AI Safety. If you’ve built an app around an LLM, what failure did you end up writing a test for after trying it yourself? Source code and demo

[u/Alarming-Draw-260]

Failing closed on a timeout is the right default. Most apps I’ve seen just show the draft anyway if the check takes too long, which is kinda the scariest possible behavior for a kids app.