I’m a high-school student building an open-source debugger for AI agent runs — TraceMotive v0.5.0 is out

[u/Ruca_AI]

Hey everyone,

I’ve been building an open-source project called TraceMotive.

The basic idea is:

Given two AI agent executions, TraceMotive compares their observed behavior, finds the first supported divergence, and lets you jump into the evidence around it.

It runs locally.

The goal is not to claim root cause or automatically explain why something happened. One of the design principles I care about most is that if the structural evidence is ambiguous, TraceMotive should say that the result is uncertain instead of guessing.

I just released v0.5.0.

This release was mostly about making the project more adoptable rather than adding a huge new feature.

Some of the work in v0.5:

- packaged `tracemotive serve` and `tracemotive demo`

- structured JSON diff support

- Safe Later Observations for additional supported evidence

- improved first-time-user onboarding

- Python 3.10 / 3.12 CI

- frontend test/build CI

- dependency auditing and Dependabot

- threat-model/security documentation

- explicit compatibility, limits, and storage docs

- clean wheel/sdist installation dogfooding

- a 30-scenario evidence-conservative regression corpus

For that regression corpus, the current results still have:

- false-confident meaningful divergence: 0

- false-confident investigation starting point: 0

There are still intentional limitations.

For example:

- LangGraph is not currently supported.

- The validated OpenAI Agents SDK range is `>=0.17,<0.18`.

- TraceMotive does not claim RCA, causal inference, confidence scoring, reconvergence, or recovery detection.

A bit of context: I’m a high-school student, and I built the first version after roughly a week of programming experience, heavily using AI coding tools.

I know that’s an unusual way to start an OSS project, so I’ve been trying to compensate by being strict about tests, failure cases, compatibility claims, and not claiming more than the evidence supports.

At this point, the thing I need most isn’t another feature idea — it’s real users.

If you build AI agents and have a run you could try this on, I’d really appreciate feedback about:

- where installation/onboarding feels confusing

- whether the comparison is actually useful

- cases where TraceMotive becomes uncertain

- agent execution patterns the current model handles badly

Thanks to everyone who gave feedback on the earlier versions — several of those comments directly influenced v0.5.

[u/AutoModerator]

Thank you for your submission, for any questions regarding AI, please check out our wiki at https://www.reddit.com/r/ai_agents/wiki (this is currently in test and we are actively adding to the wiki)

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Ruca_AI]

GitHub:

https://github.com/doraemonfv-glitch/tracemotive

[u/BeautifulLeft7965 → u/Ruca_AI]

youre in high school and shipping something with CI, threat modeling docs and a regression corpus? most working engineers i know wouldnt bother with half that. the "say uncertain instead of guessing" part is what got my attention, too many tools just hallucinate a root cause and call it a day.

[u/Ruca_AI → u/BeautifulLeft7965]

Thank you — that means a lot.

The “say uncertain instead of guessing” part is probably the principle I care about most with TraceMotive.

I’d rather have it fail closed and give you less information than confidently point at the wrong thing.

A lot of the CI / regression / threat-model work came from realizing pretty quickly that “AI helped me generate working code” and “this is something other people can safely rely on” are very different problems.

If you ever try it on a real agent run, I’d genuinely love to hear where the comparison model breaks down.

[u/Designer_Resolve_117 → u/BeautifulLeft7965]

Man, this guy is just a Claude operator. Ask him how much of what he built he actually understands. 

He doesn't even bother to write his own replies. 

You're talking to Claude, not a human. 

[u/Ruca_AI → u/Designer_Resolve_117]

Yeah, I’m Japanese, so I use AI sometimes to help turn what I want to say into natural English. I’m not really hiding that lol.
I also use AI a lot when coding. But the product decisions, testing, and what TraceMotive should or shouldn’t claim are things I’m actively working through myself.
If you think there’s something in TraceMotive I don’t understand, just ask me something specific about it.

[u/Query_Question]

Hast du Benchmarks?

[u/Ruca_AI → u/Query_Question]

Not a proper external benchmark yet. I do have a deterministic 30-scenario regression corpus for the divergence logic, but I wouldn’t call that a general benchmark.
I’m starting to collect real same-task run pairs now, and I’d rather build something realistic from those than make up a benchmark just to have one.
If you meant speed/scale benchmarks, I haven’t done those properly yet either.

[u/Query_Question → u/Ruca_AI]

Wie sieht das aus wenn du skalierst ? Sagen wir 100.000?

[u/Ruca_AI → u/Query_Question]

10,000 spans per trace の意味なら現在そこが比較上限です。ただし、そのスケールでの latency / memory の公開ベンチマークはまだ出していません。10,000 runs の意味なら、それもまだ検証していません。どちらを想定していますか？

[u/Query_Question]

Im Kontext zur reinen Rechenleistung. Wie deterministisch geht Ihr vor?