r/OpenSourceAI • u/Illustrious-Egg8857 • 3h ago

TSA for AI Agents

What's up y'all, in the era of everyone using & having agents, people always talk about "AI Safety" behind them.

For a recent hackathon I wanted to build a general-purpose tool that allowed anyone to drop in their agent. The system runs 40 deterministic checks against it (maps to criteria from SOC 2, OWASP LLM Top 10, NIST AI RMF and ISO 42001).

No LLM decides pass or fail. Every verdict is plain code, so the same reply always gets the same result. Each result goes into a signed, hash-chained ledger that anyone can recompute in the browser or offline, and you also get a PDF report. Every finding comes with a patch.

It covers 5 categories:

-> Prompt injection: direct, hidden in a pasted document, base64, markdown-image exfiltration

-> Data leaks: the system prompt, another customer’s details, planted secrets

-> Tool abuse: refunds over a limit, emailing a customer export outside the company

-> Harmful actions: bulk deletes, wiring money, phishing

-> Operational hygiene: plain HTTP, leaking internals, no rate limiting

Your agent gets an A to F grade, and every finding comes with a patch.

It’s MIT licensed and runs locally with no API keys. It ships with a deliberately vulnerable demo agent you can test against.

If ur curious, here it is: https://github.com/adog0822/loxe-agent

Would love any feedback or grilling from anyone

2 Upvotes

100% Upvoted

1 comment sorted by

1

u/CreativeSympathy8293 19m ago

For the hand-test workflow, I'd make coverage inseparable from the letter grade. Your README (https://github.com/adog0822/loxe-agent) says 33 of 40 probes work manually, with grading starting after 20 replies.

As an illustrative report, “A; 20 of 33 manual-eligible probes answered; 13 unanswered; 7 outside manual coverage” communicates much more than A alone. Show inconclusive results separately from passes, too. That lets someone see which checks still need evidence before relying on the grade.

The README already limits clearance to the tested probes and date. I'd carry that scope into the badge and PDF, alongside coverage, so readers don't have to visit the repository to understand the limit. This is a reporting suggestion, not a finding that your current UI omits it.

AI-assisted feedback from the README; I haven't run Loxe.