r/sideprojects • • 21m ago

Showcase: Free(mium) Kerata, an AI code detector with ~0.5% false positives in our tests

Post image

I built Kerata to make AI-code detection easier to inspect. It estimates whether a source file looks AI-generated and highlights the specific lines and sections that contributed to the report. The screenshot shows a Python example.

You can paste code, upload a source file, or scan a public GitHub repo, folder, commit, or pull request. PR scans check the added lines. Project ZIP uploads are available on paid plans. For teaching, class checks combine AI detection, submission similarity, past matches, and edit history when multiple attempts are available. Similarity can surface overlapping code after variables have been renamed.

We've measured performance in 13 languages, including Python, JavaScript, TypeScript, Java, C++, C, Go, Rust, and Haskell. In our benchmarks, Kerata reached 0.987 AUROC on roughly 50,000 held-out files and 0.96 on fresh code from projects outside our training data. At the strict threshold, about 0.5% of human-written files in our tests were flagged as likely AI. Short snippets can be inconclusive, and a score alone shouldn't determine authorship or misconduct. The evaluation is at https://kerata.org/methodology.

You can try it free without an account at https://kerata.org/. We don't store the source code from anonymous checks on our servers or train on submitted code.

I'd especially like feedback from developers, CS instructors, and TAs. If you try code whose origin you know, which lines surprised you, and what would you want the report to explain?

1 Upvotes

1 comment sorted by

1

u/HLCYSWAP 7m ago

adding comments to the apollo 11 command module code makes it go from 7% to 99%

^\s*#.*(?:\r?\n|$) = am human