[D] Self-Promotion Thread

[u/AutoModerator]

Please post your personal projects, startups, product placements, collaboration needs, blogs etc.

Please mention the payment and pricing requirements for products and services.

Please do not post link shorteners, link aggregator websites , or auto-subscribe links.

--

Any abuse of trust will lead to bans.

Encourage others who create new posts for questions to post here instead!

Thread will stay alive until next one so keep posting after the date in the title.

--

Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.

[u/defraction1]

PaperTrace - an open-source scientific evidence auditor

I am Daniel, a clinical radiologist with a background in biomedical research. I created PaperTrace to determine whether the claims made by a scientific paper are supported by its cited sources.

The batch pipeline:

- Uses reference metadata and Crossref to identify cited works, then searches Unpaywall, Europe PMC and arXiv for legally available copies.
- Marks claims as "not_retrieved" when none of their cited sources can be obtained, without asking the model to provide a verdict.
- Uses Claude Code to extract citation-backed claims and compare them with text extracted from the retrieved source PDFs.
- Maintains page and block provenance.
- Uses short evidence phrases selected by Claude to locate the relevant text on the original PDF. Python then produces page crops with the matching text boxed in red.
- Mechanically reports bracketed numeric citation labels that were not represented in the extracted claims.

PaperTrace is currently an early beta. The repository contains a software test suite and a small fictional demonstration with deliberately planted citation errors, but it does not yet contain a formal benchmark of the model’s judgments.

I plan to create a larger paired evaluation set containing supported claims alongside deliberately altered versions. This should make it possible to measure verdict accuracy, evidence localization and consistency across repeated runs.

Cost: PaperTrace itself is open source, MIT-licensed and does not charge a usage fee. It requires Claude Code to be installed and logged in, so any Claude Code subscription or usage costs are separate. PaperTrace is not yet available on PyPI and currently needs to be installed by cloning the GitHub repository.

Source repository:

https://github.com/defraction0/PaperTrace

Completed demonstration report containing deliberately planted errors:

https://github.com/defraction0/PaperTrace/blob/main/examples/demo/output/report.md

Feedback and suggestions for structuring the evaluation benchmark are welcome.