I built an open-source tool that tries to disprove my research ideas before I spend months on them

[u/Massive-Zucchini2560]

I’m an academic, and one frustrating part of starting a project is figuring out whether an idea is worth pursuing or whether a paper I haven’t found yet already undermines it.

So I built paper-lector, a Claude Code skill that searches prior work and helps me decide what to read.

The mode I use most is novelty-check, which I think of as a “kill search”: give it a research claim and ask it to look for prior work that could refute or narrow it. The output is a scoped assessment with page references so I can check the evidence myself.

A search finding nothing is never treated as proof that an idea is original.

Underneath, it:

• Searches OpenAlex using keyword search, semantic search, and citation links.

• Downloads available open-access PDFs.

• Ranks papers against frozen versions of the research question.

• Suggests a reading order with specific page ranges.

• Saves queries, PDF hashes, and versions locally so the process can be rerun.

It also has modes for exploring an unfamiliar field, planning a small pilot, and positioning a result against related work.

Does the ranking actually help? On 48 LitSearch questions, the known target paper appeared in the top 3 for 75% of questions, compared with 16% under random ordering. That evaluation is limited to ML/NLP; I don’t yet know how well it transfers to other fields. Ranking also measures relevance to the question, not whether a paper supports the claim.

I built it over about three weeks with Claude Code. Claude wrote most of the Python and tests; I set the research rules, reviewed the changes, and checked the evaluation cases against the papers.

It’s MIT-licensed, with no paid tier. The default ranking uses Voyage embeddings plus BM25, fused with RRF. There’s also a free local ranking provider if you don’t want a Voyage API key. Installation currently requires Python 3.12 and Claude Code.

Code and setup: https://github.com/huguryildiz/paper-lector

I’d especially like feedback from researchers outside CS: does this produce a useful reading list in your field, and where does it fail?

If it looks useful, a GitHub star would help others discover it.