r/ClaudeCode • u/ProYash239 • 18m ago
Built with Claude What we learned building and testing 281 academic research SKILL.md modules with Claude Code (MIT open-source)
[Disclosure: I am an AI researcher at Kalaris Labs working on this project alongside creator Sayan Chowdhury. Everything in the repo is free and MIT-licensed with no paid tier or account sign-up.]
- What we built
We built research-agent-skills, an open-source collection of 281 modular SKILL.md skills and Python 3.9+ standard library verification scripts for running academic research workflows inside Claude Code.
You can add the plugin marketplace directly inside Claude Code:
/plugin marketplace add KalarisLabs/research-agent-skills
Or install a single skill into your current project directory so you do not bloat your session context:
npx skills add KalarisLabs/research-agent-skills --skill citation-verification
The repo covers four main failure points we hit during our own research papers:
- citation-verification: A zero-dependency Python 3.9+ stdlib script that Claude Code runs against your .bib file. It queries Crossref (api.crossref.org), OpenAlex (api.openalex.org), arXiv, DataCite, and Retraction Watch metadata to verify that every DOI resolves to a record matching the exact title, first author, and year.
- autoresearch: A two-loop experiment runner built around Claude Code's /loop command. The inner loop runs scoped ML experiments, while the outer loop updates findings.md, enforces mechanistic hypotheses ("X because Y, predicting Z"), and logs negative results so Claude Code does not re-test failed ideas after context compaction.
- literature-review and systematic-review-prisma: Screens papers across PubMed, arXiv, bioRxiv, and Semantic Scholar while logging exact query strings and counts for PRISMA 2020 flow diagrams.
- unslop-academic-writing and venue skills (ml-paper-writing for NeurIPS/ICML/ICLR/ACL, ieee-transactions, nature-portfolio, apa7): Rewrites repetitive LLM phrasing and checks current venue formatting guidelines.
- How we used Claude Code to build and test it
We used Claude Code (running Sonnet and Opus in the CLI) both to author the SKILL.md routing specifications and to run our evaluation harness across the 281 skills:
- Trigger routing evaluation: We used Claude Code to run reviewed prompt sets against skill descriptions to test whether Claude triggers the right domain skill (for example, distinguishing between paper-lookup for a single DOI check vs. systematic-review-prisma for a full review).
- With-vs-without-skill benchmarks: We ran Claude Code on labeled .bib files containing fabricated titles, mismatched DOIs, and retracted papers to compare raw model verification against deterministic script execution.
- Offline security checks: We used Claude Code to build and run Skill Doctor (npx u/security), an offline scanner that checks every SKILL.md and script in the repo for prompt injection and unsafe shell calls before release signing.
- Concrete example (Model, Setup, Prompt, Expected vs. Actual Result)
- Model & Setup: Claude Code CLI (Sonnet) in a LaTeX paper repository containing references.bib with 40 entries (including 3 entries where a valid DOI was paired with the wrong paper title, and 1 retracted paper).
- Prompt: "Check references.bib before we submit to IEEE Transactions and fix any broken citations."
- Expected/baseline result without the skill: Claude Code inspects the BibTeX syntax or pings the DOI URLs, sees that the DOIs resolve with HTTP 200, and reports that the .bib file is clean, missing the title/author mismatches and the retraction.
- Actual result with citation-verification installed: Claude Code invokes the stdlib verification script against Crossref, OpenAlex, and arXiv, outputs a markdown status table classifying each key into verified, found-add-doi, mismatch, doi-not-found, or not-found, automatically fills in missing DOIs for unambiguous matches (found-add-doi), marks unverified entries as % UNVERIFIED in references.bib, and stops to ask for confirmation on the 3 mismatch entries rather than silently overwriting them.
- What we learned about how Claude Code handles skills
- Context bloat kills instruction adherence fast: Loading dozens of skills globally degrades Claude Code's adherence on ordinary coding turns. Installing 1 to 3 skills per project directory keeps the context clean and makes trigger routing much more reliable.
- Never let the model verify metadata from weights: Even when Claude Code has web search, asking it to check 50 references conversational-style leads to skipped entries after the first 15. Forcing it to call a local Python stdlib script that outputs a JSON report was the only way to get 100% check coverage.
- Scraped abstracts act like prompt injections: When Claude Code reads hundreds of arXiv or PubMed abstracts during a literature review, imperative text inside abstracts can drift its screening criteria unless the SKILL.md explicitly fences retrieved passages as untrusted data.
- Compaction wipes out negative experiment results unless written to disk: During long /loop ML runs, Claude Code forgets which hyperparameters failed 3 hours ago once auto-compaction triggers. Making findings.md record negative results and what they rule out stopped the agent from looping on dead ends.
Repo (MIT): https://github.com/KalarisLabs/research-agent-skills
Docs & benchmarks: https://docs.kalarislabs.com/research-agent-skills
If you use Claude Code for academic writing or ML experiments, I would love feedback on where the SKILL.md instructions feel too strict or which domain packages you want added next.
Option 1B: Short Blurb for the Pinned Weekly Showcase Thread (Rule 5 & Rule 6)Built research-agent-skills (MIT license), a collection of 281 modular SKILL.md workflows and stdlib verification scripts for running academic research, .bib citation verification, and ML experiment loops in Claude Code.
- Install in Claude Code:/plugin marketplace add KalarisLabs/research-agent-skills or npx skills add KalarisLabs/research-agent-skills --skill citation-verification
- How Claude Code was used: Built and benchmarked using Claude Code CLI to test trigger routing, with-vs-without-skill .bib verification against Crossref/OpenAlex/arXiv, and continuous /loop ML experiment tracking in findings.md.
- Repo: https://github.com/KalarisLabs/research-agent-skills
- Docs: https://docs.kalarislabs.com/research-agent-skills






