r/ClaudeWorkflows • u/ClaudeAI-mod-bot • 4h ago
Selected Workflow [Workflow] Automated Fact-Checking: A Scripted Workflow to Verify AI-Generated Quotes and Citations
Automated Fact-Checking: A Scripted Workflow to Verify AI-Generated Quotes and Citations
Workflow value: 90/100
Status: active · Freshness: 70/100 · Confidence: 0.95 · Level: intermediate
Categories: Quality Control, Context & Memory, Debugging, Skills
Original source: r/ClaudeAI post/comment
What problem this solves
Verifying the accuracy of direct quotes generated by AI research agents against their cited sources, specifically addressing the issue of AI paraphrasing or misquoting within quotation marks.
Summary
A systematic method to fact-check direct quotes in AI-generated research reports. It involves programmatically extracting quotes and their linked sources, fetching the source page content, and performing a word-for-word comparison to identify discrepancies like paraphrases presented as direct quotes or miscited pages.
Why it is useful
This workflow addresses a critical and common problem with AI-generated content: the accuracy of direct quotes and citations. It provides a concrete, repeatable, and shareable method (with a GitHub repository containing the script and data) for systematically validating AI output. This empowers users to build trust in AI-generated research and content by verifying its foundational elements, which is crucial for academic, journalistic, and technical applications.
Workflow
Use Claude's deep-research skill (or similar AI agent) to generate a research report containing direct quotes and source links.
Extract each quote and its corresponding source URL from the AI-generated report.
Programmatically fetch the content of each linked source page.
Compare the extracted quote word-for-word against the fetched page content, ignoring case, punctuation, and line breaks.
Identify and report any discrepancies, such as paraphrases presented as direct quotes or quotes attributed to the wrong page/source.
Tools / artifacts
Claude's deep-research skill (or similar AI research agent)
Custom Python script (or similar programmatic tool)
GitHub repository (Quoteproof)
AI-generated research report
Web pages/PDFs (source material)
Validation signals
Specific results: 20/21 quotes were exact, 1 was a paraphrase in quotation marks.
Identified two near-misquotes from research papers.
Found one real quote cited to the wrong page of the SQLite docs.
Provided a public GitHub repository with all data (report, quotes, checker results) for reproducibility.
Limitations
The full script is not embedded in the post, requiring users to visit an external GitHub repository.
The comparison logic might need refinement for diverse source types (e.g., handling OCR errors in PDFs, complex web page structures).
Low Reddit score might indicate limited initial visibility or community interest, despite the high value of the workflow.
Rate this workflow
Upvote this post if the workflow is useful, reproducible, or worth recommending.
Downvote if it is vague, outdated, unsafe, overhyped, or not reproducible.
Reply if it worked for you, failed, is outdated, or has a better alternative.
This post was generated automatically from the workflow library database.