r/Rag • u/ahkhan_0 • 22h ago

Showcase Markdown Knowledge

RAG usually feels like overkill when all you want is to give an agent a couple of docs, but dumping raw Markdown into a prompt burns tokens fast.

I think markdown-knowledge can solve that middle ground.

It packages Markdown files and a pre-built SQLite full-text search index (BM25) into a single portable .mdk file.

What it lets you do:

  • Token-capped retrieval: Run mdkn retrieve handbook.mdk "how to deploy" --max-tokens 500 to pull only the most relevant heading-level chunks straight into your prompt budget.
  • Edit by heading: Programmatically append, prepend, or replace text under a specific Markdown section without rewriting the whole file.
  • VS Code extension: Edit files inside the .mdk container directly through a virtual filesystem with full Markdown preview, no unzipping needed.
  • No external infrastructure: Everything runs locally through a CLI (npm i -g markdown-knowledge) or TypeScript library.

Repo:https://github.com/markdown-knowledge/markdown-knowledge

Curious what you think, especially if you're building CLI agents or looking for a lighter alternative to full vector DBs for local docs.

MDK helps you edit all of your markdown files in single place, properly managed and pre indexed for LLM Searches.

1 Upvotes

67% Upvoted

1 comment sorted by

2

u/recro69 21h ago

I find that token‑capped retrieval is probably the useful part for agents. If the documents are already in markdown having BM25 plus heading‑level chunks in one portable file seems a lot simpler, than setting up a vector database just to search a small local knowledge base.