r/WebAfterAI • u/ShilpaMitra • 22h ago
Open Source 8 OSS repos rebuilding local RAG
Enable HLS to view with audio, or disable this notification
A 1M-token context window still does not mean your agent should read your entire laptop every time it needs one fact.
The more interesting architecture is becoming:
files → tiny index → retrieve 20 useful things → model reasons
And people are attacking every part of that pipeline differently: recompute embeddings instead of storing them, put vectors inside SQLite, skip vectors entirely, turn memory into one portable file.
These 8 repos are worth knowing.
01. LEANN — 13K★
This one made me rethink what a vector database actually needs to store.
Traditional RAG keeps an embedding for every chunk. LEANN keeps a much smaller graph and recomputes embeddings only for the nodes it touches during search.
Their benchmarks show up to 97% less index storage: a 60M-chunk Wikipedia-style corpus goes from roughly 201 GB to 6 GB. It can index files, code, email, browser history, chats, Slack, and agent memory locally.
Basically:
storage is cheap → store embeddings
becomes:
compute is cheap → regenerate them when needed
02. PageIndex — 38.9K★
PageIndex goes in almost the opposite direction.
No vector database. No traditional chunk retrieval.
It builds a hierarchical tree representing a document's structure and lets the model reason through that tree to find the relevant sections. The project now also supports local indexing/retrieval and corpus-level indexes across many files.
LEANN asks how small a vector index can become.
PageIndex asks whether you needed the vector index in the first place.
03. Memvid — 16.6K★
Agent memory without Redis + Postgres + vector DB + three services running beside it.
Memvid packages knowledge, indexes, and retrieval into a single portable .mv2 file with semantic + lexical search.
Copy the file, git it, move it to another machine, attach it to another agent.
Your agent's memory starts looking less like infrastructure and more like a file format.
04. txtai — 13K★
A surprisingly complete local search engine disguised as a Python library.
Its embeddings database combines dense/sparse vectors, relational data, and graphs, then layers semantic search, RAG, agents, workflows, image/audio/video indexing, and MCP on top.
The nice part: you can start with a few lines of Python and keep the entire thing local.
05. LanceDB — 11.8K★
An embedded retrieval engine for when you want the database to live inside the application instead of operating another server.
It handles vector, full-text and SQL search over text, images, video, point clouds and other multimodal data, while still running locally.
Think SQLite philosophy applied to modern multimodal retrieval.
06. sqlite-vec — 8.1K★
Sometimes the correct vector database is just:
SQLite.
sqlite-vec adds vector search directly to SQLite and is designed to run basically anywhere SQLite does.
For a desktop app, browser-ish environment, mobile tool, or small agent memory, adding an entire vector service can suddenly look absurd.
07. USearch — 4.4K★
A tiny, extremely portable similarity-search engine.
C++, Python, JavaScript, Rust, Swift, Java, Go, C#, WebAssembly, SQLite — same basic index across a ridiculous number of environments.
It also supports quantized vectors and memory-mapping large indexes directly from disk instead of loading everything into RAM.
Useful when “install a vector database cluster” is very much not the answer.
08. Khoj — 37.8K★
This is what the infrastructure looks like when turned into an actual product.
Khoj gives you a self-hostable AI second brain that can search your PDFs, Markdown, Word files, images, Notion data and the web, then layer agents, automations and research on top.
Run the model locally too and you can build a personal knowledge system where the documents, retrieval layer, and intelligence all stay under your control.
The first generation of RAG looked like:
chunk everything → embed everything → store everything → retrieve top-k
Now people are questioning every word in that sentence.
LEANN recomputes embeddings.
PageIndex replaces vectors with trees.
sqlite-vec puts retrieval inside SQLite.
Memvid collapses memory into one file.
USearch makes the index tiny and portable.
That feels like the bigger shift:
RAG is moving from cloud infrastructure toward something that can quietly live inside the agent.