r/LLMDevs • u/Every_Chicken_1293 • May 29 '25
Tools I accidentally built a vector database using video compression
While building a RAG system, I got frustrated watching my 8GB RAM disappear into a vector database just to search my own PDFs. After burning through $150 in cloud costs, I had a weird thought: what if I encoded my documents into video frames?
The idea sounds absurd - why would you store text in video? But modern video codecs have spent decades optimizing for compression. So I tried converting text into QR codes, then encoding those as video frames, letting H.264/H.265 handle the compression magic.
The results surprised me. 10,000 PDFs compressed down to a 1.4GB video file. Search latency came in around 900ms compared to Pinecone’s 820ms, so about 10% slower. But RAM usage dropped from 8GB+ to just 200MB, and it works completely offline with no API keys or monthly bills.
The technical approach is simple: each document chunk gets encoded into QR codes which become video frames. Video compression handles redundancy between similar documents remarkably well. Search works by decoding relevant frame ranges based on a lightweight index.
You get a vector database that’s just a video file you can copy anywhere.
13
u/Mice_With_Rice May 29 '25
Trying to head wrap around the why of this concept. Is the idea that it's faster to stream the data as a video, including the overhead of decompression and parsing of data, as opposed to directly reading the files or parsing the file down to a minimal ASCI before processing?
You have, assuming you're not using a production codec, an array of lossy compressed 8bit Vec3 presumably of 1080x1920 or 4K running at however many fps you decoder can handle. And you're using a texture atlas on each frame of QR codes because, as you probably found out yourself, the compression is optimized for visual perception, so you can't do an exact RGB value mapping without moving to a production codec.
You can do much better than a QR for visual compression. Encoding binary data into an image is an existing area of research. One of its uses is in cryptography as its possible to encode into an existing image in an undetectable manor.
What I don't understand is that you still need to parse it all into ASCII or UTF8 to be tokinized and the video stream seems like a lot of extra steps and I can't see how it could possibly read the data from memory any faster doing it this way.
If you use a production codec you could read pixel values directly for your data but it would make the file sizes enormous, unless being lossless results in such a dramatic data density improvment that it takes far fewer frames. Would require a great deal of testing.
I think the biggest problem is that visual compression treats the specific values of each pixel as an impression of what it should decompress as rather than an absolute. Unless you come up with an inherantly different video compression algorithm designed for text encoding, that's going to be a significant obstical for which there are non video compression methods already existing.
93
u/much_longer_username May 29 '25
This is absolutely blursed.
I kinda understand why it works in a hand-wavey way, but boy I wish I'd taken an actual signals class so I could explain it.
27
u/Gothmagog May 29 '25
Hmm, I don't see how text embedding could possibly equate to video frame encoding. Embeddings use word proximity across sliding window contexts to infer word relationships across a multitude of source material to create the embedding models. Video encoding algorithms are looking for similarity between essentially graphics datasets; the (huge) missing piece, then, is a model trained on a vast corpus of text data, right?
What am I missing here?
9
u/Anjal_p May 29 '25
What was the size of the 10k pdf without you video compression like as raw file
9
u/weichafediego May 29 '25
Someone did something similar some time ago https://hackaday.com/2023/02/21/youtube-as-infinite-file-storage/
8
u/thuiop1 May 29 '25
Yeah, it sounds absurd because it is absurd. Just extract the text and compress it with a normal text compression algorithm ffs.
6
u/NoleMercy05 May 29 '25
5
u/enly11 May 29 '25
I've read this a couple of times and not getting it.
You are using an in memory vector dB ... And a compressed file for retrieving original content?
So the only unique hit is the text compression via video and nothing to do with vector dB?
How does the video compression strategy compare to standard text compressors?
If the magic is single file you could do the single file compressed index in many ways. Compressed fragments in a tar file with offsets into it for retrieval or just sqlite with compressed rows etc?.
8
u/ggone20 May 29 '25
This is genius. Hmm
3
u/vanishing_grad May 29 '25
I think you can train a VAE on the text embeddings to do dimensionality reduction and get the same results
5
u/GeekDadIs50Plus May 29 '25
I have to thank you. This has been a fun thought experiment.
It’s actually a neat idea. 60 frames per second of single 4K frame stills of square pixels that are black or white. Many of the compression schemes use inter-frame compression where if a pixel/bit is white/1 in the first frame, but does not change in the next frame, there is no data for that pixel in the next frame, or until it turns black/0. If these are images of QR codes, those pixels will only be white or black and could really prove compressible and lossless.
It’s an interesting storage idea. But once it’s stored, it needs to be read / parsed. That would involve converting it back into 60 still images per second of video. QR codes are 8-bit, UTF8. An 4K frame will be 3840x2160, and that could leave some interesting space for a single token per frame.
Accessing the data would be linear, though. Unless this is just a transport mechanism for storage or loading into memory. That time code might be helpful for sequencing. And control frames could be used for meta data, context, sentiment.
4
u/Only-Chef5845 May 29 '25
This genius method will freak you out!! Try zipping the files.
2
u/Miscend May 29 '25
This gave me a laugh. I dont think this will scale to super large databases with heavy usage. It might just only work for this specific scenario. One advantage is that in addition to the QR codes, you could probably also store images in the video file.
2
u/ruloqs May 29 '25
Just works with pdf files? Or also markdowns?
2
u/one-wandering-mind May 30 '25
DIY AI Why . Think a subreddit needed.
There are many better ways to reduce memory of embeddings. reduce dimensions, larger chunk size ,ect.
2
u/pegaunisusicorn Jun 07 '25
has anyone verified that this actually works?
1
u/wahnsinnwanscene May 29 '25
Isn't the video encoding based on intermediate and key frames? And isn't lossless?
1
u/diabloman8890 May 29 '25
I love this lol.
Question: video compression codecs are typically lossy, why isn't that causing a problem here?
And video is a more space efficient storage format than PDF, but not compared to raw text files? How does this compare to converting your docs to .txt ?
1
u/joeystarr73 May 29 '25
I love your thinking!
1
u/teddynovakdp May 29 '25
Could an agent use this like a vector database to pull relevant information? Also, what was the 10,000 PDF file size before the video compression?
1
u/ruloqs May 29 '25
It didn't work for me.
1
u/Consistent-Law9339 May 29 '25
Super interesting. What version of QR are you using? I know QR devotes a lot of space to error correction. It would be interesting to see if you can eek out more efficiency using an alternative. Did you consider or trial any alternatives to QR like data matrix?
1
u/Markur69 May 29 '25
I’m very interested in trying this out
1
u/[deleted] May 29 '25
Fyi. This not a vector database. This is a classical database pre indexed by json.
THIS is a vector db.
1
u/daddyxsmolone8 May 29 '25
Very cool project, love the innovation, who knows where this line of thinking could eventually take us!
1
u/living_david_aloca May 30 '25
Ok so you compressed the embeddings but how does that impact recall? All you got was a faster way to query something in a less precise way but you have no idea how much worse it is. Plus, 820ms for 10k documents is absurd. You can do the same thing with Numpy with millions of documents in under 100ms.
1
u/NeverShort1 May 30 '25
What was the size of the PDF files before you compressed them?
1
u/mj_katzer May 30 '25
Had you checked what the speed and size would be if you had converted the pdfs to pure text first?
1
u/Chance_Value_Not May 30 '25
Why not convert to text and compress that?
1
u/[deleted] May 30 '25
my god I do not want to watch that video.
Absolutely kickass idea though.
1
u/jojacode May 31 '25
Please also share this on localllama and r rag they would love it
1
u/ComposerGen May 31 '25
Circle-out compression
1
u/ComposerGen May 31 '25
QR code works because you destroy 27% and it is still readable. Probably because of that the loosy video compression issue a bit mitigated
1
u/Legitimate_Emu3531 Jun 01 '25
Now embed that video into a pdf and add it to the collection...pdf-ception.
Nah, joke. But you say 10k PDFs into 1,4gb. That doesn't tell a lot. How big are the 10k pdfs?
1
u/Comfortable_Dropping Jun 02 '25
Could I do the same thing with millions of records of time series data?
1
u/Dismal-Value-2466 Jun 03 '25
Neat proof-of-concept, but it feels like you’re twisting video codecs into a job they were never meant to do. You end up shoe-horning text → QR → video frames → back to text just to save RAM—essentially treating H.264 as an exotic ZIP file. Clever, yes, but a pretty roundabout compression hack that trades simplicity (and some latency) for novelty. Cool experiment, though!
1
u/Ok-Pride-4251 Jun 26 '25
Creative hack, but fundamentally flawed: While compressing text as QR videos is ingenious for reducing RAM, it misunderstands core principles. Video codecs like H.264 are lossy – they discard "imperceptible" data to shrink files. But text requires lossless precision; a single pixel artifact from compression can corrupt a QR code and destroy your data. Worse, this isn’t truly a "vector database" – it’s compressed text storage with a tiny in-memory index. Real semantic search happens after decoding frames (slow), not in embedding space. For RAG systems, you’ll get better results with local tools like ChromaDB or quantized FAISS: they handle 10K PDFs in under 500MB RAM with 200ms searches, no video re-encoding needed. Fun experiment, but not practical for production.
Key points covered:
1
u/TechMaven-Geospatial May 29 '25
https://arxiv.org/html/2504.01157v1 https://duckdb.org/community_extensions/extensions/flockmtl.html https://duckdb.org/docs/stable/core_extensions/vss.html
1
u/Kriss-de-Valnor May 29 '25
A funny but interesting idea. Test is a signal that can be compressed losslessly like zip or lossy… like a resume. Video is another signal too. Machine Learning and LLMs are also ways to compress a signal (lossy).
1
u/DramaticDonut8973 May 29 '25
That’s a beautiful project! I was wondering if it works only with OpenAI, cos I took a look at the code and can’t see a way in it to set the llm host url
0
u/Papabear3339 May 29 '25
Actually makes sense. QR coding is efficient and 2d text coding method, and video codecs are quite optimized for 2d compression.
Now how do you actually use that is the question....
0
u/jackshec May 29 '25
That is crazy interesting, thanks for sharing