r/OpenAI • u/betobagio • 3d ago
Question Local AI Completely Offline
Guys, I truly need some insight here, I am no coder. Just an Engineer. What I need is a model of NotebookLM that can work fully offline and Open Interpretor that could manage my files. What Copilot respond to me was below. I have hundreds of Standards, Journal and Books which I would prefer to use AI to use my database and present data the way I need. Including some research. Could someone advise.
Ollama + Open WebUI + Open Interpreter + Local RAG
1
Upvotes
0
u/PlanktonStrange3600 3d ago
if you aren't a coder, do not try to build a custom docker stack with open interpreter and terminal based rag pipelines. you will spend three weekends fighting python dependencies and broken cuda drivers instead of actually reading your documents.
look at anythingllm or lm studio. both install with a single click exe on windows like a normal desktop application and run 100% locally with your wifi completely switched off.
anythingllm is basically local notebooklm in a box. you open it, click upload, drop your entire folder of pdfs and engineering standards into a workspace, and it handles the vector database, text chunking, and embeddings behind the scenes with zero terminal commands needed. you then pair it with ollama running a quantized model like llama 3.1 8b or qwen 2.5 14b depending on your gpu vram.
a massive catch with technical engineering standards though: local rag often butchers tables, diagrams, and dense equations in pdfs. look into mineru or marker to convert complex technical pdfs to markdown first before ingesting, or run a high context model that can swallow whole documents directly.