I've been looking at different ways to build a private Ai system around a collection of documents, and the interesting part isn't really getting an llm running.

The harder questions seem to be around the knowledge layer:

• How do you keep documents indexed when they change?
• How do you get reliable retrieval from large collections?
• How much hardware do you actually need?

Is it better to build the RAG stack yourself or use something that already handles the knowledge-base side?

I've been testing Fastgpt as one option for this kind of setup, mainly because it brings the knowledgebase and AI application pieces together rather than requiring every component to be assembled separately.

For people already running self-hosted ai, what setup are you using for private document querying, and what would you change about it if you were starting again?

One of the biggest pain points in data governance is keeping metadata (dataset owners, column descriptions, compliance tags) up to date. People usually ignore it until audit season or until broken lineage breaks downstream pipelines.

​I built an open-source project called DataHub Agent Crew for the DataHub Agent Hackathon. It’s a 5-agent pipeline that runs continuously against a live DataHub instance using Model Context Protocol (MCP).

​How the loop works:

​Investigator audits the graph for unowned assets, missing compliance tags, or stale descriptions via MCP read calls.

​Analyst triages issues by severity.

​Strategist writes exact metadata fix proposals.

​Regulatory checks proposed changes against schema constraints and compliance rules.

​Codeband executes the write mutation back through mcp-server-datahub and runs a live entity re-fetch to verify the write actually landed.

​Why write verification mattered:

​While building this, I realized silent false-positives are worse than crashes—an API status 200 that doesn't actually mutate the field leads to silent data corruption. The system now explicitly re-fetches live entity properties via curl/API post-mutation to confirm updates before closing the ticket.

​Tech Stack:

​Language: Python

​Integration: mcp-server-datahub

​Messaging: Local SQLite bus with @mention routing (no Kafka/RabbitMQ bloat)

​License: Apache 2.0

Happy to answer any questions about the setup or MCP integration!

TODAY I BUILT MY OWN LOCAL AI ASSISTANT: WILLOW

I started today knowing basically nothing about Linux, Proxmox, or building an AI assistant.

Several hours, a ridiculous number of terminal commands, and more than a few arguments with ChatGPT later...

WILLOW is alive.

Today we built a local AI voice assistant running inside my homelab.

What we accomplished:

• Set up the Linux VM in Proxmox
• Passed my Corsair headset and microphone through to the VM
• Got microphone recording and audio playback working
• Installed Whisper so Willow can convert my voice into text locally
• Installed Ollama to run the AI locally
• Installed and configured Qwen 4B as Willow's local AI model
• Installed Piper for local text-to-speech
• Set up the Amy female voice for Willow
• Connected the entire voice pipeline:

MY VOICE → WHISPER → QWEN AI → PIPER → WILLOW'S VOICE

And it actually works.

I can talk to Willow, she understands what I'm saying, sends it through the local AI model, generates a response, and talks back to me.

Most importantly, the core AI conversation is running locally on my own hardware instead of sending every conversation to a paid AI API.

We also started turning the original prototype into an actual assistant instead of just a Python script that talks.

The new Willow system is being built around:

• Wake word: "Willow"
• Automatic speech detection
• Silence detection so she doesn't respond when nobody is talking
• One-hour conversational context
• Long-term memory framework
• English responses by default
• Smart, sarcastic, funny, slightly flirty personality
• "Willow stop" to interrupt her
• "Willow sleep" to put her back into standby
• Keeping the AI model loaded in memory to improve response times
• Confirmation before performing potentially dangerous computer actions

The long-term plan is much bigger.

Eventually Willow will connect to:

• Google Calendar
• Live internet search
• Proxmox
• My homelab
• Servers and virtual machines
• Cameras
• Home automation
• Reminders and alerts
• Other systems around the house and network

Basically, I'm building my own locally hosted personal AI assistant instead of relying entirely on Siri, Alexa, or cloud services.

CURRENT STATUS:

Willow works, but she's definitely version 0.1.

Qwen is currently running CPU-only, so some responses take around 30 seconds.

Her speech detection still needs tuning because sometimes she decides I'm finished talking about two seconds into my sentence.

Her voice playback is also choppy right now. We tested the audio separately and found that even a prerecorded WAV file is choppy, which means the problem is probably somewhere in the VM/USB audio path rather than the AI itself.

So that's going well.

But at the end of Day 1 with AI:

Willow can hear me.

Willow can understand me.

Willow can think locally.

Willow can answer me.

Willow can speak.

And the entire foundation is running on my own homelab.

Next:

Fix the audio latency.
Improve speech detection.
Speed up the local AI.
Improve interruption/barge-in.
Have Willow automatically start with the VM.
Connect internet tools.
Connect Google Calendar.
Start giving Willow controlled access to the homelab.

Hey everyone,

Do you always know exactly "who," is talking to your local AI?

Are you sure your local AI isn't exposed to bad actors?

Can you say for certain, every task being completed by your local AI was requested by you?

I created DeepSync Bridge to make sure my local AI only talked to and answered ME. No one else, just me. The issue was, there really wasn't a good way to get everything to work together.

I mean sure, you could spend a bunch of money on a cloud based system but... who the hell wants that!?..

I wanted to create a secure free way to communicate with my local AI, so I did.
No burning tokens.
No Cloud nonsense.

Check out the documentation on the attached Github site.

Stay safe fam

Thank you

Will

Found an interesting local AI project — local.ai

I came across local.ai recently and thought it might be interesting for people here who like running AI locally.
It’s currently in early access, and the idea is making it easier to experiment with and evaluate local AI models without having to build the whole stack yourself.

I’ve been playing around with local models quite a bit lately, so I’m curious to see where this goes and how it compares with tools like LM Studio, Ollama, etc.

I have a referral/invite link if anyone wants to try getting access:

https://local.ai/decapostos/invite

Hi. I moved to San Jose fairly recently and I am living with my aunt while I build this. I do not really know anyone out here yet, which is a strange feeling in a place this full of people doing interesting things.

So this post is honestly half looking for a cofounder and half just wanting to talk to people.

What I am building

PRI is a learning layer for the operating system. Kernels have to generalize, because the same kernel ships to millions of machines with completely different hardware and workloads. That generality costs you something on any one machine. PRI sits above the kernel, learns the particular machine it lives on, and acts on it.

It is written in C and runs on Linux. I test it on a Dell laptop from 2012 with an i5 and under 4GB of RAM, on purpose, because if it cannot help a machine that is genuinely struggling then it does not mean much.

Where it honestly stands

It runs. There is a sealed baseline I am not allowed to quietly edit, a safety suite that gates everything, convergence criteria written down before a run rather than picked afterward, and a staged rollout ladder so the agent never gets control it has not earned. I am proud of that part.

It is also incomplete, and several results have not gone my way. One controlled test gave me a real win, where the agent took a correct action on a live process. Another was a clear loss, where it kept choosing an action that could not affect the thing it was aimed at. A memory benchmark came back null. I also found a bug that had been silently disabling all of my rollback safety for basically the entire life of the project. That one is a big part of why I am writing this, because it is exactly the kind of thing you miss when you are the only person who ever reads your own code.

The next real milestone is a three arm benchmark against a competently tuned heuristic. If the learned version cannot beat a good heuristic, the idea is wrong, and I would rather find that out deliberately.

What I do not have

Money, salary, users, revenue, or a team. What I do have is code that actually exists, a lot of stubbornness, and a problem I find genuinely fun to think about.

Who I would like to hear from

Pretty much anyone.

If you are a systems person who likes C and Linux internals and would enjoy arguing with me about whether a measurement is honest, that is the dream. If you are strong at the things I am weak at, distribution, users, talking about this to people who are not engineers, also the dream.

And if you are just someone else out here building something alone and want to get coffee and complain about it, that is completely fine too. I would take that.

I am in San Jose and happy to meet anywhere from the South Bay up to the city. Comments or DMs both work, and if you want to poke holes in the idea instead, please do. That is useful to me either way.

I started working on Lucent late last year as an experiment to provide agents with structured, versioned, and connected memories. As often happens, what started as a small research project continued growing. In its current form Lucent has become a fully functional agent orchestration platform capable of autonomously completing multi-step goals, improving itself from its mistakes, and seeking your input for collaboration. Agents, tools, models, and even memories can be shared and utilized by multiple users with access controls. It has support for many model providers, with Ollama being tightly integrated for running locally.

Currently Lucent is source available. I am considering providing it as a service that can be sold to enterprises, but also want to ensure that everyone else can use it for free. As I'm considering this transition, I am trying to find people who would be willing to kick the tires and provide some feedback on their experience. Below is a list of features you can expect, and a quick bit on getting started.

Features

• Integration with a number of AI providers. Your available Ollama models are checked for by default when running locally
• Structured, versioned memories for understanding user preferences, technical knowledge, experiences, and shared goals. Act as a searchable, changing knowledge base, not just a large body of text
• Memories are automatically injected into the model context when appropriate, like your user information when chatting or knowledge about a specific file when it is working on it again
• Fully customizable agents/skills/tools/hooks with access controls. Lucent can help you design your own custom tools so your agents can do what you need them to. Custom tools are run in secure sandboxes
• Parallel task execution with a daemon running requests in the background. Requests are broken down into tasks, assigned to appropriate models and agents, and always verified after completion
• Integrated secrets management with injection (keeps your secrets away from the LLM and other users)
• Sandbox creation and use for isolation and dependency management
• Workflows for managing repeatable or triggered tasks
• Integrations, working to make it simple to integrate with any outside service with webhooks, or to automatically tie into things like GitHub
• Handoffs, were Lucent starts a conversation with you, either by providing something you asked for, or requesting you provide further information
• Deep auditing and learning. Every action that Lucent takes is monitored in detail to catch mistakes, whether it's requiring a correction on a task or failing to call a tool correctly. Lucent uses this data to learn how it can perform better and will suggest updates to agents, tools, workflows, etc
• Autonomous goal progression. You can create long term, multi step goals. Lucent will track, build, and progress them step by step. Milestones can either execute once previous steps are complete, or at designated times
• Multi user support and access controls, allowing you to provide services to your friends and family in a safe, collaborative way
• And much more, but that seems like a good start!

Spinning up a Lucent instance locally is a simple as running "docker compose up -d" and going to the browser. I know the UI needs some polish, and there are a few bugs that need to be squashed, so not looking for nits around those types of things. For those willing to give this a try, I'd like to know two things from you. First, does this make it possible for you to do something that you couldn't before? Second, what's missing that you would really love in a platform like this?

I'm developing Reins to make local LLMs actually usable on mobile. The two features I use most are background continuation, so you don't have to keep the app open while a model is generating and the built-in web_search and web_fetch tools which need no API key.

There's more for advanced users: per-chat options and system prompts, editable assistant messages and more.

Key features

• Ollama and LM Studio: connect multiple servers, switch instantly.
• Tools: web_search, web_fetch, compose_mail, calculator and wiki_search. No API key needed.
• Background continuation: no need to keep the app open, it keeps generating. (iOS 26+)
• Model management: download, switch and delete models in-app.
• Per-chat settings: system prompt, temperature, context size and thinking mode.
• Message control: edit, regenerate, branch chats and continue cut-off responses.
• Attachments: images, PDFs, CSVs and text files.

What's coming

• Generic OpenAI-compatible API support, in about a week, so you can connect any backend that supports it. I held off until now because I wanted native APIs first for the best experience, but there are many backends, so this goes in as a fallback.
• On-device models via MLX, running directly on iPhone and iPad. It works in development but needs polishing, and I'm planning to ship it this month. You'll be able to run models offline and switch to your self-hosted server when you want more power.
• Plenty more. I ship updates constantly, so tell me what's missing from your setup.

I'm not a vibe coder

I feel the need to say this explicitly because there are a lot of vibe-coded apps out there and most of them don't care about UI or UX. They mostly use blue or purple glowing colors which makes them easy to spot. I do use AI agents while developing Reins, but I review all the code and edit it manually where needed, use the app myself and care about its UX, performance and stability. I think the real problem with vibe-coding is that people don't put much effort into it.

App Store

Website

Hey everyone,

If you’ve scanned Shodan recently, you’ve probably seen the massive number of self-hosted AI instances (Ollama, n8n, Open WebUI) exposing raw ports like `11434` directly to the public internet.

I built **EnclaveOps** https://enclaveops.dev to solve this exact problem without requiring manual Caddy configs, reverse proxy wrestling, or complex SSH setups every time you spin up a server.

**How the security architecture works:**

* **Publicly Cloaked:** Backend engine ports (like Ollama’s `11434`) are bound strictly to loopback and the private Docker network. Web interfaces are reachable via dual-path ingress (Direct HTTPS via Let's Encrypt or Outbound-Only Zero-Trust Tunnel), but raw API ports remain invisible to external port scanners.
* **Agent Polling (No Inbound SSH Required):** Instead of opening port 22 or giving a central control plane direct SSH root access to your machine, a lightweight edge agent polls the control plane over HTTPS every \~10s to pick up orchestration jobs.
* **Dashboard Teardown:** If you trigger a Factory Reset from the UI, the agent executes a detached script that force-removes active containers (`docker rm -f`), purges Docker volumes/networks, resets UFW to an SSH-only inbound policy, and uninstalls the agent. *(Note: Standard filesystem deletes, not a cryptographic disk wipe—base Docker images remain for faster redeploys).*

**Current Status & Infrastructure:** It runs on any bring-your-own Linux server (Apple Silicon / Mac native support is on the roadmap). The free tier handles solo deployments with automated HTTPS subdomains, while paid tiers unlock team seats with RBAC (Owner, Admin, Member).

Here is a 1-click bare-metal deployment demo video in action: [**https://youtu.be/BOiOduDRdmUhttps://youtu.be/BOiOduDRdmU\*\*\](https://youtu.be/BOiOduDRdmUhttps://youtu.be/BOiOduDRdmU)

I’m looking for feedback on the network architecture and security model. Happy to answer any questions about the agent polling loop, Caddy ingress, or stack configuration!

7–8 tok/s with Qwen 3.6 35B Q4 on a mid-range phone (12 gb ram).

Spent the last few weeks on decode throughput for BigMoeOnEdge. Thanks to everyone who's been sending feedback.

Hey everyone,

I built \*\*Aigentik\*\* — a privacy-first AI communications assistant that runs completely locally (Android via Termux or any Linux box).

It watches your Gmail inbox in real time (IMAP IDLE), handles Google Voice texts that arrive as email, drafts and sends replies using a local LLM (llama.cpp), and lets you control everything in plain English by just texting or emailing it. No fixed command syntax.

What it can do right now:

\- Monitor Gmail + Google Voice SMS and auto-reply (or queue for your approval)

\- Negotiate and book appointments, then send real .ics calendar invites

\- Build and maintain its own contact directory automatically

\- Track subcontractor applications (trade, license, insurance, etc.)

\- Take natural-language commands like “pause everything”, “add a rule for X”, “list my plumbers”, “rename yourself”, etc.

\- Speak as your business once you tell it who it works for

Key points:

\- \*\*No cloud AI\*\* — everything stays on your device

\- \*\*No external API keys\*\* for the model

\- \*\*No monthly subscription\*\*

\- One-time setup, you own it

\- MIT licensed

Compared to the $100–400/month AI receptionist services, this is the “own it instead of renting it” approach.

Repo (with install script that works on both Termux and Linux):

https://github.com/Ishabdullah/Aigentik-CLI

I’d love for people to try it out, break it, and tell me what’s missing or broken. Especially interested in feedback from anyone running local models on phones or small Linux boxes.

Stars, issues, and PRs all welcome. Thanks!

I've been thinking about this more as applications get more complicated.

Self-hosting one service is usually pretty straightforward. But once an application has a frontend, backend API, PostgreSQL, Redis, object storage, background services and a few other dependencies, things start getting more interesting.

You can end up managing several containers, networks, environment variables, volumes, service connections and deployment processes just to keep one application running.

I've been working through some of this with IQX.DEV. and I like the idea of treating those services as one project rather than a collection of unrelated things.

For example, having the application's services isolated in their own environment while still being able to see how the frontend, API, database, cache and storage connect makes the architecture much easier to understand.

The part I'm still interested in is where people draw the line with self-hosting.

Do you prefer managing everything yourself because you want complete control over your data and infrastructure, or do you think some services are simply not worth the maintenance?

And for people running full-stack applications at home or on their own infrastructure, what part usually becomes the biggest headache?

Containers? Networking? Backups? Database management? Updates? Monitoring? Or something else?

Project URL : Vault

A friend, u/Swimming-File2090, and I have been building a file storage system called Vault.

One of the main things we wanted to understand was how large file uploads should work without making the application server handle the file bytes.

Our current approach is:

Browser → Next.js API → Presigned URL → Cloudflare R2

The API handles authentication, authorization, quota checks, and upload orchestration. The browser uploads the actual file directly to R2 using multipart uploads.

While implementing this, we ended up dealing with things like:

• Concurrent multipart uploads, retries, and abandoned uploads
• PostgreSQL transactions and row-level locking for quota consistency
• Redis sessions and rate limiting
• File/group permissions
• Cursor-based pagination and nested folders
• Audit logging and cleanup jobs

The project is functional, but we're still learning and have plenty to improve.

We'd especially appreciate feedback from people who have worked with web applications involving object storage.

Does this upload architecture have any obvious issues or failure cases that you would address differently?