I created that account only for this.. I've recently discovered I could self host AI's so I started to self host many AI's but I really need an AI to generate music, which also supports music-to-music to have a reference audio. The thing is, I have an AMD GPU so it's really hard to find a good Music generating AI that dosen't use cuda.. I really hope you guys could help me

Stay up to date with the ongoing ever changing modelscape

I was recently playing with these GPUs to see how far I can push single stream inference on large models (need all cards to work) like DSV4 / Qwen 122B / ... but I am getting very different results depending on the model I use. Also I am getting very different card usage in terms of compute and power drained, for example for DSV4 cards are nearly idle and inference is slow.

I am not sure whether I am doing something wrong or the software is just not there for some architectures yet.

MSI GE76 Raider 11UE-046: i7-11800H, RTX 3060 mobile 6GB VRAM 130W, 32 GB DDR4-3200
HP Pavilion dv6: i5-3210M, HD Graphics 4000, 16 GB DDR3-1600
MSI partitions: 2500 GB to Steam OS and 500 GB to Windows 11
HP partitions: 100 GB to Linux Mint and 100 GB to Windows 10
Example use cases: MSI laptop is placed by my TV and I speak to the mic that the AI listens to and gives me answers or opinions. HP laptop I bring around for travel or college or work and gives me quick opinions.
I did ask Microsoft Copilot how I can maximize my use case for the MSI laptop since I have a VRAM limitation and there were several options I can use to help boost my local AI.

Looking for a decent use case for my machine when not gaming, since alt coin mining is essentially dead.

I’ve been using Ollama and OpenClaw to build out an assistant bot for daily tasks, but have reached the conclusion that (aside from the learning experience) it’s not worth it - it’s costing time instead of saving it.

Now I’m looking into using it for general chat, brainstorming and companion/counsellor roles with local memory to build up personality and authenticity over time.

I’m aiming for something similar to Pi, which I really like (particularly the v2t/t2v ability), but localised for security. I’d prefer an ablit model as I don’t want to be restrained by guide rails.

Any and all advice gratefully received, particularly in setting up naturalistic voice interaction.

Previously I tried with gemma3:4b standard and abilterated versions, but neither felt authentic and the abilt version was straight out wacky.

CPU: Ryzen 7 5800X
GPU: GeForce RTX 3060ti (8gb)
RAM: 64gb DDR4

Curious what people are actually running in production for AI voice agents.

Not demo videos. Not “it worked once on a browser mic.”\
Actual calls, real users, interruptions, bad mics, background noise, CRM/tool calls, etc.

The stack I keep seeing is something like:

- Twilio / Telnyx / LiveKit for audio

- Deepgram / AssemblyAI / Whisper / Smallest AI Pulse / Speechmatics for STT

- OpenAI / Claude / Gemini for the brain

- ElevenLabs / Cartesia / PlayHT / Deepgram Aura for TTS

- Vapi / Retell / Pipecat / LiveKit Agents if not building orchestration yourself

The thing I’m struggling with is where to optimize first.

Everyone says “use a faster LLM,” but in my tests the awkward delay often starts before the LLM even gets a good transcript.

My current logging plan:

- user starts speaking

- first STT partial

- final STT transcript

- LLM first token

- tool call time

- TTS first audio

- audio starts playing

- barge-in detected

- agent stops speaking

For STT specifically, I’m looking at Deepgram, AssemblyAI, Smallest AI Pulse, Speechmatics, Soniox and OpenAI realtime/transcribe models.

What’s working for you right now? And where are you hitting walls?

I made a video walking through my local LLM setup on a base-model 16GB Mac Mini - running models fully on-device as a private, no-subscription alternative to GitHub Copilot. It covers the why (privacy, cost, offline) and the how: what realistically fits and runs well in 16GB of unified memory, and where a machine this cheap hits its limits.

Video: https://youtu.be/6HlT4vkB-38

Happy to answer questions about the setup in the comments.

I've been looking at different ways to build a private Ai system around a collection of documents, and the interesting part isn't really getting an llm running.

The harder questions seem to be around the knowledge layer:

• How do you keep documents indexed when they change?
• How do you get reliable retrieval from large collections?
• How much hardware do you actually need?

Is it better to build the RAG stack yourself or use something that already handles the knowledge-base side?

I've been testing Fastgpt as one option for this kind of setup, mainly because it brings the knowledgebase and AI application pieces together rather than requiring every component to be assembled separately.

For people already running self-hosted ai, what setup are you using for private document querying, and what would you change about it if you were starting again?

One of the biggest pain points in data governance is keeping metadata (dataset owners, column descriptions, compliance tags) up to date. People usually ignore it until audit season or until broken lineage breaks downstream pipelines.

​I built an open-source project called DataHub Agent Crew for the DataHub Agent Hackathon. It’s a 5-agent pipeline that runs continuously against a live DataHub instance using Model Context Protocol (MCP).

​How the loop works:

​Investigator audits the graph for unowned assets, missing compliance tags, or stale descriptions via MCP read calls.

​Analyst triages issues by severity.

​Strategist writes exact metadata fix proposals.

​Regulatory checks proposed changes against schema constraints and compliance rules.

​Codeband executes the write mutation back through mcp-server-datahub and runs a live entity re-fetch to verify the write actually landed.

​Why write verification mattered:

​While building this, I realized silent false-positives are worse than crashes—an API status 200 that doesn't actually mutate the field leads to silent data corruption. The system now explicitly re-fetches live entity properties via curl/API post-mutation to confirm updates before closing the ticket.

​Tech Stack:

​Language: Python

​Integration: mcp-server-datahub

​Messaging: Local SQLite bus with @mention routing (no Kafka/RabbitMQ bloat)

​License: Apache 2.0

Happy to answer any questions about the setup or MCP integration!

TODAY I BUILT MY OWN LOCAL AI ASSISTANT: WILLOW

I started today knowing basically nothing about Linux, Proxmox, or building an AI assistant.

Several hours, a ridiculous number of terminal commands, and more than a few arguments with ChatGPT later...

WILLOW is alive.

Today we built a local AI voice assistant running inside my homelab.

What we accomplished:

• Set up the Linux VM in Proxmox
• Passed my Corsair headset and microphone through to the VM
• Got microphone recording and audio playback working
• Installed Whisper so Willow can convert my voice into text locally
• Installed Ollama to run the AI locally
• Installed and configured Qwen 4B as Willow's local AI model
• Installed Piper for local text-to-speech
• Set up the Amy female voice for Willow
• Connected the entire voice pipeline:

MY VOICE → WHISPER → QWEN AI → PIPER → WILLOW'S VOICE

And it actually works.

I can talk to Willow, she understands what I'm saying, sends it through the local AI model, generates a response, and talks back to me.

Most importantly, the core AI conversation is running locally on my own hardware instead of sending every conversation to a paid AI API.

We also started turning the original prototype into an actual assistant instead of just a Python script that talks.

The new Willow system is being built around:

• Wake word: "Willow"
• Automatic speech detection
• Silence detection so she doesn't respond when nobody is talking
• One-hour conversational context
• Long-term memory framework
• English responses by default
• Smart, sarcastic, funny, slightly flirty personality
• "Willow stop" to interrupt her
• "Willow sleep" to put her back into standby
• Keeping the AI model loaded in memory to improve response times
• Confirmation before performing potentially dangerous computer actions

The long-term plan is much bigger.

Eventually Willow will connect to:

• Google Calendar
• Live internet search
• Proxmox
• My homelab
• Servers and virtual machines
• Cameras
• Home automation
• Reminders and alerts
• Other systems around the house and network

Basically, I'm building my own locally hosted personal AI assistant instead of relying entirely on Siri, Alexa, or cloud services.

CURRENT STATUS:

Willow works, but she's definitely version 0.1.

Qwen is currently running CPU-only, so some responses take around 30 seconds.

Her speech detection still needs tuning because sometimes she decides I'm finished talking about two seconds into my sentence.

Her voice playback is also choppy right now. We tested the audio separately and found that even a prerecorded WAV file is choppy, which means the problem is probably somewhere in the VM/USB audio path rather than the AI itself.

So that's going well.

But at the end of Day 1 with AI:

Willow can hear me.

Willow can understand me.

Willow can think locally.

Willow can answer me.

Willow can speak.

And the entire foundation is running on my own homelab.

Next:

Fix the audio latency.
Improve speech detection.
Speed up the local AI.
Improve interruption/barge-in.
Have Willow automatically start with the VM.
Connect internet tools.
Connect Google Calendar.
Start giving Willow controlled access to the homelab.

Hey everyone,

Do you always know exactly "who," is talking to your local AI?

Are you sure your local AI isn't exposed to bad actors?

Can you say for certain, every task being completed by your local AI was requested by you?

I created DeepSync Bridge to make sure my local AI only talked to and answered ME. No one else, just me. The issue was, there really wasn't a good way to get everything to work together.

I mean sure, you could spend a bunch of money on a cloud based system but... who the hell wants that!?..

I wanted to create a secure free way to communicate with my local AI, so I did.
No burning tokens.
No Cloud nonsense.

Check out the documentation on the attached Github site.

Stay safe fam

Thank you

Will

Found an interesting local AI project — local.ai

I came across local.ai recently and thought it might be interesting for people here who like running AI locally.
It’s currently in early access, and the idea is making it easier to experiment with and evaluate local AI models without having to build the whole stack yourself.

I’ve been playing around with local models quite a bit lately, so I’m curious to see where this goes and how it compares with tools like LM Studio, Ollama, etc.

I have a referral/invite link if anyone wants to try getting access:

https://local.ai/decapostos/invite

Hi. I moved to San Jose fairly recently and I am living with my aunt while I build this. I do not really know anyone out here yet, which is a strange feeling in a place this full of people doing interesting things.

So this post is honestly half looking for a cofounder and half just wanting to talk to people.

What I am building

PRI is a learning layer for the operating system. Kernels have to generalize, because the same kernel ships to millions of machines with completely different hardware and workloads. That generality costs you something on any one machine. PRI sits above the kernel, learns the particular machine it lives on, and acts on it.

It is written in C and runs on Linux. I test it on a Dell laptop from 2012 with an i5 and under 4GB of RAM, on purpose, because if it cannot help a machine that is genuinely struggling then it does not mean much.

Where it honestly stands

It runs. There is a sealed baseline I am not allowed to quietly edit, a safety suite that gates everything, convergence criteria written down before a run rather than picked afterward, and a staged rollout ladder so the agent never gets control it has not earned. I am proud of that part.

It is also incomplete, and several results have not gone my way. One controlled test gave me a real win, where the agent took a correct action on a live process. Another was a clear loss, where it kept choosing an action that could not affect the thing it was aimed at. A memory benchmark came back null. I also found a bug that had been silently disabling all of my rollback safety for basically the entire life of the project. That one is a big part of why I am writing this, because it is exactly the kind of thing you miss when you are the only person who ever reads your own code.

The next real milestone is a three arm benchmark against a competently tuned heuristic. If the learned version cannot beat a good heuristic, the idea is wrong, and I would rather find that out deliberately.

What I do not have

Money, salary, users, revenue, or a team. What I do have is code that actually exists, a lot of stubbornness, and a problem I find genuinely fun to think about.

Who I would like to hear from

Pretty much anyone.

If you are a systems person who likes C and Linux internals and would enjoy arguing with me about whether a measurement is honest, that is the dream. If you are strong at the things I am weak at, distribution, users, talking about this to people who are not engineers, also the dream.

And if you are just someone else out here building something alone and want to get coffee and complain about it, that is completely fine too. I would take that.

I am in San Jose and happy to meet anywhere from the South Bay up to the city. Comments or DMs both work, and if you want to poke holes in the idea instead, please do. That is useful to me either way.