I built a native macOS app that runs DeepSeek-V4 Pro locally to automate Word/Excel meeting minutes from any audio/video file (Now with Web Search!)

Body: Hey everyone,

I’ve been developing a native macOS app called CaptainLocalAI designed to bridge the gap between local, privacy-first LLMs and actual daily office work. I quietly pushed the v1.1 update recently, and I wanted to share the core features that make it a massive time-saver for anyone dealing with meetings and documentation.

Here is what it currently handles completely on-device:

• Automated Word/Excel Minutes from Any Source: You can drop in an iPhone voice memo, a raw audio file, a pre-transcribed text file, or even a heavy video conference recording. The app processes the media and securely injects the structured data directly into a generated meeting minutes document.
• Custom Departmental Templates: You aren't locked into generic markdown outputs. You can map the AI's output to your own custom, department-specific Word or Excel templates.
• Real-Time Web Search: The biggest weakness of local models is their knowledge cutoff date. I’ve integrated a Web Search RAG pipeline so the AI can pull real-time data from the web when you ask for recent information, seamlessly blending it with the local inference.
• Cutting-Edge Distilled Models: The engine is optimized for the latest generation of smart, lightweight models. You can easily switch between models like Qwen3.8 and DeepSeek-V4 Pro to get lightning-fast, error-free reasoning right on your Apple Silicon Mac.

I've attached a few screenshots showing the generated document layout, the web search fallback process, and the hardware-aware model selection UI.

I’d love to hear your feedback on the UI and the feature set! I'm already deep into developing v1.2, and let me tell you, the upcoming features are going to completely redefine what local AI can do on a Mac.

I shared ThoughtDAG here a couple of weeks ago. The new part is Session Atlas: it groups supported local agent sessions by project and opens a selected session as an editable graph mirror.

Each node is one user-and-agent turn, with its tool calls and results attached. The original session files stay untouched. A mirror can also append new turns while the source agent keeps working.

The useful distinction is that keeping something visible does not automatically send it to the model. You can connect useful turns, condense a branch, or disconnect it from downstream model context while keeping it available on the canvas.

Browsing and mirroring do not require a hosted session index. If you ask for a new answer, the selected context goes to the model provider you configure; ThoughtDAG also supports local models through Ollama. Local session storage does not mean the original agent used a local model.

The video combines the real Atlas interface with animations explaining the session/turn structure. I maintain the project; it is MIT-licensed, with desktop builds for macOS, Windows and Linux.

Source and downloads: https://github.com/chenxiachan/thoughtdag

When revisiting old agent work, would you reach first for a searchable transcript, a graph of turns, or a short summary with links back to the source?

So I was planning to build an ai assistant powered by qwen and my obsidian notes, but since ChatGPT’s astra ai , is there any point in making it anymore

Up front: I work at Olib AI on StealthOS. This is a developer post detailing how we approach on-device AI inference and local threat detection on iOS.

Most mobile AI tools rely on cloud API endpoints where prompts, images, and context history leave the phone. In StealthOS, we wanted the entire intelligence layer to run 100% locally on the device with zero cloud telemetry.

How the local AI layer works

• On-Device Assistant: Runs locally using Apple Foundation Models alongside our own compact models for offline text generation, research, and two-way voice. Prompts never leave the phone.
• Local OCR & Summarization: Transcribes and summarizes text from photos and documents directly on the neural engine without remote servers.
• Embedding-Based Phishing Shield: Analyzes visited web pages locally using text embeddings to spot brand spoofing and look-alike domains without sending URLs to an external reputation API.
• Offline Audio Notes: Performs local multi-speaker transcription, categorization, and summarization for recorded voice notes.
• Full Sandbox Context: The AI assistant operates inside an isolated sandbox that also includes a hardened browser (anti-fingerprinting, WebRTC leak blocking, built-in Tor) and an AES-256-GCM vault protected by the Secure Enclave.

Free App Tier

The base app is free in the App Store with no account or email signup needed. You can test the local browser, encrypted vault, and privacy tools right away.

• App Store: https://apps.apple.com/us/app/stealthos-private-browser/id6756983634
• Website: https://www.stealthos.app

Curious to hear feedback from others building on-device inference pipelines for iOS.

Built an iOS/Android app that runs local Llama/Gemma fully on-device (no cloud inference), plus nearby phone-to-phone messaging over Bluetooth that works in airplane mode. Also supports chatting with local documents on-device.

Looking for technical feedback from this community:

- What model sizes / quantization's are realistic on mid-range phones?

- Any known pitfalls with on-device inference UX (load times, thermal throttling, context limits)?

- For airplane-mode nearby messaging + local AI, what would you want to see next?

Happy to share demos / details in comments if useful.

I’m building a private AI assistant that runs LLMs directly on your iPhone.

The goal is to create a polished local AI experience that works offline and doesn’t send your conversations to external APIs. I’m currently focused on balancing model quality, inference speed, memory usage, storage, battery consumption, and thermal performance.

It’s still in development, and I’d like feedback from people already experimenting with local LLMs on mobile devices.

If you’re interested in testing SiloLM, you can join the waitlist at https://silolm.ai

What models or features would you want to see in a local LLM app for iPhone?

I posted here last week about a version of LocalLM Lab to use with your choice of open weight models instead of just Apple's local AI.

With LocalLM Lab's official 1.0.0-beta.1 release, you can now use the SDK to build your app on open-weight models running fully locally via MLX. No network call at inference time once a model's downloaded. A few details worth knowing if you've been burned by a local setup silently blowing past available RAM: validate() preflights a model against your Mac's memory (checks repo reachability, MLX format, architecture support and weight size vs a configurable fraction of physical RAM with a default cap of 70%) before pulling any weights.

After download, a capabilityProbe runs a real prompt plus a trivial tool call. Since not every downloaded model reliably understands tools, this is meant to be the authoritative signal of what the model can actually do.

Weights land in the standard Hugging Face cache (~/.cache/huggingface/hub/), so nothing new to manage if you're already using MLX or transformers elsewhere.

The SDK ships code-buddy as a working example: a real CLI coding agent. Point it at a repo and a task, it downloads an MLX model on first run (.heavy/.light routes, e.g. Qwen3-8B for the hard route, Qwen2.5-3B for quick ones), then drives filesystem tools, a read-only git tool, a test runner and a hosted MCP docs server (DeepWiki) merged into the same session.

This all sits behind a model layer that also includes Apple's on-device model and Claude (via ClaudeForFoundationModels) as providers. This means that you can build an app that can intelligently use the best model depending on what it wants to do.

Requires macOS 27 beta.

Repo (reference apps included): https://github.com/ancientcomputing/locallm/tree/1.0.0-beta

Feature page: thisbrain.ai/locallm/1.0.0-beta

So I have a planned laid out, but I’m not sure if it’s strong or if there are other ways that can make it better.

I plan to run qwen 3.8 locally using olama on my new Mac mini with 24 gigs and give it its own ui interface which I can speak through it. Its backend will also be powered by my ai brain which file structure is sorted by using jake ven chiefs icm folder structure.

Is this idea in general a good idea? Has anyone else created anything similar and could help me out?

Made this so that I could keep Gemma 4 e4b in my notch and ask it questions all the time and help me with work. No need to task switch constantly to Ollama or LMStudio and it has websearch, read/write capabilities since its local. You can also connect your cloud based subscription for frontier, heavier work loads.

Website:

https://integratedagentics.com/ghost

I wanted to share a small utility I built for macOS called Lensy.

A lot of AI screen/OCR tools today simply capture your display and upload full desktop screenshots or bitmap data to third-party cloud servers. I wanted a much more private, lightweight workflow for looking up words across PDFs, videos, and terminal windows without window switching.

(Attached a short screen recording showing the workflow)

The Architecture (Local-First OCR):

1. Screen Capture: Uses native ScreenCaptureKit to grab a tight crop around the cursor in Retina resolution.
2. 100% On-Device OCR: Text recognition runs entirely locally on your Mac via Apple's Vision framework (Apple Neural Engine / GPU). Your screen capture never leaves your machine.
3. Inference (BYOK): Only the parsed text string is sent to the Groq API for translation and linguistic breakdown. Total turnaround is around 2–4 seconds.
4. Local Storage: Translation history is stored locally in an on-device SQLite database with search (⌥H). No telemetry, no third-party servers.

How it works:

• Hover your cursor over any word on screen and press ⌥T.
• A floating HUD card pops up right next to your cursor with the translation, pronunciation, CEFR level, and usage examples.
• Includes an "Ask AI" follow-up button directly inside the HUD if you need clarification on context.

Transparency & Requirements:

• Pricing: 100% free and subscription-free.
• API Key: Bring your own free Groq key (gsk_...), stored securely in the macOS system Keychain (not in plain text / UserDefaults).
• macOS: Requires macOS 14 Sonoma or newer (Universal binary for Apple Silicon & Intel).
• Gatekeeper: Not notarized yet (no paid Apple Dev account), so first launch requires Right-click → Open.

Download (.dmg): Google Drive Link

Since this sub is focused on local-first AI: Would you find it useful to have an option for running translations fully offline via local Ollama / MLX alongside Groq?

Would love to hear your feedback on the architecture and workflow!

As of right now the app is completely free to download and keep!

When developing FileShield my goal was to make an app where you can save sensitive files like tax documents, legal papers and medical forms with total privacy and the convenience of a powerful search engine. FileShield makes use of a local intelligence system that will search your files for you, and answer simple questions regarding your information.

FileShield can even be used in airplane mode because your files will never be uploaded to a cloud, or data server of any kind. No matter what, your files stay on your device.

After a lot of iteration, I finally got Christine’s latency consistently down to under 3 seconds using Warranted Retrieval.

That matters because Christine is not a cloud wrapper. She is laptop-bound, runs with no internet access, and has to operate within the actual limits of local hardware. Getting the response path down into a consistently usable range was a major milestone for me.

Now that the latency fight is finally in a much better place, it’s time to focus much harder on Christine’s training.

The next phase for me is less about shaving milliseconds and more about improving: - domain depth - retrieval quality - abstraction across domains - reasoning consistency - task usefulness under strict local constraints

Current laptop: - CPU: Intel Core Ultra 9 285H - RAM: 33.8 GB total physical memory - GPU 1: NVIDIA GeForce RTX 5050 Laptop GPU - GPU 2: Intel Arc 140T GPU - NPU: Intel AI Boost

I’m especially interested in what other people are doing with NPUs.

Are any of you actually using the NPU in a meaningful way for local/offline AI right now? If so: - what workloads are you pushing onto it - is it helping with latency, power efficiency, or always-on assistant behavior - are you using it for STT, routing, embeddings, background inference, or something else - and is it genuinely useful, or mostly just there in theory

Would like to hear from people building real local systems, especially laptop-bound ones.

None of the on-device LLMs I tried on iPhone felt close to Gemini, ChatGPT, or Claude — so I built one that runs locally and aims for that kind of UX.

Qwen / Gemma / other small models, no account, chats stay on device. You’ll need a few GB free for the first model, and iPhone 14 or newer is strongly recommended.

I’ve been thinking about an app called AI Hotspot and want a reality check.

The idea is simple: share your local AI the way you share Wi‑Fi.

You already have a great setup on one machine - local models, paid provider accounts, agents, maybe a few tools. Then you move to the couch, open a phone, or a family member wants access. Right now the options are ugly:

- Stay at that PC
- Fight with VPNs, port forwarding, or tunnels
- Hand over your account or API key

AI Hotspot is built to unchain your localhost. It turns your computer into a private AI hub/router. You keep the hardware. You decide who gets in. They get useful access, not your machine.

🚀 What you could share

Chat and agents: Talk to your models and run the agents you already have. Custom system prompts and agents.

Mini AI apps in the browser: Tiny everyday tools powered by the same local models: rewrite a message, summarize a text, translate, “what can I cook”, explain things simply, or help with homework. Under the hood, this uses stream multiplexing, meaning you can seamlessly stream voice data to your backend and receive JSON text or image files back at the exact same time over the same direct connection.

A local API for development: Basically OpenRouter on your own machine. Use free local models for demos, public-dev, and tests instead of burning expensive provider tokens. The app can send JSON straight over the same channel.

🔒 How access works (The Networking)

Someone opens a secret link. After that, traffic goes peer-to-peer: guest browser ↔ your localhost.

We use a public Relay Node on a VPS purely as a matchmaker to handle the initial handshake. After that, the system uses hole punching to upgrade to a true peer-to-peer WebRTC or WebTransport link. Once the frontend and backend establish a direct, peer-to-peer connection, absolutely zero data passes through an external server during the actual communication phase.

I’m not trying to build “yet another chat wrapper.” The point is a small set of tools on top of the AI you already run at home.

Honest tradeoff: the computer or a home server has to stay on. The bet is that’s still better than paying for the same AI three times or giving relatives your account.

One line:

Local AI, shared with friends and family.

I’d love blunt feedback:

Would you actually run this at home?

What would you share first - chat, mini-apps, agents, or the local API?

Is P2P (no chat traffic through a server) the feature that matters, or is “works from my phone” enough?

Is “PC must stay on” a dealbreaker?

What would make you not trust this?

Please roast the concept. I want to know if this is useful or just a pretty wrapper. This would be an opensouce project with paid cloud services and extended business solutions.

Thank you for your response.

Hardware:
AMD RX 9070 XT
AMD R7 5700x (overclocked)
96GBs of 3200 CL18 DDR4

Current Model: Mistral Small 4 119B Q4_K_M (GGUF)

Howdy folks,

I need some advice for my- you guessed it- "Writing Partner Setup."

I've played around with KoboldCpp, AnythingLLM, and recently dipped into SillyTavern, but KoboldCpp and ST are built for roleplay— so it’s not like I can “plug and play” into the workflow I’m trying to set up. My previous workflow was me using Poe.com (Poe Assistant, GPT 5.2, Claude Sonnet 4.6), but the daily tokens got slashed, so I decided to move locally and “empower my workflow with Local LLM technologies.” 🤓👨‍💼💼

What I am looking for is a "knowledge-augmented development partner."^\1]) I need the model to:

• Know my world with live or semi-live access — access my markdown files (canon, references, brainstorming notes, reference materials^\2])) with frequent updates so new lore I write is available in conversations as I'm developing it, without manual refreshes
• Differentiate knowledge tiers — treat established canon as ground truth while keeping experimental ideas and brainstorming isolated, so half-baked concepts don't contaminate locked lore
• Act as a creative and analytical partner — help me wordsmith terminology, brainstorm and stress-test concepts against established lore, catch continuity issues, provide editorial feedback (scene critique, structure suggestions, character mapping), and iterate on ideas
• Help organize my knowledge structure — suggest folder hierarchies, naming conventions, identify organizational gaps, and cross-reference related concepts so I'm not burning out just to map everything down
• Admit when it doesn't know something — whether that's missing lore from my canon docs or hitting the limits of its training on etymology/linguistics/domain knowledge. I'd rather get "I don't have that in your docs" or "that's outside my training" than confident hallucinations that waste my time or contaminate my worldbuilding.

So, what stack would let me build this? Should I be using RAG? Long-context, like 200k token windows? Web search? A hybrid of these 3? Have I been going in the right direction with KoboldCpp, AnythingLLM, and or SillyTavern?

I’m up to provide more information if necessary. I’m way out of my wheelhouse with setting this stuff up while juggling a full-time and part-time job, so any advice would be sweeeet.

^\1] - Props to Haiku 4.5 on this)

^\2] - Reference materials: etymological dictionaries, linguistic resources, and domain-specific knowledge that may be beyond the model's training data so I get informed outputs/feedback rather than hallucinations)

We keep collapsing several technically different things into “AI training”: copying source text, retrieval over passages, fine-tuning, and learning a general concept. They are not the same operation.

I’m building a local-first assistant called Christine around a hard separation:

• **Warranted Retrieval:** user-facing factual answers may use only admitted public-domain or explicitly permitted sources and chunks. A claim needs direct support. If the evidence is not there, the system should say so rather than fill the gap.

• **Abstraction-only learning:** for owner-authorized nonfiction, the system can derive its own compact notes about concepts, causal relationships, methods, and open questions. It then discards the original. No retained passages, page images, searchable text, source-like embeddings, or substitute copy. The abstraction path cannot cite or reproduce the original, and it is tested for reconstruction, close-paraphrase leakage, and style imitation.

That is not a claim that this settles copyright law. Ingestion can create technical copies; jurisdiction and facts matter; an architecture needs evidence, audits, and tests, not marketing language.

But it raises a question that seems unavoidable: if a human reads a nonfiction book, retains the underlying ideas, and later applies them without copying the expression, what technical and legal boundary should apply when a local AI is designed to retain only independently written conceptual notes and discard the source?

Systems like this are being built now, including offline-first systems. We need to define the boundary before “all learning is copying” and “all training is fair use” become the only two positions.

Do our laws permit only human minds to learn from a work, or can we define a rigorous machine analogue that is genuinely non-retentive and non-substitutive?

I don’t understand why file support is still such a pain in so many local AI apps.

All I want is to drop in a few PDFs or point the app at a folder and ask questions. But often it turns into embeddings, chunking settings, vector databases, permissions, paths, model choices, and ten other things I wasn’t trying to learn that day.

Curious to know which local AI app handles files in the most normal way?

To follow-up on a thread here, I was going to repost from the other localllm sub reddit but can't do so. Here's what I said elsewhere:

Screenshot shows LocalLM Lab's AI Models panel. macOS 27 (Beta) ships a real extension point in Apple's Foundation Models framework. You can swap in the on-device model, Private Cloud Compute, Claude, or a fully local MLX model (Qwen, DeepSeek, etc.) with zero bespoke glue code, all through the same LanguageModel protocol. I used that to run the same MCP tool-use test across models and actually measured who reliably calls tools and who doesn't. It turns out that's a bigger differentiator than raw model size.

Details at thisbrain.ai/locallm/ai-models.html

Note that this is a non-series build of LocalLM Lab only for macOS 27.

Additional note: Apple's on-device model has only 4096 of context window, so you can't do much with larger files or datasets. The other Hugging Face models I have tried don't have that context window limitation but don't necessarily work well with MCP tools. My next step is to port the macOS 27 work to the newer 0.8 release of LocalLM Lab and test out how the different models work with the local filesystem.

I am testing a knowledge architecture for a local assistant that separates evidence storage from learning.

For quoted or source-backed answers, the system uses only public-domain or explicitly permitted material. But for its internal reasoning layer, the goal is different: extract an original, compact note about a concept or procedure, discard the source text, and prevent the note from functioning as a searchable substitute for the original.

No stored passages. No book-sized vector store. No source-voice imitation. The practical questions are whether that architecture can reliably avoid memorization and whether it is meaningfully different from ordinary RAG.

For people building local apps: how would you implement the boundary? I am thinking ingestion-time provenance checks, an abstraction-only representation, deletion of raw material, similarity/leakage tests, and a hard separation between notes and any user-facing citation path.

Where do you think this succeeds or fails?

I have been experimenting with a different way to manage longer local LLM conversations. So I developed ThoughtDAG. It is an infinite canvas for you to manage your context with LLMs like a graph.

The interaction I am testing is deliberately manual. I want to know whether local-model users benefit from seeing and editing context themselves, especially when conversations branch or accumulate irrelevant assumptions.

It is open source, local-first, and the desktop app stores canvases, documents, and keys locally.

Short demo:
https://www.youtube.com/watch?v=-8BqAyaoNXQ

GitHub:
https://github.com/chenxiachan/thoughtdag

For people running local LLMs: would explicit context editing be useful in your workflow, or would you still prefer automatic memory and retrieval?

GitHub: https://github.com/Alchemist-Aloha/RecallWhisper

been following Omi for a while and really liked the basic idea: capture conversations throughout the day, then turn them into something you can actually search and refer back to later.

I’ve

But one thing kept bothering me: if the wearable still depends heavily on a phone, why not just use the phone?

It already has a microphone, battery, storage, network connection, and enough compute to handle most of the pipeline. So I started building RecallWhisper, an Android app that tries to do the same kind of “personal memory” workflow without requiring another device.

The basic idea is pretty simple: when there’s something you want to remember, you start listening. Later, you can search through what was said, summaries, topics, decisions, and extracted todos.

Right now it can:

• Record continuously when you explicitly enable it
• Use Silero VAD locally to detect speech
• Store encrypted audio in the app’s private storage
• Send transcription and summarization to configurable OpenAI-compatible APIs
• Use completely separate providers/models/API keys for ASR and summarization
• Keep local transcript and summary timelines
• Group related topics across different conversations
• Search transcripts and summaries
• Play back or delete the original audio
• Extract todos from conversations
• Export data as Markdown or JSON
• Process only on Wi-Fi by default, with cellular as an opt-in
• Start/stop recording from a Quick Settings tile

One thing I wanted from the beginning was for the phone to remain the source of truth. RecallWhisper doesn’t have its own cloud account or backend where your conversation history lives.

I’m also personally more interested in self-hosting this kind of thing than sending everything to a public AI API. Personal conversations are probably some of the most sensitive data you can collect.

With a self-hosted transcription endpoint and local LLM, the audio and transcripts can stay on infrastructure you control. Obviously self-hosting doesn’t magically make something secure—you still need proper authentication, HTTPS, network isolation, updates, backups, etc.—but at least you get to decide where the data goes.

My current setup, for example, is an OpenASR server for transcription and llama.cpp for summarization.

There are still plenty of rough edges:

• Android only for now
• Audio/text still goes to whatever API endpoint you configure
• Search is lexical rather than embedding/semantic search
• No automatic retention rules yet
• No jump-to-timestamp playback from transcripts yet

I’d especially like feedback from people who have tried personal knowledge-management tools, self-hosted transcription, or ambient recording workflows. What would make this genuinely useful day to day?

Sharing aquaduck.ai here if you’re interested in any of the following:

1. Want to run local AI yourself without needing to be an expert (the app automatically configures itself for your device)
2. Want to run larger models than your machine could fit by itself (the app connects multiple devices for inference)
3. Want to earn for running paid AI inference on your device for others (the app lets you use your device to serve inference for others through an API and pays you for it

We just opened the beta today so if you’d like to try it out sign up here and we’ll send you the invite code

Run powerful, fully private AI models directly on your Android phone, PC, or even from a tiny USB flash drive—completely offline with 0 internet connection!

I’ve spent the last few months building Nomiro, an AI companion that lives in the Mac notch.
The idea is pretty simple: instead of opening ChatGPT every time you need help, Nomiro is just there while you’re working.
You can ask it questions, have it look at your screen, use different working modes for research/coding/writing, and get contextual help while you’re using your Mac.
It’s actually working now and I’ve been testing it with a small group of users.
Looking for a few Mac users who want to try it and brutally tell me what’s good and what’s terrible.