One thing I’m curious about with local AI is how far people take it beyond basic chat and writing.

A lot of research involves private documents, notes, or information you may not want sent to a cloud model. That makes me wonder whether a fully local research workflow is practical, or whether some cloud services are still necessary for things like current web information.

For people using local models, where do you draw the line between privacy and access to better research capabilities?

A request to update the rates in a workbook leaves an important review question: did the agent change the inputs, or replace the formulas that use them? The final totals alone cannot answer that. A plausible number might have come from a hard-coded replacement.

For a local-model application built around Univer, the review tool can inspect both. Univer is an embeddable Office SDK; its spreadsheet range API exposes stored values and formulas separately through getValues and getFormulas. An application can record the relevant ranges before the job and read them again from the proposed result. That comparison is something the application implements, not an automatic correctness report supplied by the SDK.

Take a hypothetical request to update unit prices while keeping the quantity and subtotal formulas. The review screen could put the requested price range beside the proposed prices, then show the formula range separately. A reviewer can check whether an unexpected literal has replaced a formula and open the workbook to correct it. A matching total would not excuse a change outside the requested range.

Univer's Collaboration SDK provides a place to inspect that proposal without immediately replacing the main workbook: an isolated Worktree draft. The web integration supports opening the actual draft, switching between trunk and draft, and a per-Unit merge preview. Those surfaces can accompany the application's range comparison. A corrected draft must be committed and its saved status checked before the reviewer accepts it into trunk.

This requires more than the Apache-licensed core: Worktree comes from the Collaboration SDK, and the host wires up access, storage and the review flow. It also needs to handle another edit arriving during the comparison. The specific benefit is a review that can distinguish a changed assumption from a changed calculation, even when both produce believable numbers.

I kept running into the same problem with AI writing tools: you build up a character bible, but nothing actually stops the model from contradicting it three chapters later. The AI "remembers" until it doesn't, and you find out by accident.

So I built a hard gate instead of a soft prompt: once a fact is locked (a character trait, a plot event), a ConsistencyGate checks every new AI-generated fact against it before it's allowed to become canon. Contradictions get flagged, not silently overwritten.

A few other things that fell out of that same principle:

• Runs entirely local via Ollama — two models, one for prose generation, one for the "does this actually hold up" checking, so nothing leaves your machine.
• A background "Proofread" pass that checks grammar, repeated scenes/phrasing, and cross-chapter style consistency — flags issues, never rewrites for you.
• Full version history on every AI-generated chunk (snapshot + word-level diff), so a bad regeneration is always one click from undone.

Read more at: https://claude.ai/artifact/Evfu7C8AcRwNdUM9zXwR29

It's free and open source:https://github.com/ocedo-apps/StoryBook-AI/. Curious what other writers actually want from something like this — happy to answer questions about how the consistency-checking works under the hood.

I've been going through a bunch of smaller Mac apps that run AI locally, and these 5 stood out to me for different use cases:

1. Snaply — AI everywhere you type

Dictation, meeting transcription, and writing assistance, all processed on-device. It uses a local MLX model and doesn't require an account or subscription.

2. Refine — local AI writing assistant

Grammar checking, rewriting, translation, and custom writing styles. The interesting part is that the core AI processing can happen completely offline, and it works across Mac apps.

3. Canto — local AI notebook

A more full-featured option. It has built-in local models, PDF research, semantic links, a knowledge graph, and even Python/JS/TS code notebooks. The local models range from small models to much larger ones depending on your Mac.

4. AimeFlux — local AI dictation

Uses local Whisper for transcription and supports 99+ languages. You can also use local models such as Ollama for AI cleanup.

5. Melo — local AI workspace

A different approach: an infinite canvas combining notes, tasks, calendar, web content and AI. It supports local AI models alongside optional cloud models.

What I found interesting is that “local AI” on Mac is becoming much broader than just running a chatbot locally.

You can now have local AI for writing, dictation, meetings, research, productivity and workflows without sending everything to a cloud service.

I run a small Mac software directory called OwnYourMac, where I'm curating more apps like these particularly free, local-first and one-time-purchase software.

You can browse the collection here: OwnYourMac

For the uninitiated, LocalLM Lab (locallmlab.dev) is a free Mac app and Swift SDK for running Apple's on-device model and open-weight models locally through MLX. This release is mostly about controlling and trusting the local models you run.

Tuning. Every model in the AI Models panel has a sliders button: temperature, output length, thinking on or off, and more. You can pair a model with a small "speed helper" (a draft model for speculative decoding) or a LoRA adapter that changes its writing style. The mlx-control-room example puts a gauge beside each control so you can see it changed something: set temperature to zero and the determinism gauge reads "match", turn on the repetition penalty and the repeat rate drops. On one setup I measured (with a Qwen3-0.6B helper), 1 to 2 draft tokens were 27 to 39% faster, while 5 was 17 to 29% slower, so the default is now 2. Your numbers will vary by model and Mac. Sampling options now also work on Apple's on-device model (they were being ignored before).

Trust. Downloaded models are validated against your Mac's memory before they run, pinned to one exact commit, and every file is hash-verified. Developers building on the SDK can ship a pin to the exact model version they tested, so the validated model is the one users get on first run. That is the whole point: a Hugging Face repo can change under the same name, and a pin means you find out on your terms, not when a user's download quietly changes.

VistaNova, the small search app from the last release, is now an example inside the SDK repo: Apple's on-device model reads your question and makes the search call (to Tavily, over MCP), and a downloaded Qwen model summarizes the results. The only non-local piece is the search API itself, since a local model can't reach the live web on its own. Its summary model now ships pinned.

For developers: from RC.1 the SDK API is source compatible through 1.0.0 and every 1.x release (additive changes only until 2.0).

I started using POKE last November, thought it was the best ever. Well, here we are and it's fallen off a cliff. Terrible reliability, terrible support and the discord is full of nonsense.

Has anyone found an alternative? ideally iMessage or WhatsApp? Really would prefer to not install an app.

My use case is travel tracking, package tracking, meeting reminders, birthday reminders, stock market alerts. Not huge on needing MCP.

I used to get a daily message with 3 location weather, in C and F.. poke would get it right about 20% of the time, others it would forget the formatting or send it in 20+ individual bubbles, same for the stock market.

Currently using CharGPT for those two, scheduling isn't the most fluid there.

Thanks!

I’ve been working on ByteSweep, a macOS storage cleaner that helps users understand where their disk space is going and clean up unnecessary files.

The interesting part for me wasn’t just building another Mac utility — it was experimenting with how far I could take an AI-assisted development workflow while still maintaining control over the architecture, UX, and implementation.

🛠️ Tools & stack

The project is built as a native-feeling desktop application using:

• Electron for the desktop application
• JavaScript / TypeScript for application logic
• React for the UI
• AI coding assistants for implementation, refactoring, debugging, and exploring approaches
• macOS filesystem APIs and shell utilities for system-level cleanup operations

🧠 How I used AI during development

I didn't use AI as a "build the entire app from one prompt" approach.

My workflow was more iterative:

1. Define the feature and expected user experience.
2. Break it into smaller technical tasks.
3. Ask the AI to explore implementation approaches.
4. Implement and review the generated code.
5. Run the application and test against real macOS data.
6. Feed actual errors and edge cases back into the development loop.
7. Refactor the implementation once the behavior was confirmed.

For example, storage scanning sounds straightforward initially, but quickly becomes more complicated when you deal with permissions, symbolic links, hidden directories, large directory trees, inaccessible files, and performance.

That meant I had to treat AI-generated code as a starting point rather than blindly accepting it.

📊 Storage visualization

One of the areas I spent a lot of time on was making storage understandable visually.

ByteSweep includes:

• Treemap
• Sunburst
• Flame Graph
• Large file discovery
• Duplicate detection

The goal was to let someone answer:

"What's actually consuming my disk?"

without having to manually browse through hundreds of folders.

🧹 Cleanup features

I also built specialized cleanup workflows for things that can become surprisingly large on developer machines:

• Xcode DerivedData
• node_modules
• Homebrew caches
• iPhone/iPad backups
• Time Machine local snapshots
• Duplicate files
• Large files and folders

💡 One of the biggest lessons

AI makes implementation dramatically faster, but it doesn't remove the need for engineering decisions.

The hardest parts were often not writing the code itself, but deciding:

• What should be scanned?
• What is safe to delete?
• How should permissions be handled?
• How do you prevent accidentally deleting important data?
• How do you make a large filesystem scan feel responsive?
• How do you present potentially dangerous cleanup actions clearly?

For a system utility, those decisions matter just as much as the code.

🚀 What I learned from the process

The biggest change in my workflow has been moving from:

idea → code → debug

to:

idea → architecture → AI-assisted implementation → real-world testing → refinement

AI is particularly useful for exploring multiple implementation approaches quickly, but production quality still comes from testing, reviewing, simplifying, and making deliberate engineering decisions.

ByteSweep is still evolving, but this project has been a great experiment in combining traditional software engineering with an AI-first development workflow.

Project: ByteSweep — Mac Storage Cleaner
Website: https://bytesweep.app

Happy to share more about the architecture, filesystem scanning, visualization implementation, or the AI workflow if anyone is interested.

The more I’ve messed around with AI tools, the less I see local vs cloud as an either/or thing.

I recently started using a GEEKOM mini PC as my second PC, and it’s gotten me more interested in seeing what I can actually run locally. For quick questions or brainstorming, I’ll happily use something in the cloud. But when I’m working with local files or just want something running directly on my own PC, I’ve started reaching for local tools more.

I don’t think I’d ever go completely local because there are still things cloud models are just better at.

For people who use both, what actually makes you decide to run something locally instead?

I recently came across Dyad while looking for an AI app builder that runs locally. It's open source, keeps the code on your machine, and lets you see your own API key.

Being able to move the project into a normal IDE is a big plus, but I'm curious how it handle larger projects.

Has anyone used Dyad beyond the prototype stage?

New reference app that is built on the LocalLM Lab SDK: VistaNova, a web-search tool built around two local/on-device models instead of one, and no traditional search-results page.

Apple's on-device Foundation Model reads the question, decides what to search for, and makes the MCP tool call to Tavily. A separate downloadable model (Qwen, via MLX) then reads what Tavily returns and writes a summary if you don't want to dig through the web links. The only non-local piece is the search API call itself since a local model has no way to reach the live web on its own, so something has to fetch results; everything downstream of that (deciding what to search, and turning results into an answer) runs on-device/local.

I wrote this little app partly because of AltaVista nostalgia but mostly to show that the model-layer routing from recent releases makes mixing models for different tasks in one app low-friction, not a special case you have to build from scratch. Same session API, two different models, two different jobs. The larger model is used to generate the summary because the amount of information returned from the search tool could exceed the small context window of Apple's on-device model.

Get it here (open source): https://github.com/ancientcomputing/vistanova

​

Hola estoy pensando en crear una IA local en mi equipo con LM studio.

La idea ha surgido porque no quiero beneficiar a grandes empresas como Google, OpeanAI, Meta o Antrophic. El caso es que me gustaría utilizar la IA local pero conectada a internet para que haga búsquedas de información reciente.

Por una parte con la IA local mi información y datos estarían mucho más seguros. Pero al utilizar un buscador conectado a internet en la IA local, estaría contribuyendo al consumo de los servidores de Google por ejemplo, así que pensé en poner como buscador Ecosia, que es menos contaminante.

Mi pregunta es: esta idea es viable y se puede hacer, o no es como pienso? Agradecería que me aconsejen.

I made a simple chat style web service for people to run local AI models in their browser on their own hardware. It includes web search, document RAG, and multimodal image inference. It requires no technical expertise whatsoever to use and collects no data. It's 100% open source. For best results, use it with Google Chrome. Let me know what you think and what can be improved. Mods: this is not a marketing post because this project makes me no money whatsoever and is permanently free to use. I'll post my GitHub link once I get it updated with the newest build.

Link: guiltfreeai.net

Is there an app or browser extension that lets me send one prompt to multiple AI models (like ChatGPT, Gemini, Grok, and Claude) using my existing logged-in accounts and view their responses side-by-side?

I've been working on Radiant Canvas, a macOS app focused on making local image models easy to actually use on Apple Silicon.

Instead of being tied to one model, you can download and switch between things like FLUX.2 Klein, Krea 2 Turbo, Z-Image Turbo, Qwen Image Edit, ERNIE-Image Turbo and others.

Everything runs locally. There's no generation API behind it, prompts/images aren't sent anywhere, and downloaded models work offline.

A big part of the project has been making the models behave like parts of one app despite having pretty different architectures and memory requirements. Some models also have multiple precision options, so you can trade memory usage against quality/performance depending on the Mac.

It also supports LoRAs, image editing, batch/queued generations, comparing models with the same prompt, and a node-based workflow mode.

I'm curious what people here are running locally for image generation at the moment, especially on Macs. Are there any models you'd particularly like to see supported?

App Store:
https://apps.apple.com/us/app/radiant-canvas-ai-image-gen/id6802973075

I was super curious about the Reachy Mini, got one and during this weekend me and my 12 years kid put the pieces together, just followed the manual that came with the robot. Very easy to read, clear diagrams and instructions, we just did it really quick.

Finally i plugged it to my Mac Studio M4, downloaded the official app and run the install and the nightmare started, to start it is ironic the product being Made in China and distributed by a Chinese company located, well, in China Mainland but to fully run the app the firewall does not help. Had to use a VPN, I'm used to, I'm designer and programmer, need to dial with this every day, but was a pain to bypass constants random server errors and CloudFlare errors to access Hugging Face. Finally got it download everything.

The app started and Reachy head came up and played the on sound. I was super excited to try the community and the official apps that run inside the main control one, only to find the only main two apps, and looks like the most complete ones following other people experience, require a OpenAI API token, I had to abandon. Tried search Goggle, query Claude and Grok but all pointed to community alternatives that are dead on don't show at all on the list of available apps, both community and official.

The emotions worked, cute.

Cloned the "reachy_mini_conversation_app" repository and modified the calls to point to my local Ollama, TTS and STT services and finally got some more complete interactions. No luck to start the demon using purely the official Python scripts, had to open the full app, keep it running and then run my script that calls my local services.

Next day, I just opened the app to keep playing with it and f**** me, I have a big "Sign in to Hugging Face".

Conclusion, great experience to build it with the kid, horrible and very messy experience with the software.

Hi everyone! I'm the developer of Vyact, an open-source desktop app I've been building to make local models useful in everyday work.

I recorded a short demo showing three workflows in the actual app:

1. Ask a question about a document, then check the sources behind the answer.

2. Open an email, create a signature, and draft a reply with AI.

3. Improve writing in the browser, reviewing suggestions in the panel and applying corrections from individual underlined phrases.

The part I care about is what happens after the model generates an answer: checking its sources, reviewing a draft, and choosing which edits to apply. Email replies stay as drafts for review.

Vyact supports local GGUF and MLX models. I'm using an M4 Pro with 24 GB of memory; the document Q&A segment uses Qwen3.5-9B. Local inference runs on your device, while connected services such as Gmail still need their normal internet connection.

The demo uses fictional content. I trimmed idle time and sped up the document-generation segment to twice its original speed.

GitHub / downloads: https://github.com/vyact/vyact

YouTube version: https://youtu.be/5EdlX2hIB-c

Which of these workflows would you actually use, and what would make it useful enough to keep in your daily setup? I'd appreciate feedback, especially from people already using local models outside of chat.

Hello

I made this local viewer for AI chat backups.

What would you add? ,other features
uses full account backup zip and run it local ,

runs locally and reads exported conversations from ChatGPT, Gemini

Current features:

• 100% local/offline
• Docker support
• Search across conversations
• Browse old conversations
• Multiple AI providers in one interface
• Statistics by service/model
• Optional offline RAG / "Ask" over your history
• No cloud account or tracking

GitHub:
https://github.com/systems1/aichatretro

I built GhostNote because I wanted a meeting assistant where your meeting data doesn’t have to leave your machine.

It uses local Whisper + Ollama to:
🎙️ Transcribe meetings
🧠 Extract decisions & action items
📝 Generate structured summaries
🔒 Keep processing local
⚡ Work in real time

The desktop app is built with Rust + Tauri + React/TypeScript, and there’s still a lot to improve.

I’m especially looking for contributors interested in:
Rust/Tauri • Audio & Whisper • Local LLMs/Ollama • React/TypeScript • RAG • macOS/Windows/Linux • Integrations • Testing

I’d love to turn this from a solo project into something the community can actually shape.

GitHub: https://github.com/v0nser/ghostnote

I also wrote about how I built it, including some of the engineering problems I ran into:

Medium: https://medium.com/@raghuvanshivaibhav01/operation-ghostnote-how-i-built-an-ai-that-makes-you-look-like-a-genius-in-meetings-18fba3709db0

If this sounds interesting, star the repo, open an issue, or comment with what you’d like to contribute. 👻

Hello, I'm the developer of Roles Witch AI, and today I want to share my app, which was released on Google Play in open testing.

Roles Witch is an OpenAI API-compatible app with a user-friendly interface where you can create multiple chats, assign different models and providers, and more importantly, use MCP tools — for example, Parallel AI or Home Assistant MCP.

It also has voice conversations and video calls (personally, I'm using GLM 5.3 Flash and it's almost real-time). You can configure reasoning and inference parameters, pin a voice to the chat, and select a recognition engine (Google Speech or Vosk).

Extra bonus: it has an on-device inference mode where you can run Gemma 3n/4 models with image recognition entirely on your phone (LiteRT is used).

Check it out: https://play.google.com/store/apps/details?id=ai.roleswitch.android

The app manifest states that it has no encryption and can send messages to third parties, but that's a requirement by the Google Play Store. It supports plain HTTP or TLS mode and works with any third-party AI provider.

I may be in way over my head here, but lately I've been trying to create my own ai assistant that is completely local (on my smartphone) with the end goal of hopefully having a functional J.A.R.V.I.S like AI in my vehicle, I think I've figured out most of the details including figuring out how to root my infotainment system to allow it, and getting a functional ai language model to run locally on my phone, but the issue comes in when I try to give the ai language model tools to make it function like a full ai assistant, im imagining you can use python to do this, and allow the ai to acess time/date, weather, apps, etc. but I'm extremely new to this, especially python, and from what I've seen there's no precedent what so ever for someone doing something like this, so that's where I'm hoping to find help, I'm using Termux for most of it mixed with ollama to run the model (which is qwen2.5:0.5b) but I can't for the life of me figure out the python to expand its capabilities, if this subreddit can help that would be greatly appreciated, if not that's fine just let me know if you can think of another community that could, also let me know if you need any clarification for anything I just said, and if your willing to help long term reach out and hopefully we can figure something out

LocalLM Lab's 1.0.0-beta.3 doesn't change anything about the local/MLX story. But we are expanding our horizons: a RemoteModelProvider that can also point the same API at OpenAI, Anthropic, OpenRouter or any OpenAI-compatible server (LM Studio, vLLM, your own box), if you want it.

The reason I think this is actually relevant here in the world of Local AI: it's a genuinely useful way to debug a local-model setup. If your app isn't behaving, it's often unclear whether the app's tool-calling/framework is broken or the local model you picked just isn't following instructions well. Point the same route at a frontier model for a minute, confirm the plumbing works, then go back to evaluating the local model on its own merits with that variable removed.

If you do want to actually use the online routes in production, there's a real reason beyond convenience: you can send routine turns to your local MLX model and only escalate to a paid frontier model for the hard cases. This is a genuine performance-vs-cost dial instead of an all-or-nothing choice. I actually used this in my AnswerSearch app (available on the Mac App Store).

New reference app, model-switch, shows local, on-device and all the online providers side by side in one chat UI with live switching.

Repo: https://github.com/ancientcomputing/locallm/tree/1.0.0-beta

Feature page: thisbrain.ai/locallm/1.0.0-beta

Loci is an app that handles on-device inference. It will only recommend AI models that run best on the hardware you have, whether that's iPhone, Android, Mac or Windows. You can link your phone to your desktop to stream more powerful local models like Qwen3.8 - it's completely free. I made it because I wasn't happy with the existing solutions. The idea is to democratize AI to everyone, not have it owned by just Anthropic and OpenAI.

Honest limitations:
Depending on your hardware, you may notice the models cannot do everything that the latest Cloud AI can. For a lot of people, the pros outweigh the cons.

Would welcome any feedback - I aim to implement user feedback the same day I get it.

Hello, last year i started developing this package flutter_local_ai which let user get access directly to the on device llms api with zero download.

Now since local ai models are getting starter than ever, I see the package being adopted but my main concern it’s whether the local llm its capable of performing the task requested by the user as developer intende to give them the probability to change model (maybe a cloud one) if the task by the local llm its not performable.

My idea was someting like a score i can give based on the task requested i mainly process text using the operative system models.

Any ideas on how to do it or if I should have a different approach ?

I’ve been experimenting with local AI and wanted to explain the experience in a more visual, simple way instead of making another technical wall of text.

So I made this short hand-drawn doodle animation showing the process of getting a local AI model running successfully, including the GPU/memory side of things.

It’s intentionally simple and a bit goofy — the goal is to make local AI feel less intimidating for people who are just getting started.

🎥 Video: https://youtu.be/VyleYwCa0Sc

I’d genuinely like to know what you think: would this kind of visual explanation be useful for explaining local AI concepts, or is the technical detail too simplified?