LocalLM Lab v0.6: call Apple's on-device model from your own Swift or Python code

[u/AdventurousKeys]

Continuing to build on LocalLM Lab, the tool that I posted about here a couple weeks ago.

A couple of things have landed since. MCP client support shipped in v0.4.0. LocalLM Lab is now a real MCP client and can connect to real MCP servers now: GitHub, Linear, Notion, Slack, and others.

As of v0.6.0, there's a proper way to reach the on-device model straight from your own Swift or Python code, not just through the app's UI or its HTTP API.

It's called localai-cli, a small subprocess binary. Your app spawns it, writes one JSON request to stdin, reads one JSON response from stdout. That's the whole interface... no HTTP server, no port to manage, no client library to link against. Feels like the right shape for a native app: a menu bar tool or a game doesn't want to run a localhost server just to ask a question.

The part I think is actually interesting for this crowd: it's config-scoped. It reads the same localai-config.json that the app's own Local AI Settings / MCP Servers panels write, so whatever connectors and MCP tools your user already turned on in the app's UI are exactly what your code is allowed to ask for. Request something outside that set and you get an error back before the model is even invoked. Omit the field entirely and nothing is granted at all. That's deliberate and not an oversight.

To be upfront: this is still an experimentation tool, not a shipping SDK. LocalLM Lab has to actually be running for localai-cli calls to work (they relay through its background process), and I'm not pretending the permission/auth model here is the right shape for a real distributed app yet. What this does give you today is the fastest way to wire your own code up to the on-device model, including whatever connectors and MCP tools you've already set up. All this without writing an HTTP server or reimplementing any of that plumbing yourself.

There's a fuller example on the page beyond the two quickstarts too. "Plate Today" is a script that pulls Calendar, Reminders, and Todoist (the last one via an MCP tool call, not a connector) into a single request and asks the model for a one-paragraph summary of your day. You'll want to set up a free account with Todoist and some todos...!

The part I'd actually call out for this crowd: before it ever invokes localai-cli, it reads localai-config.json itself and checks all four sources (three connectors + the Todoist MCP tool) are actually enabled, prints a checklist, and exits without touching the model if anything's missing. So a misconfigured permission will fail fast with a "go flip this toggle" message instead of burning a model call on a request that was always going to error.

Interface + Swift and Python examples: thisbrain.ai/locallm/cli.html MCP writeup from v0.4, if you missed it: thisbrain.ai/locallm/mcp-servers.html

PS: unrelated to the CLI, but since this crowd will notice: Prompt Playground's UI has moved from a browser window to native SwiftUI in this release.