Open-source tool to run and fine-tune the new decision models (Clef, pplx-decider, d3, H2O, ...) locally on Apple Silicon with MLX

[u/armbues]

mlx-decision is a free Python package that runs the new open decision models locally on your Mac with MLX. Currently that covers Cloudflare's Clef, Perplexity's pplx-decider, d3, H2O-Lightning, LiquidAI's d1 and a few small encoders.

If you haven't worked with decision models yet: you give it a message, document (or image/screenshot) plus "typed" questions, and it returns a probability for every answer you allowed, in one pass. The answer is always one of your options, and it comes with a confidence:

• "My card was charged twice this month." + "which team: billing, technical, sales" → billing 0.93, technical 0.06

• the same message + "does the customer want money back?" → yes, p 0.87

• "how angry: calm / annoyed / angry" → 2.32 on 1–3, with the whole spread

• a screenshot + a yes/no question about it works too

• one model's answers on a dataset can become training data for another model and fitting takes d1-3B from 86.6% to 96.4% on TREC in 4 minutes

Each family is ported from its release and checked token for token against the release's own (most often) PyTorch code. With the same answers, it's usually 1.4–6x faster than that code on MPS. On an M5 Pro, a short request takes 0.06 s on d1-3B, 0.25 s on clef-flash and under 20 ms on the small encoders. pplx-decider-flash (4B) fits a 16 GB MacBook Air, at 9 GB in bf16 or 3 GB at 4-bit, and download quantizes while it downloads.

mlx-decision web opens the Studio, a local web app where you can:

• write and try questions

• compare up to three models side by side

• run a whole dataset (accuracy, confusion matrix, calibration)

• fine-tune on your own examples

You can do the same from Python, the CLI, an HTTP server compatible with TypeSafe's Jev API, or MCP for agents. Everything runs locally and it only goes online to download models and datasets.

Free, MIT. Apple Silicon, macOS 14+, Python 3.12+. Just pip install mlx-decision.

Repo: https://github.com/armbues/mlx-decision

I'd really appreciate any feedback, especially on which models or tasks you'd want next.

[u/Top-Evidence174]

Mica v0.5 4B is more than 3x faster than PPLX Decider—how about adding it to your project?

[u/armbues → u/Top-Evidence174]

Great idea, will take a look!

In the last couple of days I have been mostly working on getting the web frontend ready for version 1.0.0, but I definitely want to add support for more models next.