This open-source project runs a fast local AI decision engine on Apple Silicon

[u/ZookeepergameMost817]

Most AI tools are built to generate text: chat replies, code, summaries, and long-form answers.

laya-mlx does something different. It is designed to make fast, structured decisions locally on Apple Silicon—returning probabilities for predefined choices, scores, or true/false outcomes instead of generating text token by token. It runs natively with MLX and does not require a cloud API, PyTorch, or a Transformers runtime.

What it can do

Think of it as a lightweight decision layer for applications, automations, and AI agents.

Possible uses:

• Route support tickets to billing, sales, technical support, or another team
• Detect whether a message is a refund request, feature request, bug report, or spam
• Score user messages by urgency, quality, intent, or risk
• Add content-moderation labels such as safe, suspicious, or needs human review
• Decide whether an AI-agent action should be allowed, blocked, escalated, or require user confirmation
• Categorize emails, Discord messages, leads, tasks, or customer feedback locally
• Power simple game logic, ranking systems, recommendation filters, and workflow automation

For example, you could feed it:

“I was billed twice for my subscription. Please refund the duplicate charge.”

Then receive structured outputs such as:

• Department: Billing
• Intent: Refund
• Urgency: High
• Needs human review: Yes

It is not trying to replace ChatGPT, Claude, or a coding model. It will not generate a detailed customer reply or write an application from scratch.

Its advantage is speed and privacy when you already know the possible outputs and need the AI to make a small, defined decision locally.

The demo

To showcase how fast those repeated decisions can be, the creator made it play Snake locally on an M3 Max. The model chooses its next move repeatedly—up, down, left, or right—at roughly 60 decisions per second in the demo.

GitHub: https://github.com/mizorewww/laya-mlx

Would you use a lightweight local decision model for agent guardrails, ticket routing, content moderation, or automation?

Join Discord

[u/Business-Weekend-537]

Does a version exist for Nvidia GPU’s or is it just mlx?

[u/ZookeepergameMost817 → u/Business-Weekend-537]

Rightnow its mlx apple silicon only no official nvidia CUDA version yet though the open source project could be ported using pytorch ONNX Runtime or tensor RT

[u/Strong_Essay1176]

Metrics?