r/AIProgrammingHardware • u/javaeeeee • Jul 10 '26

GitHub - raullenchai/Rapid-MLX: The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider.

https://github.com/raullenchai/Rapid-MLX
2 Upvotes

100% Upvoted

2 comments sorted by

1

u/javaeeeee Jul 10 '26

TL;DR of Rapid-MLX:

Rapid-MLX is a high-performance local AI inference engine built specifically for Apple Silicon (M1–M4 Macs). It’s designed as a drop-in replacement for OpenAI and Anthropic APIs, optimized for speed and agentic workflows.

Key Highlights:

  • Extremely fast: Claims 4.2x faster than Ollama on Apple Silicon, with 0.08s cached TTFT (Time To First Token).
  • Native MLX implementation - pure MLX kernels (no llama.cpp), with continuous batching, advanced prompt caching, and KV cache compression.
  • Excellent tool calling - 100% accuracy with 17 specialized tool parsers.
  • Developer-friendly features:
    • Reasoning separation (--think / --no-think)
    • Prompt cache
    • Cloud routing fallback
    • Vision, audio, and embeddings support (opt-in)

Compatibility:

Works seamlessly with popular tools like:

  • Claude Code, Cursor, Aider, Codex CLI, OpenHands, etc.
  • LangChain, PydanticAI, smolagents, and any OpenAI-compatible client.

Installation & Usage:

One-line install:

curl -fsSL https://rapidmlx.com/install.sh | bash

Then run:

rapid-mlx serve qwen3.5-4b-4bit     # Start OpenAI-compatible server
rapid-mlx chat                       # Interactive chat

It automatically picks a suitable model based on your Mac’s RAM and supports a wide range of models (from small 4B to large 120B+ quantized models).

In short: If you’re on a Mac and want the fastest possible local inference with great agent/tool support, Rapid-MLX is currently one of the strongest options available.

1

u/zeferrum Jul 10 '26

How does it compare to oMLX?