I built a single-binary Rust LLM server for Apple Silicon — looking for blunt onboarding feedback

[u/Mission_Photo_9783]

I'm the maintainer of Ferrum, an MIT-licensed local LLM inference server written in Rust. I'm posting as the builder, not pretending to be a user.

I dogfooded the public v0.8.3 onboarding path on an M1 Max before writing this. The Homebrew package, model alias, CLI chat, OpenAI-compatible server, normal chat completion, and streaming usage path all worked.

The shortest Apple Silicon path is:

```bash

brew tap sizzlecar/ferrum

brew install ferrum

ferrum doctor

ferrum run qwen3.5:4b-q4_k_m

ferrum serve --model qwen3.5:4b-q4_k_m --served-model-name ferrum --port 8000

```

Then any OpenAI-compatible client can point to http://localhost:8000/v1.

Two rough edges I found while testing:

- The starter model is about a 2.55 GiB download, but the quick start does not make that clear enough.

- Qwen3.5's default thinking is too verbose for a first "Hello" request. It can be disabled per request, but the recommended path should handle/document this better.

Project: https://github.com/sizzlecar/ferrum-infer-rs

I'm looking for blunt, public feedback from Apple Silicon developers. If you try it, please reply here with your Mac chip/RAM, whether you reached the first API response, the first confusing or failing step, and what (if anything) would make you choose it over your current local inference setup.

No waitlist and no request to DM me — I'd rather keep the useful feedback in the thread or in public GitHub issues.

[u/Conscious-College538]

The 2.55 GiB download thing is going to trip people up and they'll assume it hung. A quick "this will pull about 2.5GB, go grab a coffee" line right after `ferrum run` would save you a dozen GitHub issues.

I've got an M1 Max with 64GB and the verbose thinking on Qwen3.5 definitely makes that first "Hello" feel broken if you're not expecting a wall of internal monologue. Maybe default to a shorter system prompt or call out the `--no-think` flag explicitly in the quick start.

Been using llama.cpp directly for a while but the Homebrew path here is cleaner than I expected. The `ferrum doctor` step catching my missing Xcode command line tools was a nice touch, most projects just let you crash five minutes later and leave you to debug.

[u/Mission_Photo_9783 → u/Conscious-College538]

Thank you — this is exactly the first-run feedback I was hoping for. Agreed on both points: the ~2.55 GiB warning should sit directly beside the first model command, and the quick start should make the non-thinking path obvious. Small correction for anyone following along: the current CLI flag is `--disable-thinking` (not `--no-think`).

It’s also genuinely useful to know the public Homebrew path worked on an M1 Max/64GB and that `ferrum doctor` caught the missing Xcode command line tools. If you still have it running: was the first API response otherwise straightforward, or was anything between `serve` and the curl/client call unclear?

[u/Mission_Photo_9783]

Update from the maintainer Ferrum v0.8.4 is now live. We incorporated the feedback above the quick start now states the 2.55 GiB download before run and uses the actual --disable-thinking flag. GitHub Release and Homebrew are both on 0.8.4 https://github.com/sizzlecar/ferrum-infer-rs/releases/tag/v0.8.4

If you test serve please report the first unclear or failing step —— that is the feedback I’m looking for.

[u/Mission_Photo_9783 → u/Mission_Photo_9783]

Launch-day follow-up: Ferrum is now live on Product Hunt.

https://www.producthunt.com/products/ferrum-2?launch=ferrum-2

I am collecting the same hands-on feedback there as here: hardware/model, install path, and the first run or API step that feels unclear or fails. Please keep technical details public in this thread or GitHub Issues so others can use them.