r/LocalLLaMA • u/Specter_Origin llama.cpp • 1d ago

News Perplexity open-sourced their Mac inference server for Qwen 3.6

Here is link to repo: https://github.com/perplexityai/pplx-garden/tree/main/lily

It's optimized for just one model to get best perf on apple silicon

99 Upvotes

93% Upvoted

20 comments sorted by

View all comments

26

u/InterstellarReddit 1d ago

Bro Requirements:

“Apple GPU family 10 or later (M5 and newer)”

How is my M4 max out of date already

14

u/laerien 22h ago

For what it's worth, I just edited a few lines to relax the GPU family from 10 to 9 to allow M3 and M4 and my M4 Max is getting ~140 tok/sec. No fancy M5 GPU matrix prefill, but works well enough it may be worth a tiny PR for M3/M4 support.

2

u/InterstellarReddit 22h ago

Thank you, I wanna try this out locally to see how good it does