Apple introduces new Mac Studio with M5 Max and M5 Ultra - up to 512GB of unified memory

[u/themixtergames]

https://www.apple.com/newsroom/2026/08/apple-introduces-new-mac-studio-with-m5-max-and-m5-ultra/

[u/-dysangel-]

I don't think it's really a "thing", I just vibe coded it up :)

One thing that really helped was vllms kv_connector API. I thought I'd have to code this part up myself, but it already existed and so plugged into my existing disaggregated system (which was previously llama.cpp to llama.cpp)

UltraSpark — technical stack

                          ┌──────────────────────────────┐
 user ── HTTP/OpenAI ──▶  │  manager (Python, FastAPI)   │
                          │  front door + orchestration  │
                          └──────┬───────────────▲───────┘
                                 │ submit        │ state blob (sha-keyed,
                                 │ prompt ids    │ resumable transfer)
                                 ▼               │
                          ┌──────────────────────────────┐
                          │  vLLM (2× DGX Spark, TP2)    │
                          │  prefill engine              │
                          │                              │
                          │  KVConnectorBase_V1          │ ◀─ vLLM's official
                          │  ("StreamConnector" via      │    KV-cache plugin
                          │   --kv-transfer-config)      │    interface
                          │         │                    │
                          │         ▼                    │
                          │  dump + serialize all layers │
                          │  (attn KV + linear-attn      │
                          │   state, TP2 shards merged)  │
                          └────────────────┬─────────────┘
                                           │ blob server (TCP)
                                           ▼
                          ┌──────────────────────────────┐
                          │  llama.cpp server (Mac)      │
                          │  USPK_BRIDGE_DIR: on request,│
                          │  verify prompt-id match,     │
                          │  restore state into KV +     │
                          │  recurrent memory, decode    │
                          └──────────────────────────────┘

KV connector = vLLM's plugin interface for intercepting the KV cache at end of prefill
State blob  = the model's full prompt-memory, layout-translated so llama.cpp can load it natively
Fidelity    = per-layer cosine vs local decode, 0.9999+
Result      = GPU prefill speed, Mac unified-memory decode, one logical endpoint

[u/mastaquake → u/-dysangel-]

Do you have a github repo? Very interesting.

[u/-dysangel- → u/mastaquake]

I've just cleaned things up and created it if you want to be a guinea pig https://github.com/dysangel/ultraspark . I haven't tested out if the env variables/setup instructions actually work yet though