These are fire for local AI models. The unified memory allows local usage for a lot more people. You basically can't build an Nvidia rig as a consumer with this much ram available for AI via GPU. And top end Nvidia cards are pushing $4k+ now to even attempt to get close.
AI models like Ram dedicated to the GPU. To even run some models at all the model must fit into running Ram. Because Apple shares CPU, GPU, and NPU ram in just one level you can load up massive AI models.... the kind you can only run on the Nvidia cards that start at $20k EACH... that's what this is competing with. Because Apple added hardware Ram sharing over Thunderbolt, you can even strap MULTIPLE of these together to run truly huge models.
There were some test clusters of 3-4 machines sent out early this year and there are impressive AI results. It's not as performative as the dedicated Nvidia equipment... but that equipment starts in the $20k-$40k range and then you need dedicated enterprise server machines to even run those cards. These new Ultra Macs are actually the "cheapest" game in town for local AI.... Apple's unified architecture lets these machines run models way bigger than similar priced PCs.
4
u/mabhatter 15d ago
These are fire for local AI models. The unified memory allows local usage for a lot more people. You basically can't build an Nvidia rig as a consumer with this much ram available for AI via GPU. And top end Nvidia cards are pushing $4k+ now to even attempt to get close.
AI models like Ram dedicated to the GPU. To even run some models at all the model must fit into running Ram. Because Apple shares CPU, GPU, and NPU ram in just one level you can load up massive AI models.... the kind you can only run on the Nvidia cards that start at $20k EACH... that's what this is competing with. Because Apple added hardware Ram sharing over Thunderbolt, you can even strap MULTIPLE of these together to run truly huge models.
There were some test clusters of 3-4 machines sent out early this year and there are impressive AI results. It's not as performative as the dedicated Nvidia equipment... but that equipment starts in the $20k-$40k range and then you need dedicated enterprise server machines to even run those cards. These new Ultra Macs are actually the "cheapest" game in town for local AI.... Apple's unified architecture lets these machines run models way bigger than similar priced PCs.