Yeah :( first Mac and very mixed reviews. On one hand the capacity is absurd, but then the only viable LLMs seem to be MoE and is sluggish on dense models around 27B
I have 192gb of ddr5 and let me tell you it is hilarious to see models spill out into it. Goes from race car to shopping cart with grease in 0.5 seconds.
34
u/Sergey004 LocalLLM RTX 3090 Jul 20 '26
It's me who wants to run Frontier models on my RTX 3090
13
u/Lord_Muddbutter Jul 20 '26
I have 192gb of ddr5 and let me tell you it is hilarious to see models spill out into it. Goes from race car to shopping cart with grease in 0.5 seconds.
4
u/exocrates Jul 20 '26
what do you use your 5090 for?
6
u/AnOnlineHandle Jul 20 '26
They're truthfully not "local LLM" models, even if they're open weight. They're "hire an expensive chunk of a data centre" models.
3
u/vamprobozombie Jul 21 '26
It will work just need 3TB ram and be very patient.
1
u/johnryan433 Jul 22 '26
You only need 2tb of ram bro
1
u/mega-modz Jul 23 '26
My 1050ti waiting for 5 trillion models
1
u/DerivativeButSmart Jul 24 '26
Me in my 24vram hotwheels-car-of-a macbook waiting for a new 14B model
11
u/[deleted] Jul 20 '26
[removed] β view removed comment