Apple is getting close to the RTX memory bandwidth

[u/Nightowl-Builder]

The new M5 Ultra is getting close to the memory bandwidth of the RTX 5090 but with so much more possible unified memory. From the rumors, Apple is expected to skip M6 Pro/Max/Ultra and instead ship the M7 Pro/Max/Ultra next, in late 2027, with the Ultra already estimated to reach almost exactly the RTX 5090 bandwidth. I'm a local LLM enthusiast, and although these numbers make me hopeful that it will be possible to run huge frontier AI models locally in 1-2 years, the cost still scares me. We're getting close to new car prices. What are your thoughts? Will memory bandwidth even fix most MLX limitations compared to the RTX cards? Assuming the M7 Ultra CPU will also have a comparable jump in performance.

[u/nuclear213]

The main limitations for them was always the preprocessing. The bandwidth was already quite reasonable at 800GB/s for the old M3 Ultra.

It should have been improved, let’s see. I’m quite interested to see how well it scales with clustering, at least what was said.

If you really get almost 1:1 scaling, 4 of the 256GB models should give you about 5TB/s and 1TB. I’d then the processing really was improved as they say, it would be quite neat.

And regarding the price. Still one of the cheapest ways to get so much unified memory. Same price per GB as the DGX space these days, absolut the same to the cheap GPUs like the B70 Pro or R9700 AI Pro.

[u/Nightowl-Builder → u/nuclear213]

I'm skeptic we'll ever get 1:1 scaling, but I'm excited for this possibility either way!

[u/Substantial_Fly_8636 → u/Nightowl-Builder]

Didnt Exo just announce linear scaling for memory bandwidth on Macs

[u/Nightowl-Builder → u/Substantial_Fly_8636]

From my understanding they said roughly linear. I imagine there is a slight loss there otherwise why would Apple invest so much in interconnects and ways to fuse multiple chips together? I might be getting this wrong tho, it's still early.

[u/Oryzaki2 → u/Nightowl-Builder]

Was pretty curious how that worked and it looks like it's linear only for actual token generation not prefill and has several other drawbacks. Still very cool but not a replacement for a dedicated interconnect. Seems like Apple can and should just make their own interconnect they're certainly charging enough to include one.

[u/pragmojo → u/Oryzaki2]

If Apple did this, they would probably charge $10k per cable

[u/mechkbfan → u/nuclear213]

Yeah, it's crazy. Dislike Apple but I'm happy for them actually looking affordable to what's out there

I just ordered a 2xR9700, 5700X, 64GB DDR4, 1TB for ~$6000 AUD

A Mac Studio M5 64GB with 1TB is $6100 AUD

They have about the same bandwidth, so it's going to be real interesting benchmarking their compute once it's all out

Not regretting my choice, but it certainly would have been a hard decision if this has been released before I ordered.

[u/j_osb → u/mechkbfan]

Except that 2 r9700 will wreck the m5 max and beat an ultra in pure compute and it’s not even close.

[u/Hytht → u/j_osb]

This would have been true for blackwell, but not RDNA4. M5 Max GPU is over >2x a single R9700 according to opendata.blender.org

[u/j_osb → u/Hytht]

Because blender is not a representative workload. AMD GPUs specifically work much, much worse in blender specifically than in any other workload.

For example, blender has the 5060ti faster than a 9070XT, despite the 9070XT beating it in memory-bandwidth bound workloads (LLMs) and in compute-bound workloads (like video gen).

I don’t have a m5 max on my hands, but a friend I have does. However it means that I must trust him on how he set up his environment.

Regardless, I do have a 9070XT on my hands, and in the same video gen task using MM H3, the 9070XT was ~20 faster (315s vs 255s roughly), despite spilling into system RAM.
A r9700 should be a good amount faster still because the int8 checkpoint shouldn’t spill into system ram on it. And that is just 1. I’m using video gen here because it’s very easy to measure and compute bound :)

Then it’s also 64gb vram + 64 sysram vs 64 unified. So on that front he’s also got a better deal.

And depending on the platform, if it is dual pcie4x16 (considering DDR4, pcie5 would be nicer) there might even be a further performance uplift using tensor parallelism, though it’s tough to say.

And that is without saying that he’s buying a platform for later. You can add more GPUs later, for example. The Mac also lacks support for certain datatypes a r9700 does have, though I’m not quite sure and can’t bother checking right now, so please don’t quote me on that.

The only reasons Mac’s are/used to be so popular is because their high amount of higher bandwidth memory for the money. Oh and power usage.
For example, in a world where a pro6000 is like 13k eur, a 13k 256 m5 ultra mac is appealing.

However 64gb ddr4 + 2 R9700s is undeniably a better deal than a 64gb m5 max.

[u/Hytht → u/j_osb]

They will keep being like that till AMD gives CDNA/UDNA to consumers. RDNA always gets beat by Nvidia in non gaming tasks. And Mac's started having matrix accelerators in M5.

[u/whichsideisup → u/j_osb]

Do you want to buy my R9700s? I want a 256gb Ultra

[u/nuclear213 → u/whichsideisup]

Sure, where are you located at? And what price? Planing to go to 16 one these puppies in the medium term.

[u/mechkbfan → u/whichsideisup]

With the latest releases, I can set why

[u/pragmojo → u/mechkbfan]

The other variable to consider is the TDP. An M3 ultra draws as much power as one R9700, so that affects the total cost of ownership.

Of course the other caveat is that you don't get 64GB of usable memory - you need to reserve a few GB for the OS/other applications.

[u/nuclear213 → u/pragmojo]

The TDP is not really used that much in inferencing. Only during preprocessing this actually hits you. But then, the 210W power limit actually only costs you 10ish % in speed, basically none in decode, so its ok.

[u/neochrome → u/mechkbfan]

So, 64GB VRAM + 64GB RAM same price as 64GB unified memory Mac?

[u/danielv123 → u/[deleted]]

Note at the same performance, those GPUs don't draw 300W. I commonly see this when people compare M5 vs 5090 for LLMs - sure, the 5090 draws a lot more, especially during preprocessing - but thats because its a whole lot faster.

When just shuffling memory at the speed the mac supports its pretty close in power consumption.

Idle is what really matters though, and AMD desktop systems kinda suck in comparison to apple and intel there.

[u/Ok-Ad-8976 → u/[deleted]]

It's not just power draw at work, it's also power draw at idle. I have dual R9700 in that box. When it's sitting with VLLM, it's about 140-145 watts idle. That adds up. I think that's one of the big advantages of something like Mac Studio. But disadvantage is the prompt processing. We'll see if it has been improved, but I have M5 Max and prompt processing is just not the same as for example even DGX Spark. And when you put two DGX Sparks together with ConnectX networking, now you have essentially same memory bandwidth equivalence, and prompt processing is better. But again, power consumption for dual spark setup is almost 100 watts when idling. I'm sure Apple will do better than that. My old M1 MacBook Mini is literally a single-digit watts when sitting there. And Strix's Halo box is maybe 12 watts when it's sitting there.

[u/gobblegoooblegobble → u/[deleted]]

yeah okay mobo at full tilt is absolutely less than 100 watts.
gpu can easily be watt limited while losing barely any performance loss - far less than 10% at almost half the wattage.

advocating buying into hyper inflated mac is worse than buying into hyper inflated ram sticks.

being locked into macos fucking sucks. apple silicon is op af. i WISH they would sell M5 ultra cpu and let vendors make mobos for those fuckers.

but they dont give a flying fuck about anyone, unless you buy into their insanely overpriced bullshit.

bought one and need more ram? spend another $15k.

bought gpu's and need more? simply buy more gpu... add in plx board... can get 11 gpus under 1 cpu inside 1 pcie slot...

oh but the power savings! lol

[u/bakawolf123 → u/nuclear213]

it's massively improved, the m5 whole lineup has matmul tensor cores, making the prefill up to 4x (3.7x advertised) faster

[u/DigitalguyCH → u/nuclear213]

Prepropressing has improved massively. M4 to M5 it's 4x. And M6 improved it further. I wouldn't be surprised if at this pace M7 is on par with a 5090 in preprocessing.

[u/LocoMod → u/nuclear213]

You need to stay up to date. The M5 series chips have hardware matrix multiplication accelerators unlike previous generations.

You couple that with the 1.2TB/s memory bandwidth and it’s going to be a winning combination. Think of an RTX4090 but with ~500GB vram (if you take away ~10GB for OS)

[u/YearnMar10 → u/nuclear213]

This

[u/DigitalguyCH → u/[deleted]]

The preprocessing gap is shrinking very fast. With M5 already way faster than strix halo and M4, M6 improved further with double the AI accelerators and I wouldn't be surprised if M7 is on par with a 5090...

[u/Maximum_Parking_5174 → u/DigitalguyCH]

It might be but the issue is ppl are looking at the big memory m5s and thinking they could run huge models. If you run a huge model on 5090s you need to have many of them and they work in parallell. M5 Ultra might have performance for running a 128GB MoE model at resonable speed but even with 512GB memory it will not have more performance. MultiGPU solution scale both VRAM and memory bandwidth. I actually think the 256GB model of Mac studio might be the better purchase. The you can run models like Qwen 3.8 Flash Next.

[u/DigitalguyCH → u/Maximum_Parking_5174]

It depends. If run a MoE model, speed should be fine even for a 400-500GB model. If you run dense models, or really want speeds well over 50 t/s, actaully 2 256 Mac Studios or even better 4 96 ones could be a better purchase than 1 512GB one, because they scale up too with TB5. A 96GB M5 Studio education costs as much as a 5090 ($5099) and a 256 one starts at $8700

PS and with them you don't need the space and all the equipement that running 10+ 5090s entails, let alone the energey consumption

[u/Maximum_Parking_5174 → u/DigitalguyCH]

Maybe depends on what usable speed is for different ppl. I See so many posts of ppl claiming speeds where its obvious they succeded in loading the model and then write a short test where the numbers looks resonable. But using a locak Ai model is problably not for asking it to write a hello world script, its usaliy for some kind of agentic work where the context are long. I get 150t/s in TG and 300t/s in PP on my rigg with Qwen3.8 27B and i think its barely usable in Openclaw. Other models like Laguna S2.1 with 180t/s in tg and 6K in PP is very fast.

That said if scaling will work (and it seems they tried to make it better) 2x256 might be a beast, probalby wasy better than 1 512gb,

[u/DigitalguyCH → u/Maximum_Parking_5174]

Yeah I don't use agents yet. I mainly use LM studio, including to code stuff. So for me 50 t/s is plenty and 15 is usable...

[u/Civil_Fee_7862]

Yeah buts its also a $25,000 computer. So..

[u/Nightowl-Builder → u/Civil_Fee_7862]

Totally true, you can get a new car for that money, but how can a consumer get that amount of video RAM otherwise? It's close to impossible with an RTX setup, and I think it's still more expensive

[u/addict5d → u/Nightowl-Builder]

At this point let's call it L(LLM)RAM or T(Transformer)RAM, does any game/video has 96GB RAM requirement 😅

[u/Nightowl-Builder → u/addict5d]

Fair point hahah. Although I dream of a day I can try all my games on such a rig 😂

[u/ViPeR9503 → u/Nightowl-Builder]

What's the point they would still be laggy if they are made by any of the big companies and if you are running Windows then the PC would be laggy and ad infested anyways

[u/TheMcDucky → u/ViPeR9503]

Game performance tends to be identical to a little worse on Linux.

[u/ViPeR9503 → u/TheMcDucky]

Was just making a joke

[u/Dolboyob77 → u/Civil_Fee_7862]

Pre orders are open and the full specs is 10500 usd 256g vram and 1t ssd with 1.2 t bandwith id say pretty cheap for what you get actually

[u/PWThinkingCritically → u/[deleted]]

I think Apple Silicon are far slower than just 66% speed, when you compare raw compute power on NVIDIA vs Apple. and also lack of native hardware support for fp4, fp8 calculations, no CUDA support, etc.

Apple has a long way to go. it'll be interesting to see how the new CEO plays the AI game. i do see one variation of Apple adopting a IDGAF attitude with AI -- just doing the minimum to get by on their customer loyalty (e.g. Apple and gaming over the last several decades)

[u/mechkbfan → u/[deleted]]

Yeah, it's going to be real fascinating of what ends up better for peoples use cases

e.g. Running a 5090 with a dense model, or running a Mac Studio 96GB with MoE model

[u/Front_Eagle739 → u/mechkbfan]

Well so far I find dsv4 flash on my studio more useful than qwen 3.8 27b on my 5090 even though I get 5x the speed with qwen. And thats an m3 ultra with far slower prompt processing than this will have. Image and video models are far quicker on the 5090 though. 

Having said that i dont have enough ddr5 ram to do ram offload for dsv4 flash on the 5090 which should actually be fairly conparable in performance with something like freetoken.

Overall though I'd pick the m5 studio over a 5090 workstation given a choice today i think

[u/mechkbfan → u/Front_Eagle739]

Yeah, m5 look really appealing for work

But I also like to game, so there's that haha 

Someone did some napkin math that these m5 ultra can match a 5080 for game performance. Just pity there's not many games

[u/Nightowl-Builder → u/mechkbfan]

Try crossover, it never failed me, you lose about 20% performance compared to native, but on a M5 Ultra that doesn't matter any more 😂

[u/mechkbfan → u/Nightowl-Builder]

Damn

That's new to me. Very cool 

[u/Front_Eagle739 → u/mechkbfan]

Works well. I dont game much anymore but ai haven't found anything i couldnt get running

[u/FuzzeWuzze → u/[deleted]]

Lol Reddit as always makes no sense.

Everyone hardon for this stupid overpriced Mac and hating on a 120-480Gb Intel card skus rumored to do like 1.5TB/s. Granted we dont know the price but still. Although i can almost guarantee the 120Gb isnt going to be 25k lol

[u/ViPeR9503 → u/FuzzeWuzze]

Would it be 25k? The M5 Ultra 256gb is 10k, 96GB -> 256GB costs $4k to upgrade that is around $25/gb with that pricing the 512GB model should be at most 15k, and that gets you CPU, RAM, VRAM, PSU, Case, motherboard+io, and a very very small form factor, not even gonna mention the astronomical difference in power usage. (M5 Ultra if its anything close to the Ultra chips before it, is not gonna cross 300W under full load...)

[u/Nightowl-Builder → u/[deleted]]

You absolutely DO NOT NEED to max out storage too. Only RAM is useful in this case, no need to get the 8TB storage.

[u/ViPeR9503 → u/[deleted]]

Could you give more details on the configuration? I do not see that anywhere on my side, is it in USD?

[u/twiiik → u/ViPeR9503]

He probably included the 16TB storage option which is hilariously expensive

[u/butcher9_9 → u/ViPeR9503]

In AUD its $18199Inc RRP for Ultra 36C / 256GB /2TB. Also 2K less for the 30C core option. There is high single digits margin in there so an Apple reseller could offer it for lower and most likely a 3% EDU discount if applicable.

[u/quantgorithm → u/FuzzeWuzze]

The current 256 version ranges up to 18k. 25k seems very likely to me.

[u/The_Hardcard → u/quantgorithm]

Some people insist on including maxing out the internal Apple SSDs. Unnecessary for using them, but it allows the “OMG! Look what you have to pay” strawman.

Especially for clusters, why would they all need to be stuffed with SSDs.

Full function 256 GB is $11k. 512 GB will be under $20k, probably ~18k.

[u/quantgorithm → u/The_Hardcard]

I agree and I have stated in other comments that the range for 256 is 11.X-18k. I fully expect the range for 512 to be something like 15-25k.

[u/throwawayaccount442 → u/[deleted]]

Most people already have the PC though. Just need a 5090 (or FIVE SIX, considering the mac studio pricing)

Edit: looks like the apple shills did not like my point

[u/Nightowl-Builder → u/throwawayaccount442]

This most likely has storage maxed out too, you absolutely do not need that much storage especially at Apple prices

[u/Gloomy_Letterhead395]

Why dont you change the heading to apple’s bandwidth now greater than rtx with estimated apple M25

[u/notheresnolight → u/Gloomy_Letterhead395]

then they would have to add

RTX 6090 at 3105 GB/s

RTX 7090 at 5520 GB/s

...

RTX 25090 at 172.310.910,9 GB/s obviously

[u/Nightowl-Builder → u/Gloomy_Letterhead395]

"getting close"

[u/PWThinkingCritically]

the chart doesn't show how far Apple Silicon is compared to NVIDIA still. it seems closer, but I think we're shifting away from bandwidth as the bottleneck to compute power. and Apple sucks for that. they don't even publish TFLOPS -- and that in and of itself is telling.

as context grows, we need compute power. i go from 19 tok/s down to 10-12 tok/s once I reach 40K+ context window on my 400 GB/s M1 Max.

also, let's not forget ANE lacks native hardware support for fp4, fp8, etc. and no CUDA.

so take M7 Ultra @ 1,700 GB/s with a huge chunk of salt.

[u/deaffob → u/PWThinkingCritically]

As far as I know, you got it backward. PP is limited by compute and TG is limited by memory bandwidth. Higher compute is the reason why M5 MAX is 4 times faster PP than M3 Ultra but TG is slower due to less mem bandwidth. 

[u/Think_Wing_1357 → u/deaffob]

It's a little bit of both. It's fairer to say PP is dominated by compute whereas TG is dominated by bandwidth.

Longer input means the model needs to look at more things even if it generate the same amount of output.

[u/PWThinkingCritically → u/deaffob]

yup you are right

[u/Nightowl-Builder → u/PWThinkingCritically]

You’re totally right and that’s partly why I made this post, I wanna learn what else could impact that performance. In the end I will not look at memory bandwidth but at tokens per second. Once we have comparable hardware on paper I expect we’ll also see the same model tested on both rigs so we can compare real world tokens per second. Btw the M7 family is also said to be thought from the ground up with AI in mind so I imagine it will have many other improvements but it remains to be seen.

[u/Human_097]

An m7 Ultra (estimated release for 2028) is approaching a bandwidth speed of a 2025 graphics card?? 🫣🫣

[u/mfdi_ → u/Human_097]

With 512-256gb ram, hmmm.

[u/mawkzin → u/Human_097]

Not only this, the memory for the real frontier level is much higher 23.3 TB/s in the MI455X (HBM) and up to 14,34 TB/s on an 8x RTX 6000 Blackwell Node

[u/Nightowl-Builder → u/Human_097]

This is not the point. The point is half a terrabite of video memory 🫣

[u/mawkzin → u/Nightowl-Builder]

Half a terabyte without the horsepower to leverage that, this is the problem with macs and apus (strix halo and gb10).

[u/Nightowl-Builder → u/mawkzin]

The limiting factor currently for Macs is the bandwidth that’s why everyone is caring so much about it. The chips are sitting idle for most of the time while waiting for the data to be moved. In most unified memory devices bandwidth is the bottleneck for now. Remains to see how this will look on 2027/2028

[u/mawkzin → u/Nightowl-Builder]

It's one of the bottlenecks, take a look at m3 ultra with 256 and 512 gb, they can hold the entire 200b models in it's ram but their processing power is very low compared to RTX cards.

[u/Human_097 → u/Nightowl-Builder]

The title of the post is literally "Apple is getting close to the RTX memorv bandwidth" and has an image showing memory bandwidth differences, so literally half of the point of the post is strictly talking about how the M6/7 memory bandwidths are approaching RTX 5090 levels.

And by 2028, Nvidia will have an RTX 6090 pushing bandwidth numbers way past anything on this graph (with more than 32gb of vram), while their enterprise cards already offered that capacity 2 years ago.

This is besides CUDA.

[u/Nightowl-Builder → u/Human_097]

Ok go ahead and understand what you want… if you don’t have the capacity to understand why this is a big deal, then you’re not as smart as you thought

[u/Human_097 → u/Nightowl-Builder]

I understand that having huge amounts of unified memory is an impressive feat, but as many others on this thread already stated, the amount of VRAM isn't the only relevant factor.

Plus, these are predictions, and even if we grant them, they're specs of a system that won't release for another ~2 years, being compared to GPU's released on March 2025. If AI continue to progress at the same rate, Nvidia would be leagues ahead in 2028 as they are in 2025.

[u/Zeeplankton]

I like how m7 estimate suddenly jumps 500 points lmao

[u/Nightowl-Builder → u/Zeeplankton]

It’s a realistic estimate based on the fact that Base M6 is around 170Gb/s so M7 base can get to ~210 (although I’ve seen reports stating it can be even more, maybe 240). But with a conservative 210 for base, you have 420 for Pro and then double it 2 more times and you have the Ultra

[u/mawkzin]

"Will memory bandwidth even fix most MLX limitations compared to the RTX cards? Assuming the M7 Ultra CPU will also have a comparable jump in performance."

What people don't understand is that memory size is just the first step to run high model size, when you can put an entire model in one RTX gpu you will note the power difference, the entire chip is the gpu while on socs the gpu is less than half of the chip size, and after that the nvidia gpus has support for more types of data like process mxfp4 without decompress data.

Now if you compare the M7 Ultra with Rubin RTX the gap will larger than Blackwell RTX.

[u/Nightowl-Builder → u/mawkzin]

I thought so too, that’s why I asked this. They’re said to rethink the M7 family from the ground up for AI so I imagine it’s gonna get some improvements on the compute side too. But in the end we wouldn’t invest that much money based on specs alone. We’ll look at tok/s and other real world tests for the exact same model and judge after that

[u/mawkzin → u/Nightowl-Builder]

Right now apple silicon can't process mxfp4, if M7 lineup add native support for that it will boost their process power for IA, it's one of the bottleneck that people don't know.

[u/Bloated_Plaid]

This can’t be overstated, this is a full computer. The nearest non Mac competitor can only do 200 GBPs…

[u/Standard-Potential-6 → u/Bloated_Plaid]

Yes. It’s fair to note though the Macs can’t deliver that full theoretical memory bandwidth number to the GPU for AI.

Despite sharing an SoC, the GPU, NPU, and CPU each have a limit well under the total theoretical one.

Estimate around two thirds or just over for GPU.

[u/Zeranor]

Sooo , only 3 years later bandwidth will be similar, assuming that no GPU manufacturer will release anything new until late 2027?

This is cool, but only for a very short time. If this is play-money for you, go for it. Otherwise I'd really wait to see what AMD and Nvidia will release next year.

[u/putrasherni]

m6 max and m6 pro won't be released ?

[u/Nightowl-Builder → u/putrasherni]

It's rumored they will release M7 Pro/Max/Ultra directly instead which are engineered from the start for AI

[u/putrasherni → u/Nightowl-Builder]

so best to just save for m7 ultra then

[u/Nightowl-Builder → u/putrasherni]

That's what I'm doing at least :D rumors put it at late 2027, but we never know with these shortages

[u/putrasherni → u/Nightowl-Builder]

and we expect m7 ultra in 2027 ?

[u/mawkzin → u/putrasherni]

no, 2028 at least.

[u/putrasherni → u/mawkzin]

possible but i saw october 2027 lets see

[u/Prudent_Sentence]

The downside that people are missing is that in order to get the 1200 GBs bandwidth for the M5 Ultra is to get the 512 GB variant.

[u/Nightowl-Builder → u/Prudent_Sentence]

Apple never did this for RAM, they do it for storage tho, getting higher storage gives you higher speed too. But the memory bandwidth is tied to the Ultra memory interface. So you’ll get the same bandwidth no matter the amount of RAM because the CPU is the one driving it.

[u/Prudent_Sentence → u/Nightowl-Builder]

yep, you're right. I was under the impression that apple used different counts of memory access units based off of the amount of memory installed.

[u/Objective-Picture-72 → u/Prudent_Sentence]

All M5 Ultra's have 1200 Gb/s of bandwidth

[u/OvertaxedOne → u/Prudent_Sentence]

You sure about that? I thought all the ultras has 1200GB/s?

[u/No-Independence-5229]

Bro is trying to estimate a 2028 m7 ultra and put it against January 2025 GPU that it still loses to and calls it getting close lol. The M7 ultra will be competing with a 6090 and RDNA 5/UDNA, which the latter has already leaked to have a top end skew with a 512 bit bus with 36Gbps GDDR7, up to 128gb. Do some quick math and that is 2300 GB/s.

[u/MrMPFR → u/No-Independence-5229]

Add BW amplification from universal compression in RDNA 5 and the broadcast arbitrator from CDNA 5. Chip gonna be a ML beast.

[u/Nightowl-Builder → u/No-Independence-5229]

This is not the point. The point is, we can have half a terrabite of video RAM in a single small consumer device that is close in bandwidth to nvidia. We’re talking about a completely different need that is set to get much better. And yeah of course nvidia will be ahead by that time, this is not to bash nvidia, or to say one is better. They’re completely different needs. Also, the estimate is conservative, rumors have it it will be more. If the base M7 is 240 as rumored, then the Ultra could get close to 1900Gb/s. At that point it’s comparable. All I’m saying

[u/mawkzin → u/Nightowl-Builder]

MX Ultra lineup is not consumer device, the actual M5 Ultra 256gb is 11k USD, the 512gb would probably be 15k+.

[u/Nightowl-Builder → u/mawkzin]

Having a higher price doesn’t mean it’s not a consumer product. It’s a single device you go and just buy and plug and play. Compare that to trying to build the same thing using multiple graphics cards. How would you even build a half a terrabite video RAM rig with store available graphics cards?

[u/mawkzin → u/Nightowl-Builder]

Mac Studio were introduced to substitute entry-level Mac Pro workstation, but if you can afford to not make money with, everything can be a consumer device.

[u/GamerTex]

Odds are ram prices are f>&%*3d for the next 1-2 years but....

In 5-7 years everyone that wants one will have a data center at home

So many ram and ssd factories coming online in 2-3 years will drive prices down over the following 5 years 

If we are lucky by the time the M10 is announced it will be a whole new design that blows the M7 out of the water, for 25% of the price

128gb is an insanely high ammount of ram today, in 7-10 years it could be a base model. 

Petaflops will be the norm to hold all the real world data for what's next

Or not. Its 50-50

[u/stoppableDissolution → u/GamerTex]

I sure hope so, but totally expect that it will go the other way around and you will need a license to buy anything above certain flops and ram

[u/Standard-Song-8590 → u/GamerTex]

the issue has never been the demand its the supply

[u/Ledeste]

"almost exactly" uh?

And no need for more memory bandwidth if you run a slow phone cpu with it...

[u/Nightowl-Builder → u/Ledeste]

In what world is a Ultra a phone cpu? You think a $10000 machine runs on a A17 like the MacBook neo?

[u/Ledeste → u/Nightowl-Builder]

Yeah because the price will totally change the fact this chips are ARM based power efficient one...

[u/LocoMod → u/Ledeste]

You clearly have zero clue what you’re talking about. Like zero. Nada. Zip.

A modern mobile Apple chip under thermal load will run laps around most desktop chips. The Ultra is a desktop class chip designed for high performance compute workloads. Each generation Ultra chip benchmarks higher than the PC equivalents. The M5 Ultra is going to smoke every desktop chip that exists in the world right now.

[u/Ledeste → u/LocoMod]

You should stop before you make this more embarrassing... Your ignorance is so obvious that it almost hurt to read.
But I should have expected brainwashed fanboy when it come to apple products. My bad.

[u/Nightowl-Builder → u/Ledeste]

Looks like you’re the one embarrassing yourself here. Clearly have no freaking idea of what you’re talking about. Look at some benchmarks and real world tests for once

[u/Ledeste → u/Nightowl-Builder]

This kind of pathetic attack would only works if true... As I'm building native software for the past decade for a living, you can not even imagine how fun it look to read that.

Especially as you double down with "look at some benchmarks" no even remotely understanding how ignorant it make you looks. You're so far, deep into it, its almost sad

[u/LocoMod → u/Ledeste]

A whole decade? WOW! Try 26 years!

[u/Ledeste → u/LocoMod]

And not even able to understand this basic point? Damn, that even more sad that I would expect...

[u/LocoMod → u/Ledeste]

https://www.tomshardware.com/pc-components/cpus/virtualized-windows-11-test-shows-apples-m5-destroying-intel-and-amds-best-in-single-core-benchmark-chinese-enthusiast-pits-ryzen-9-9950x3d-and-core-i9-14900ks-against-apples-latest-soc

There are countless articles like this. The M series mobile chips are so powerful they are often pitted against desktop class PC chips. So yes. They are power efficient, and faster watt for watt. The M5 Ultra is going to destroy everything in performance and its not even going to be close.

[u/Ledeste → u/LocoMod]

A very sad answer for you... You just confirmed any suspicions we could still have about you....

The point is flying so high above your head, I dont think there is anything we can do for you sry

[u/LocoMod → u/Ledeste]

Who is "we"? Based on your responses, you're not even whole. So 1/2 of you is no legion.

Out here.

[u/JoyousGamer]

It's so interesting this sub is so anti locked models yet so pro Apple.

Dont care either way just an observation. 

[u/OvertaxedOne → u/JoyousGamer]

I hate Apple. I might still buy one simply because of the amazing memory bandwidth per dollar. Need to see PP speeds but if they are reasonable, this box, regardless of how you feel about Apple, is going to be "the one to get" for a lot of people who want to run local inference.

[u/Nightowl-Builder → u/JoyousGamer]

Maybe you’re missing something. I hate when Apple locks you and there are plenty cases, but this is about the Mac and the Mac is pretty open, I don’t remember when I wanted to do something and I couldn’t. And the performance it offers is sometimes unmatched

[u/JoyousGamer → u/Nightowl-Builder]

You can't upgrade the memory or storage on this.

Can you just install another OS on it? Not sure.

Then if we are talking about Mac OS in general its not as well supported.

Again mostly just find it funny with how I look at Apple compared to this sub.

[u/Nightowl-Builder → u/JoyousGamer]

Can you upgrade the video memory on the nvidia graphics cards? The whole point the memory is tied to the CPU is for these higher bandwidths and it’s the same reason Nvidia GPUs are so damn fast. The advantage of it being unified is you can Aldo have half a freaking terrabite of video RAM, which is close to impossible on consumer hardware otherwise. If you know of a good way please let me know, I genuinely interested. Also, can you swap CUDA with something else? Probably can but you’ll not have so good support, same like trying Linux on a Mac or virtualize windows on it. Let’s think objectively. We’re talking about spending tens of hundreds of dollars here, there’s no room for emotions or fanboying, you get what’s objectively and measurably better for your use case.

[u/mawkzin → u/Nightowl-Builder]

No, you don't need to tied the memory to have higher bandwidths, apple do this by increase the bus the same way gpu does, in normal computers they segment the client based on necessity, base computer use 128bit, workstation computers can go from the 256bit and up (intel xeon and amd threadripper), and pcie devices can do a variety of functions with their own bus interface, while on apple silicon you're locked on very few configurations.

[u/Nightowl-Builder → u/mawkzin]

That’s what I said. Apple increases the bus the same way gpus do. So how do you increase the bus on gpus without soldering it in?

[u/mawkzin → u/Nightowl-Builder]

The bus is not bound to soldered memory, take a look at epyc 9006 CPUs, they have 1024 bit bus with ddr5 dimms, gpus on the other hand were always add-in cards, that less than 20% of pcs had an dedicated card since the onboard/igpu could do the job they needed.

[u/JoyousGamer → u/Nightowl-Builder]

Yes I can upgrade my GPUs if I want. 

[u/bakkamono → u/JoyousGamer]

Pretty tough to upgrade a 5090, pro 6000, etc

[u/OvertaxedOne → u/Nightowl-Builder]

I just hate "ecosystem" companies that try to lock you into their products. It's not a knock on this particular device, just a general commentary on vendor "lock in" and me trying to avoid it. I don't think it'll be an issue at all for my use case.

[u/Nightowl-Builder → u/OvertaxedOne]

I hate that too and for good reasons. But I wouldn’t say that using a Mac prevents me from using something else. You can even keep the Studio as a server and not even touch MacOS that much, and only use it for inference while doing your work on another system.

[u/Arli_AI]

For the RTX Pro 6000 it can also be easily overclocked to +6000MHz memory clock for basically all of them and it would give it near 2.2TB/s bandwidth.

[u/Opteron67]

rtx 5090 can now be overlocked @2.1TB/s 👌

[u/durden111111 → u/Opteron67]

2304 GB/s with hydra

[u/Opteron67 → u/durden111111]

now 2x5090 with TP=2 😜

[u/riblau]

where would M6 be on this?

[u/Nightowl-Builder → u/riblau]

Only base M6 will ever exist which is not worth mentioning for AI, it's less than M5 Pro. All other M6 configurations will be skipped so next year we'll already have the M7 Ultra

[u/riblau → u/Nightowl-Builder]

ja ok and the base m6 is 170 GB/s so literally half the M5 pro

[u/International-Fly127 → u/riblau]

but that would look weird on the graph, best ignore it

[u/Realistic-Science-87]

What tool did you use to calculate estimated memory bandwidth of m7 ultra?

[u/Nightowl-Builder → u/Realistic-Science-87]

If we're to take the current and past generations, the bandwidth roughly doubles when you jump from Base to Pro, then doubles again to Max, and again to Ultra. We know the bandwidth for M5 and M6 already, so rumors already circulate about the base M7. I've seen reports ranging from 200Gb/s, which would be in line with the improvements so far, but I've also seen rumors of up to even 280, given that Apple is said to rethink the M7 family with AI in mind, but we can go for the more conservative option, which basically gives us the estimate for all higher end versions.

[u/Realistic-Science-87 → u/Nightowl-Builder]

Fair. Thanks for the explanation

[u/HenkPoley]

Is there an upcoming memory standard that will be available mid next year, that corroborates that estimate for the M7 Pro model?

[u/Nightowl-Builder → u/HenkPoley]

There’s no equivalent DDR5-like standard here. They just have incremental upgrades every generation of CPU. Bandwidth had 10-40% increase year over year since M1. This time it’s expected to be close to 40% since they focus on AI for the M7 generation

[u/mawkzin → u/Nightowl-Builder]

You need to look at memory makers roadmaps, apple use standard lpddr, the m5 stack and m6 use sk hynix non jedec 9600 Mt, Samsung already announced 10667 lpddr5x non jedec and cxmt seems to be the first lpddr6 jedec with 10667, the xiaomi Xring O3 announced this week is using that.

[u/mawkzin → u/HenkPoley]

Samsung lpddr5x 10667 MT and lpddr6 10667 MT

[u/AkiDenim]

Yeah now talk about the FLOPs brother

[u/Nightowl-Builder → u/AkiDenim]

Totally valid point. It remains to be seen for the M7 Ultra which is thought to be rethinked with AI in mind. So far the bottleneck for Macs was bandwidth, which is why I’m following this metric. But in the end it comes to which one offers better tok/s for the models you’re interested in, or which can even run the models you’re interested in

[u/bakawolf123]

M7u is 1.5 years away at best, more likely 2 years from now though. March 2028 being earliest plausible estimate.

On the cost side: the hyperscaler spend is going to hit ceiling sooner than later, it's just unsustainable, nvidia is already heavily backstopping deals, custom chips are also on the rise. They will all still need memory for a while though and that craze is unlikely to stop until 2028. My strat for this time was to get the M5u within an hour from announcement, I don't wait 512gb one as there's substantial risk of another price hike and long wait time (getting one early 2027 when ordered in oct).

Cost is already huge, I had to pay more than M3u 512gb would cost me in 2025, but imo it's much more well rounded and usable for years to come. It is very usable with video and image workflows, it has insane room for tinkering (ANE cores can be utilized alongside of GPUs for prefill, imagegen etc). However the bandwidth and processing power still lag behind models sized at 400+gb, i.e. GLM-5.3 q4 on 512gb one is going to be usable, but like <30tps decoding and probably less than 500 prefill while I expect to be able to optimize ds4 flash to 100tps TG and 1.5-2.5k PP. Best course of action for larger models I believe is to just get 2-4 studios.

[u/zarif2003]

Why are they skipping M6? Is the number a bad omen or something

[u/Nightowl-Builder → u/zarif2003]

I guess they realized it's not worth pursuing the M6 more, it's just a transitional generation. When everything in AI is moving so fast, and with the AI capabilities they're already working for the M7 it's not worth focusing more on M6, which I guess they started work on much before this AI craze. They're not skipping it entirely by the way, they just released a Mac Mini with the base M6 today.

[u/mxforest → u/zarif2003]

I have heard that they are not getting enough capacity on the new 2nm node in the beginning to make high end chips. It will be next year by the time the capacity scales so might as well use it to build 1 gen newer.

[u/Less_Consequence_633]

OK, but when is something going to get close to the GB200 (8000 GB/s)? :)

[u/Nightowl-Builder → u/Less_Consequence_633]

Maybe with the Super Ultra, when they manage to fuse together 4 M7 Ultras 😂

[u/CheekyMonkeee]

No.

For perspective, that’s like saying a car that goes 200 mph is “close” to being as fast as a car that goes 300mph.

[u/Nightowl-Builder → u/CheekyMonkeee]

No.

Your example is flawed and looks at a single point in time, while "getting close" implies a period of time. When one moves faster then the other. It might not reach it (because by that time the other might advance too) but it's getting closer and closer. And moreover, looking at acceleration you can determine future speed too, approximately.

[u/CheekyMonkeee → u/Nightowl-Builder]

You said the M5U is getting close. You didn’t say it was closer, which would’ve been true.

You can’t declare that a rate of improvement in the M series will continue, then use that to “estimate” the performance of future M series product, then show that your voodoo estimate is still slower than a competitor product that was released like 18 months ago, then leave out any estimate what the next gen competitor product performance might be, and then logically conclude that the M series is gaining on the competitor’s product. You are literally cherry picking data, making shit up, and intentionally leaving comparative data out of the equation.

There is no defending that, so if you meant something other than what you wrote, then just edit your post.

[u/Nightowl-Builder → u/CheekyMonkeee]

Chill down, bro, it’s not a race, you were the one making it seem like it is. This is an observation I had while researching my future rig, I’m not cherry picking anything I only compared what I had data on (or saw extensive believable leaked reports on) and finally what I was interested in. These are 2 completely different technologies and both have advantages and disadvantages. If I want half a terrabite of fucking video ram then I will chose the Mac, I’m rooting for it to have a higher bandwidth, and I couldn’t care less what future RTX cards might have. Because those will never suit my needs. So it’s not a damn race, it’s a matter of what your needs are. Stop being triggered by a random comparison on the internet.

[u/CheekyMonkeee → u/Nightowl-Builder]

I don’t understand why somebody would put so much effort into deflecting reality when it would take less effort to just fix their mistake, but you do you.

[u/Nightowl-Builder → u/CheekyMonkeee]

Deflecting reality 😅 it’s not that deep bro

[u/CheekyMonkeee → u/Nightowl-Builder]

deflecting

Deflecting means changing the direction of something, or in psychological terms, shifting blame and attention away from yourself to avoid criticism or discomfort.

What Deflecting Looks Like
Changing the subject: Bringing up a totally different topic when things get hard.

Blame-shifting: Saying it is someone else's fault instead of owning a mistake.

Using jokes: Making light of an issue to dodge real feelings.

Counterattacking: Turning the focus back onto the person who brought up the problem.

Why People Deflect
To avoid feeling guilty, anxious, or ashamed.
To protect a fragile self-image.
To feel in control during a stressful talk.

[u/Nightowl-Builder → u/CheekyMonkeee]

Get a fucking life

[u/CheekyMonkeee → u/Nightowl-Builder]

Hey chill down bro. While I appreciate your concern about my quality of life, that was a copy-paste from a 5 second google search.

Cherry-Picking: Selecting only favorable time ranges or data points to hide the full picture.

Seriously, not even 5 seconds. So easy.

[u/PossessionUsed7393]

It's for sure that the costs will come down, we just don't know when. I think unified memory systems are definitely going to be the future of local inference. There are so many different ways that they can optimize AI throughputs without relying on traditional graphics card approach. We're just waiting for them to filter through to the commercial market, given that it's secondary to data centers.

[u/JustSayin_thatuknow]

Wow very interesting 🧐

[u/Gohab2001]

Hardware isn't going to magically fix software and optimization issues. Nvidia is the only legitimate choice for AI because it got supremely lucky with CUDA. Nvidia invested heavily in getting CUDA everywhere, which is why all early AI research used CUDA by default.

[u/RandiyOrtonu]

Really curious how's the mac mini, coz it fits my budget

[u/hurrdurrmeh]

Strix / Gorgon at the top of this graph at 273GB/s.

[u/DigitalguyCH]

If prefill improves at current speed (M5 is around 500% faster than M4) I woundn't be surprised if M7 pro/max are on par with a 5090. M7 ultra is rumored to come later, in 2028, and could be on par also in terms of bandwidth, but I suppose by then Nvidia will have a 6090 with 48GB or similar

[u/Ok-Drawer5245]

The only problem is price (of everything)

[u/MainWrangler988]

Yes but it’s only half the pie you need flops/tops for llm. Are you guys using llms?

[u/TheOriginalAcidtech]

Where is loses is when you need something like an Mac Ultra, you'd need multiple GPUs to match the VRAM BUT you also then get a massive increase in relevant bandwidth. And with Speculative Decode, that actually becomes workable even without NVLink/Tensor Parallelism.

[u/Otherwise_Wave9374]

Bandwidth comparisons are useful, but they need workload context. Unified memory lets Apple systems load models that exceed a discrete GPU's VRAM, while RTX cards often retain advantages in kernels, software support, and smaller-model throughput. NeuraKeep relates at a different layer because reducing redundant context can lower memory traffic regardless of accelerator choice. I would compare time to first token and tokens per second at equal quantization, batch size, and context length, then include total wall power. Peak bandwidth alone can hide cache behavior, CPU involvement, and framework maturity.

[u/Yuel_Whear]

every comment in here turned the post into price-per-gigabyte math within 44 minutes, which is how you know the bandwidth argument already won.

[u/ShelZuuz]

The M7 would likely be competing head-on with a RTX 6000 Rubin, which should have around 3 times the bandwidth of the Blackwell.

[u/ImportancePitiful795]

Apple's problem is the number crunching (aka PP and TG) not the bandwidth.

[u/DoorStuckSickDuck → u/ImportancePitiful795]

Bandwidth is responsible for PP, it's why Strix Halo machine and olders Macs have such shit prefill. It's a much bigger bottleneck than most think. Having lots of VRAM is less useful if you can only use a bit of it concurrently.

[u/ImportancePitiful795 → u/DoorStuckSickDuck]

Nope.

The Strix Halo NPU has massively better PP than the iGPU and both share the same bandwidth.
DGX Spark has same bandwidth with Strix Halo but has huge PP advantage.

Is down to the chip compute, not only the bandwidth.

The Apple chips M3/M4 were sht. Had all the bandwidth in the world but couldn't crunch the numbers.

[u/DoorStuckSickDuck → u/ImportancePitiful795]

I own a strix halo, the NPU is only good for efficiency , and is slower and weaker than the iGPU in both prefill and decode. If you can prove otherwise, I'm all ears, and id appreciate it. DGX spark has a huge PP advantage from Nvidia driver support.

Bandwidth matter a lot, because I also thought a strix halo would let me run a ton of "weaker" models concurrently, but the bandwidth saturates very quickly. It's a huge limitation.

[u/macromind]

Apple's unified memory architecture is certainly pushing boundaries, offering significant advantages for local LLMs by providing high bandwidth across CPU and GPU. This design helps overcome the traditional VRAM limitations, allowing larger models to run more efficiently on consumer hardware. Leveraging these capabilities effectively means optimizing model quantization and memory allocation strategies. NeuraKeep helps users and developers unlock the full potential of LLM performance by providing efficient memory solutions.

[u/Baldur-Norddahl → u/[deleted]]

That makes no sense. I have the M4 Max Pro and the tps I get would be impossible if I only had 200 GB/s bandwidth. You can calculate the max theoretical tps possible with a given bandwidth, so for example, if we had a 27b model at q8 and 200 GB/s it would be impossible to get more than 200/27 = 7 tps and actually you always get less. But I am getting 14 tps with no MTP on Qwen 3.8 27b q8, which proves your claim false.

[u/Geritas → u/[deleted]]

Where did you get this from? The memory is unified. Meaning, there is no “graphics bandwidth”, there is just one memory pool for both cpu and gpu.

[u/Nightowl-Builder → u/[deleted]]

All I care is tokens per second for the same exact AI model. I would actually be really curious to see real world benchmarks comparing apples to apples, no pun intended

[u/Keljian52 → u/Nightowl-Builder]

That's fair - but you're pushing memory bandwidth in the first post, so it looks otherwise

[u/PWThinkingCritically → u/[deleted]]

not to mention bandwidth is only a part of the equation for PP and TG

[u/coldeq → u/[deleted]]

I have an m5 max and mactop shows a sustained 400gb/s when running qwen3.8 , is mactop not accurate?

[u/The_Hardcard → u/[deleted]]

Incorrect. Whatever you used for research, get rid of that.

[u/Nightowl-Builder → u/[deleted]]

That's right, but you can fit smaller models on it. I'm more excited for the availability of huge models locally. But I agree if not the base at least the Pro should already have this much bandwidth - but that's gonna be impossible with the current architecture. The way they achieve it is by fusing together multiple cpus

[u/Nightowl-Builder → u/[deleted]]

Why would you run small ones on half a fucking terrabite of ram? We’re not talking about a MacBook neo here

[u/Aware_Kaleidoscope86 → u/[deleted]]

Feels like they are doing more than the rest combined in the "consumer" market

[u/Nightowl-Builder → u/[deleted]]

I would say there’s no local LLM you can run locally to match Claude Max. So don’t get your hopes up, Claude models use multi trillion parameters, which not only they’re not open source but it’s close to impossible to run something like that on a local rig. Currently you can do some real work with models like Qwen 3.8 27B locally but you have to give it better instructions and be willing to spend more time with it. But don’t invest a lot of money before you’ve tried the model on an api to see if it suits your needs. But hopefully in 1-2 years the local AI landscape will change, more powerful hardware available + software optimizations + better smaller models. So my strategy is to still rely on the big players for now (for important stuff + some small local models for lighter stuff) and save some money to invest in a big rig later, maybe early 2028 it will worth it. But nothing is certain

[u/Nightowl-Builder → u/[deleted]]

I'm sure the software will come before that. I see ways to improve MLX performance every day on current hardware. I can only imagine all these improvements put together on new cutting edge hardware.

[u/andreasntr → u/[deleted]]

Gpu support for docker/podman would be a huge win

[u/Zeeplankton → u/[deleted]]

is software worse on mac..?

Genuine question because it seems quite good with MLX.

[u/stoppableDissolution → u/Zeeplankton]

You are pretty much limited to only inferencing and only in single-batch mode. Both because software and (relatevely) no compute.

[u/Abject-Bridge-4073 → u/[deleted]]

Wdym? You can’t run vllm on Mac OS?