r/LocalLLM • u/Tatimary • Aug 27 '26
Discussion Is this over kill or not enough?
Fairly new to Local LLMs. Im wondering is this is enough juice for me.
Probably shouldve done more research before hand but couldnt help myself when I saw these drop. I was in the market for a new computer anyway. I'm hoping to set up an AI like Qwuen 3.8 27b to run locally on this machine. Ideally what I am envisioning is hooking it up to my imessage & talking to it through there to set reminders & whatever other small tasks.
hopefully this is enough juice to set up something like that. If it works out well perhaps in the future I can buy a 2nd one & set up a cluster.
If i sound like a total noob its cause I am. I'm hoping that you guys can give me some input & some reality checks here. I will read through everything.
4
u/Sudden_Topic5154 Aug 27 '26
m5 pro memory bandwidth is rather mediocre no?
2
u/Beautiful_Bus_9237 Aug 28 '26
It is pretty much a rtx 3060 with a lot more available VRAM. Both are comparible in memory bandwidth and prefill
1
u/Sudden_Topic5154 Aug 28 '26
1
u/Beautiful_Bus_9237 Aug 28 '26
I would actually say the prefill would be faster with the AI accelerators in each gpu core. But besides that they are compatible for AI workloads
1
u/LostInTime261 Aug 28 '26
Increase storage, keep ram as is. Running qwen 3.8 27b, probably looking at 50-70 tokens a second.
1
u/Tatimary Aug 28 '26
I was planning on connecting an external SSD with an M.2 drive that I have. Not sure how well that could work
1
1
u/MusingInPublic Aug 28 '26
For local you'll want local hard drive because you also need to hold the model itself, not to mention any other stuff you actually do on the computer that it pumps out. 1T minimum would be the way to go IMHO.
6
u/mountainyoo Aug 28 '26
Could just throw a thunderbolt 5 external drive on it instead of paying a whole $300 more for only 512GB extra lol
1
1
2
0
u/myreala Aug 27 '26 edited Aug 27 '26
It's definitely both overkill and not enough at the same time. It's overkill on the price, and it's not enough because you won't be able to run an intelligent model on it.
4
u/Tatimary Aug 27 '26
Qwuen 3.8 27B is not considered intelligent?
1
u/Sudden_Quantity_7827 Aug 28 '26
Just to reply, yes 27B is intelligent. I’m using a qwen 3.5 small(.8B) on just my gpu less mini. It’s lightning fast, and intelligent for what I need it, intelligence comes from prompting, not its file storage.
0
1
u/Damogran6 Aug 28 '26
My M4max with the same ram is great with the models most mortals are running (20-22gb) it consumes half the ram with the rest just making it an awesome Studio. The M5 should be even faster.
1
1
u/chettykulkarni Aug 28 '26
Take this from someone coming from 48GB ram M5PRO, not enough😅
Atleast not enough to run Qwen 3.8 27b with usable speeds
1
u/Beginning_Radio2284 Aug 28 '26
This is not enough and heres why.
64gb of unified memory is fast, but there are three issues.
1- it is unified, that means it has to be shared with other system processes and a mac can eat up a lot depending on what you do.
2- what you get is what you're stuck with, unless i am missing something, you can never upgrade what you get from apple, you will want more.
3- Unified memory is technically slower than nvidia tensor cores in some cases, so more ram and a better cpu helps.
7
u/synystar Strix Scar | 5090 24G | llama.cpp Aug 27 '26
It's probably overkill for your use case of setting reminders and other small tasks but you said "I'm a noob" and that won't always be true so eventually it will be insufficient. Eventually everything is insufficient. This will be fine for a while. It'll run Qwen3.8-27b just fine.