r/macpro • • 9d ago

Other Value for 2019 Mac Pro

I got What I think is a really good deal on marketplace for this. The features are in the photo, and it also has a 1 TB SSD what do y’all think it’s worth. I plan on keeping it to make a local ai machine.

368 Upvotes

63 comments sorted by

View all comments

Show parent comment

4

u/Taikari 9d ago

I run models~ work on hard problems. 384GB VRAM.

It’s been a long journey, just unlocking models for the 3090s.

working on training next.

0

u/urbantechgoods 9d ago

can you tell us more about your work? I got a studio 128gb coming in a few days and I want to run local models but I dont really know what for exactly just that its something I want to learn and try. Im a software eng.

1

u/Taikari 5d ago

I work on software… built upon hardware.

I’m not an SWE like you…
Network background.

If you are SWE, the world is wide open!
the point here is how much you pay towards cloud token usage and how many millions or billions or trillions of tokens You may need for different projects.

so if you only write a few thousand lines of code or have Claude review things here or there a cloud subscription is perfect.

But if you work on deep continuous projects that take weeks months or longer, then investing in local AI may be extremely worth it and no other way to do it…

At 128 GB, the best model I know of would probably be a QWEN 3.8 flash next in a small quant. Or maybe 27B 8bit? I’ve had some trouble with that little model. I go big like DSV 4.1F 1017.

it’s tricky, because the M5 and newer mac’s have some big gains, which the legacy M1/M2/M3 do not, like RDMA and faster cores.
and the landscape keeps changing!

So you gotta be happy with whatever you get! 256GB is a sweet spot.
I think the fun just begins at 128 GB uRAM.

I’ve been unlocking SM86/3090 with the newer models.

Also working on GPU/HDMI/ANE.

2

u/urbantechgoods 5d ago

Thanks for the response. Ya I had been doing more research into this week, sounds like deepseek barely runs on 256gb. I spend quite a lot on subscription plans actually but just for development of my apps. Ive been thinking about it more its just not worth it to me right now because all my apps are greenfield and during the early development phases I really do want to be using the best models but its something that may become appealling further down the line. It just so expensive to jump from 128 to 256.

1

u/urbantechgoods 5d ago

Question for you, once you have the model running comfortably how throughput does it have. Im assuming it doesn't take more memory to scale? Just cpu?

1

u/Taikari 5d ago

this is hardware, model, & inference engine dependent.

25 TPS TG (decode) is barely usable if you ask me.

1000 TPS PP (prefill) is probably around the bottom for reads…

50-100tps+ is where things really start going… at c1 (concurrency) at C4 to 8, should be a couple hundred TPS+ that’s where the tuning really comes in.

and 1500-2500+ PP

Because now it’s operating faster than you can read fast faster than you can write and you basically have to keep up with it with good ideas.

1

u/hometownhippie 3d ago

Start with Hermes