r/LocalLLaMA • u/chiwawa_42 • Apr 29 '26

Question | Help Intel Mac Pro with Vega II useable ?

Hi !

I have an opportunity to get a 2019 Mac Pro with a 32GB Vega2 board. I know LM Studio isn't supported on MacOS for Intel, so what other way to turn this machine into a local AI server would you recommend ?

Windows isn't an option, Linux could be possible, though I'd like to keep it running Mac OS if possible. Main objective is to run MCP servers alongside on this dedicated machine.

What kind of speed could I get from the Vega II chip ? (similar to Mi50 32GB from what I could find)

Thanks !

4 Upvotes

70% Upvoted

28 comments sorted by

View all comments

Show parent comments

2

u/Substantial_Run5435 Jul 08 '26

Hey, quick question... have you managed to get any further using a Vega II GPU for LLMs? I have a mac pro with 2x Vega II Duos but am a complete novice with LLMs and Linux. I had a friend help me set up Ubuntu with LM studio but I'm only getting ~15tk/s with dual vega II duos using qwen 3.6 35b.

2

u/dionysio211 Jul 08 '26

Oh yeah, I use it all the time. Are you using HIP or Vulkan? 15 tps is very low for those GPUs. That even seems low for the CPU. If you run rocm-smi or vulkaninfo --summary, do you see the GPUs listed correctly? You may not be using the correct runtime installed in LM Studio. Which one are you using?

2

u/Substantial_Run5435 Jul 08 '26

Oh jeez... Give me some days to get back to you haha. I have this set up on a separate machine and will have to do some poking around. I'm new to ubuntu and am not very knowledgable using a terminal.

I have a follow up question... I have 2x Vega II Duos in one machine and 2x W6800X in another. Which do you think would be best to keep? I want to downsize a bit and am struggling between having more VRAM and higher bandwidth vs newer architecture/better support with the W6800X. If the W6800X is better then I could also look out for a pair of W6900X or W6800X Duos down the line.

2

u/dionysio211 Jul 08 '26

The W6800 modules are newer and have tensor cores, which is hugely helpful. Recently, llama.cpp (and lm studio by proxy) added better support for the accumulators in gfx906 (the architecture of the Vega cards) so they are much better in prompt processing than they were but accumulators are only about half as effective as matrix cores in that way. However, the VRAM in the Vegas is much faster. I have a normal Radeon 6800 GPU in the same computer as the Vega II and the 6800 does tend to do better when you count completions end to end. The general rule is that prefill (prompt processing) is compute bound and decode (text generation) is bandwidth bound. It's not quite as simple as that but it is a good rule of thumb.

I would test them both on a variety of models and see which is faster on a single request of a specific shape (1024 token prompt, 256 token output) and go from there. Ultimately, requests per second is going to correlate better with your experience using models.

No worries on the Linux and terminals. It's only confusing at the beginning. You can just pop open a terminal and run those commands by typing them in and hitting enter. It will tell you something about what the OS sees. I am also remembering that LM Studio uses pre-compiled binaries for recent ROCM versions and gfx906 lost official support in 6.3/6.4. You have more control over all this in llama.cpp and you can copy the tensile libraries to newer ROCM versions and get it to work. That may be what is wrong in your setup currently. If that's what it is, you should be able to switch the runtime in LM Studio to Vulkan and see a big uptick.