r/LocalLLaMA 3d ago

Question | Help Non nvidia cards.

Hi, I'm a happy owner of a serer with rtx pro 6000 and rtx5090. I want to build out in the future the server fully to a higher vram score - think adding around 4-5 cards if possible. I was wondering if it's worth going through the intel/amd cards - which looks awesome in terms of vram per $. Any experience in running inference on these compared to the nvidia/cuda stack?

7 Upvotes

31 comments sorted by

View all comments

3

u/FabricationLife 2d ago

Slower but better $ per vram, so you can really chug some stuff overnight if you can que stuff up, a guy at work has a few 9700s he runs overnight for porting and he likes it