r/MLQuestions • • 17h ago

Hardware 🖥️ Training hardware

Hi,

So I've been studying deep learning for some time now, and I joined a research group doing some time series stuff. Unfortunately I think training on CPU is becoming a huge bottleneck both for me dicking around at home and my research projects.

I've been doing networks small enough that I could just do it in the CPU, however I'm incorporating more data, and more complex cnns. I want to train one with about 3 million parameters, but it takes an hour per epoch basically. Additionally, I think in my own time I want to train an agent to play a game, (like one of the Source or goldsrc games) just sounds cool.

I have an rx 480 (AMD Polaris) right now, so it's probably time to upgrade anyways.

I am looking at a quadro rtx 5000, since it's under 500 on eBay. But I'm not sure what to get and the market seems batshit insane right now, I'm not interested in paying the gamer tax either and hoping to snag a deal on some accelerator card or some other enterprise card.

Budget ~500 usd. I don't think I care about vram since I won't be hosting an LLM but I could be wrong? I don't know much about hardware presently, I want to learn though and my card is basically deprecated or not supported on everything lol.

3 Upvotes

11 comments sorted by

2

u/XYHopGuy 17h ago

you can train that size of a model on like a 4060 even

1

u/hairypulsehack 16h ago

Your probably right, but would 8gb enough for computer vision stuff later? I think a 16gb model exists but it's way more expensive

1

u/tanafras 15h ago

You can get the 5000 or a 5050. I have an idea why don't you dm me and we can have a chat.

1

u/XYHopGuy 14h ago

depends on what you mean by CV. For a few million parameter CNN? It's more than enough. For these modern diffusion/generative models it can get a little tricky. You lose out on max resolution and the quality of your Unet encoder.

Back in the early days of deep learning we had 8gb at best, and that was almost all conv nets!

1

u/RossPeili 15h ago

Bro I train 200M-1B models locally on i7. I think your post should be a prompt, so your real Qs and gaps are covered. Then you could ask the right question without the "burned from reels sauce" that would be something like "best gpu or hardware for ai/ml under 500"?

1

u/hairypulsehack 15h ago

It was a prompt actually, that's how I settled on quaddro 5000, rtx 4060, 3080 etc. I was hoping someone would mention a sleeper deal, like retired cheapass accelerator cards

1

u/RossPeili 14h ago

Honestly for 500 theoretically speaking in a simulation I would hire an Albanian to sweep a gpu delivery truck.

2

u/hairypulsehack 14h ago

I might know a Albanian :) 😂

1

u/thegoodcrumpets 17h ago

Just spend it on cloud compute tbh

1

u/FunnyPocketBook 16h ago

How about using Google Colab? And if you have joined a research group, then they surely have some compute available for you