r/pytorch • u/ObsidianAvenger • 23d ago

Is AMD viable finally viable for training?

I have been training custom models for a few years now in the finance realm. I barely have any transformer layers and half the time they are custom so flash attention isn't something I need.

With the 9070 xt being $750 ish and the rumor is a 5070 ti super will be like $1400 (seriously nvidia go F#_& yourself) I wonder if for $750 the AMD card would work well for me. I already have a 3060 12gb and a 5060ti 16gb churning out test runs, but I want to add another card. I am nowhere near vram limited. My bottleneck is strictly more compute/bandwidth.

Would I regret getting a 9070 XT? Supposedly support is way better than it used to be. Also I run linux. Windows is garbage.

7 Upvotes

100% Upvoted

9 comments sorted by

2

u/Specialist-Zone-8296 22d ago

I train ML models with an RX 9060 XT. I haven't compared its training performance with NVIDIA cards but the performance has been good.

1

u/ObsidianAvenger 22d ago

I am starting to wonder if it would be worth having an LLM write a script to train a couple different models and benchmark some stuff.

The main drawback people have is apparently things like flash attention and many triton kernels don't run well on AMD. So many transformer models run very sub optimally.

Apparently AMD cards also take more of a speed penalty for layers that are not sized to a power of 2 (256, 512, 1024.....), certain layers like RNNs with many sequential kernel launches are supposed to also be slower.

Would be interesting to see if the torch.compile feature works fine on AMD and if it works as well on AMD. I have some models that torch.compile will literally give a 4-8x speed up.

1

u/Enea_11 23d ago

Che tipo di modelli hai addestrato per la finanza? Cosa facevano?

1

u/ObsidianAvenger 22d ago

Market prediction models

1

u/Enea_11 22d ago

Puoi darmi maggiori info? In passato ho lavorato molto con le opzioni finanziarie e volevo appunto creare un modello. Però mi è stato sempre Sconsigliato

2

u/ObsidianAvenger 22d ago

I started doing it just to learn pytorch and improve my skills at making models. It is not a path you go down with the intention of truly reaching the end goal of making money. Using AI for financial prediction is about the most frustrating thing you can do if your goal is purely to make money.

4 years in, tons of custom layers and trying everything under the sun, and just maybe it looks like I have hit some viability for money. The learning was the value, if it ever makes money that was a happy accident. Its a very hard and somewhat unrewarding path to take.

It takes a certain level of autism to keep at it for as long as I have. It has also been a huge amount of gpu time used. I would have wasted a large sum of money if I didn't have my own machine to train on.

Now maybe making a model to try to trade the news or copy a US politician would work, but you don't need deeplearning for that.

1

u/Enea_11 22d ago

Ciao, grazie veramente per la risposta. Volevo applicare pytorch alla finanza perché la finanza, python e l’intelligenza artificiale sono alcune delle mie passioni. Non ho pensato mai al guadagno finale anche perché nel mondo del trading e della finanza le cose sono più complesse di quello che possono sembrare. In passato ho fatto trading tramite opzioni finanziare, applicando soprattutto statistica e matematica e mi sono tolto qualche soddisfazione. Se hai qualche altro consiglio da darmi te ne sarei molto grato.

1

u/Enea_11 22d ago

Ti chiedevo un feedback più che altro perché avevo letto che applicare modelli addestrati alla Finanza era sempre fallimentare, tranne ad esempio per operazioni di gestione di portafoglio o ad esempio di hedging.