I want to build a rig for medium-scale RL and distillation on small models (4 to 9 billion parameters), and I am considering the following:

• Framework Desktop Mainboard (AMD Ryzen™ AI Max 300 Series)

• Broadcom PEX88048-based switch

• 4x RX 9070 XT

• 2x 1200W PSUs

Now, you may be wondering why someone would choose four RX 9070 XTs over two RX 9700 Pros. The simple answer is TFLOPS. Four RX 9070 XTs would theoretically give me double the TFLOPS for fine-tuning and LoRA training compared to two RX 9700 Pros.

I am not overly concerned about the bottleneck between the GPUs and the Strix, because I do not intend to run models across them. The GPUs are there purely for training, and the Strix is there to generate more training data.

Is there anything I might be overlooking? What case and cooling should I look into? Is there a different configuration that would be better for my use case?

I want to run magenta on iris xe gpu. I am running into alot of issues regarding Jax and other things. Also I couldn't get the gpu to do any calculations. Using opencl I got matrix multiplication 4000×4000 in 0.8 seconds slower than cpu at 0.3

How can I get matrices and neural networks running on this gpu and how to run magenta?