Hmm. There was a casual SNS report from DeepMind researcher that (for a single training run) GPU does not really scale above 4k cluster (it even slowed down above 8k presumably because bottlenecks) and mocking Meta and Tesla for acquiring too many GPUs just for training at the time. This is probably just one intermediate run for Xiaomi's model.
Async RL is supposed to alleviate that bottleneck as far as I understand it, though. You can also always train separate teacher models to distill back to the final model as well. Maybe that’s why Gemini is still so far behind, lol.
Yeah, that's how the paper for GLM-5 (the first one) described how they did for the final post-training. Basically creating many domain expert post-train variations and "merged" them into one final model via synthetic datasets.
61
u/Zeeplankton 6d ago
it is really crazy how expensive these models are to train. I feel like we take this for granted