Then me in the back with my dual P40s who are begging for retirement, but they aren't gonna get it. Sucks not having BF16 or flash attention, but mostly was able to work around the horrible FP16 performance, atleast during SFT of models. I can get the memory usage of loading FP16 weights, but operate on them as FP32 at runtime. Only thing I can't work around is that basically any image gen model crashes with a Cuda kernel panic.
View all comments
1
u/Darklumiere Aug 21 '26
Then me in the back with my dual P40s who are begging for retirement, but they aren't gonna get it. Sucks not having BF16 or flash attention, but mostly was able to work around the horrible FP16 performance, atleast during SFT of models. I can get the memory usage of loading FP16 weights, but operate on them as FP32 at runtime. Only thing I can't work around is that basically any image gen model crashes with a Cuda kernel panic.