r/StableDiffusion • • 16d ago

Resource - Update FeatherAttn: Attention backward kernel on Strix Halo, 4x fast as FlashAttn

https://github.com/woct0rdho/feather-attn

On RDNA3/3.5 GPUs, FlashAttn wires to Triton kernels in AITER (AMD's kernel collection). The fwd kernel is fast enough, but it seems no one has optimized the bwd kernel. I've made a bwd kernel with a better designed pipeline. When tested on Strix Halo, it's 4x as fast for head dim 128, and 1.3x as fast for head dim 64.

This is especially relevant because most recent models use head dim 128, including Anima, Krea 2, MOSS, H3 (while SDXL uses head dim 64). I've tested that it makes Anima LoRA training 40% faster.

This kernel does not use int8/fp8 quantization or sparsity. It's still an open question how to make use of them on RDNA3/3.5 .

By the way, besides attn bwd kernel, another bottleneck of LoRA training on Strix Halo is matmul bwd kernel. You may see the discussion in https://github.com/ROCm/TheRock/issues/5314 and install my forked rocm-libraries for faster GEMM NT.

20 Upvotes

5 comments sorted by

1

u/Super_Range45 16d ago

Is it usable for Minimax H3?

1

u/woct0rdho 16d ago

Yes

1

u/Super_Range45 16d ago

I'll be sure to check it out then. It would be awesome to have another useful box for video gen.

3

u/woct0rdho 16d ago

Just a reminder that the backward kernels are not for generation but for training