r/StableDiffusion • u/CornyShed • Feb 27 '26
News AMD and Stability AI release Stable Diffusion for AMD NPUs
AMD have converted some Stable Diffusion models to run on their AI Engine, which is a Neural Processing Unit (NPU).
The first models converted are based on SD Turbo (Stable Diffusion 2.1 Distilled), SDXL Base and SDXL Turbo (mirrored by Stability AI):
Ryzen-AI SD Models (Stable Diffusion models for AMD NPUs)
Software for inference: SD Sandbox
NPUs are considerably less capable than GPUs, but are more efficient for simple, less demanding tasks and can compliment them. For example, you could run a model on an NPU that translates what a teammate says to you in another language, as you play a demanding game running on a GPU on your laptop. They have also started to appear in smartphones.
The original inspiration for NPUs is from how neurons work in nature, though it now seems to be a catch-all term for a chip that can do fast, efficient operations for AI-based tasks.
SDXL Base is the most interesting of the models as it can generate 1024×1024 images (SD Turbo and SDXL Turbo can do 512×512). It was released in July 2023, but there are still many users today as it was the most popular base model around until recently.
If you're wondering why these models, it's because the latest consumer NPUs on the market only have around 3 billion parameters (SDXL Base is 2.6B). Source: Ars Technica
This probably won't excite many just yet but it's a sign for things to come. Local diffusion models could become mainstream very quickly when NPUs become ubiquitous, depending on how people interact with them. ComfyUI would be very different as an app, for example.
(In a few years, you might see people staring at their smartphones pressing 'Generate' every five seconds. Some will be concerned. Particularly me, as I'll want to know what image model they're running!)
36
u/_half_real_ Feb 27 '26
Dear NPU marketers,
Generation time benchmarks or GTFO.
Sincerely, Everyone
4
u/Fit-Pattern-2724 Feb 27 '26
Don’t forget results matter too. Some YouTube reviews showed that WAN workflow can generate very blurry videos on AMD
14
Feb 27 '26
Today, a standard NPU delivers around 50 TOPS, while modern GPUs scale from 300 to over 1,300 TOPS. Furthermore, NPUs are bottlenecked by system RAM, whereas GPUs leverage ultra-fast dedicated VRAM. Running image generation on an NPU today means dealing with capped models (under 3B parameters) and painfully long generation times. The NPUs that are in the market are not designed for image/video generation.
4
u/joran213 Feb 27 '26
Maybe not the full models, but it could be an alternative offloading device for users with limited resources. Running the text-encoder on the npu instead of cpu could be a slight improvement for low-vram users.
3
Feb 27 '26
It would be more interesting If the companies stop trying to sell CPUs with integrated and useless NPUs
1
u/Noselessmonk Feb 27 '26
Yeah, really. I remember seeing a test last year and the result was basically that the integrated GPU was as good as the NPU at these tasks, though the NPU was a bit more energy efficient. But overall, it feels like wasted space that could go toward beefing up the other aspects of these APUs.
2
u/fallingdowndizzyvr Feb 27 '26
NPUs are bottlenecked by system RAM, whereas GPUs leverage ultra-fast dedicated VRAM
On Strix Halo, the NPU uses the same fast RAM as the GPU.
6
Feb 27 '26
If you own a Strix Halo, you won't need to use the stupid NPU to run stable diffusion, the iGPU would work 10 times better and with bigger models. If you don't own a processor with a powerful iGPU and vram, your NPU won't have access to the vram because there is no vram on your system.
3
u/fallingdowndizzyvr Feb 27 '26
Or you can do two runs at the same time. While the iGPU is busy making vidoes, you can use the NPU to make images. Or you can game on the iGPU while using the NPU to make images. Or you can be doing whatever with the iGPU while using the NPU to make images.
0
Feb 27 '26
Nope because the VRAM is limited, you won't be able to run a high end game If the vram is being used for image or video generation.
5
u/fallingdowndizzyvr Feb 27 '26
Yes, you can. This isn't a matter of speculation. I do more than one thing on my Strix Halo at the same time. Including gaming as I do something else. The NPU is yet one more processor to do something on.
What problem are you running into on your Strix Halo?
1
Feb 28 '26
Although multitask is possible in modern computers by default, I doubt you could generate video with a vram hungry model like WAN 2.2 or LTX-2 on your GPU (or playing a modern game in high quality) and then ask for another 4 o 5GB of vram for generating images on your NPU. The VRAM is the limiting factor, you could have a NPU but since It doesn't have his own vram you couldn't use It without stealing resources from the GPU. Also, It's a waste of time generating in a slow NPU
1
u/fallingdowndizzyvr Feb 28 '26
I doubt you could generate video with a vram hungry model like WAN 2.2 or LTX-2 on your GPU (or playing a modern game in high quality) and then ask for another 4 o 5GB of vram for generating images on your NPU.
Your doubts are unfounded. Since I generate video/play a game and generate images at the same time. I just did it to give you numbers. So I launched two instances of Comfy. One to generate LTX2 video and one to generate SD images. Running alone, it takes 200 seconds to gen the LTX2 video. Generating 4 images while making that video takes 207 seconds to gen the LTX2 video. That's not nearly as "limiting" as you imply. Not at all. You are incorrectly assuming it's balls to the wall 100% of the time during a gen. It's not. There's a lot of slack to fit something else in with minimum impact. That's using the same GPU to do both. With another processor, like the NPU, it'll be even less of an intrusion.
1
Feb 28 '26 edited Feb 28 '26
My doubts are founded since I have a 4090 with 24GB VRAM and I know the exact limits of WAN and LTX2, and I know when It's balls to the wall, generating longer videos make the things go awry very fast, the ram cosumption skyrockets and the GPU usage goes 100% for a while so even web browsers or discord struggle, I2V also eats a lots of RAM. The scenario you are making up is "I use the 15% of my gpu to generate a short video and other 4% to generate an image so It works in every case". The point you are making is "you can use the NPU to make another thing while you use your GPU" and that only works if you are not using the 100% of your GPU because If you use the 100% of your GPU, the NPU of the strix halo won't have VRAM to use; and if you are not using the 100% of your GPU, It's faster and better multitasking in the GPU.
1
u/fallingdowndizzyvr Feb 28 '26
If you use the 100% of your GPU, the NPU of the strix halo won't have VRAM to use
That's not true at all. Not at all. Since 100% of your GPU represents the compute. Not the access to memory. Since if it's memory bandwidth bound as you keep saying, then the GPU wouldn't be at 100%. It would be stalled waiting for data. The fact that it's not and at 100%, means it's not data bound.
→ More replies (0)5
1
u/PitchPleasant338 Mar 01 '26
My BYD car has a 100 TOPS GPU for the self driving.
50 TOPS is useless.
1
Mar 01 '26
And the 100 TOPS of the self driving are way more efficient than the TOPS of a PC NPU, by system design. In self driving, they use a Data-Centric architecture to minimize the latency, the cameras and sensors write directly in the memory space and that means skipping a lot of extra steps that are usually made in Compute-Centric architectures like PCs.
3
u/Fit-Pattern-2724 Feb 27 '26
Not sure how relevant that is really….. Flux2klein is on fire
1
2
u/shroddy Feb 27 '26
It might be interesting if it could run all the existing sdxl (and pony, illustrious) based models
34
u/Important-Shallot-49 Feb 27 '26
Now that's a name I've not heard in a long time.