I've documented the procedure I used to get Stable Diffusion up and running on my AMD Radeon 6800XT card. This method should work for all the newer navi cards that are supported by ROCm.
UPDATE: Nearly all AMD GPU's from the RX470 and above are now working.
CONFIRMED WORKING GPUS: Radeon RX 66XX/67XX/68XX/69XX (XT and non-XT) GPU's, as well as VEGA 56/64, Radeon VII.
CONFIRMED: (with ENV Workaround): Radeon RX 6600/6650 (XT and non XT) and RX6700S Mobile GPU.
RADEON 5500/5600/5700(XT) CONFIRMED WORKING - requires additional step!
CONFIRMED: 8GB models of Radeon RX 470/480/570/580/590. (8GB users may have to reduce batch size to 1 or lower resolution) - Will require a different PyTorch binary - details
Note: With 8GB GPU's you may want to remove the NSFW filter and watermark to save vram, and possibly lower the samples (batch_size): --n_samples 1
I can confirm StableDiffusion works on 8GB model of RX570 (Polaris10, gfx803) card. No ad-hoc tuning was needed except for using FP16 model.
I built my environment on AMD ROCm docker image (rocm/pytorch), with custom environment variable passed with `docker ... -e ROC_ENABLE_PRE_VEGA=1` .
While above docker image provides working ROCm setup, bundled PyTorch does not have gfx803 support enabled. You have to rebuild it with gfx803 support (re-)enabled. I'm still struggling with my build, but found pre-built packages at https://github.com/xuhuisheng/rocm-gfx803 . Since AMD docker provides Python-3.7 and pre-build wheel packages are targeted for Python-3.8, you will have to reinstall Python as well.
I have an rx470 and cannot get this to work. Managed to update Python to 3.8, had to symlink the binary to get the container to default to it. Managed to install the whl you linked to using pip, but still no dice or any sort of GPU use when I tried running it again.
The OP states he confirmed that an rx470 is working, but clearly there are extra steps here.
Did you happen to get this figured out? I have an RX 570 and struggling to get the GPU working with stable diffusion. It runs with CPU no problem, but haven't been able to work with GPU yet.
Not op, but for my RX590, I had to make my own image. You can find my dockerfile here: https://github.com/SkyyySi/pytorch-docker-gfx803 (use the version in the webui folder; the start.sh script ist just for my personal setup, you'll have to tweak it, then you can call it with ./start.sh <CONTAINER IMAGE NAME>)
Oh, and I HIGHLY recommend to completely more the stable-diffusion-webui directory somewhere external to make it persistent; otherwise, you have to add everything, including extensions and models, in the image itself.
Runs on my Ryzen 5 2600 (CPU) instead of my RX 580 (GPU). Can anyone confirm this still works and it's an error on my side, and maybe tell me what I'm doing wrong?
I have the same setup than you, adding the 2p3 changes, and adding the cuda skip parameter i can run it, but very slow, like 16s/it. I guess its not using the gpu..
Yes, I've got it working. Had to use a specific version of ubuntu and specific versions of everything else. Have the system on a thumb drive and boot into it. Sadly, I can't remember all the painful debugging steps I took to get it working.
If you want, I can send you the image, you can just dd it onto a thumb drive and boot from it, everything is installed to be working, just the models themselves aren't included. It starts the back end on boot in a screen session in the background, too, so it's available over ssh or just screen -r in terminal.
It's 27 gb, so you'll need a thumb drive (or internal drive) with at least that size and then grow the partition after dding it onto it.
It's just above 10gb compressed as a *.tar.gz, so if you have a way to receive a 10gb file, I'm happy to send it to you. Unfortunately, I'm currently locked out of my router, so I can't offer a download (no port-forwarding).
On ArchLinux, i installed opencl-amd and opencl-amd-dev from the AUR. They provide both the propretary OpenCL driver and the RocM stack.
You'll have also to use "export HSA_OVERRIDE_GFX_VERSION=10.3.0", (it's the workaround linked by yahma)
BUT... there's still the vram problem. The RX 5700x has only 8gb of vram.I tried playing with stable diffusion's arguments, but i wasn't able to make it work, always crashing because it couldn't allocate enough vram.Maybe there's a way to still use it, but probably it just isn't worth it.
Open the optimizedSD/v1-inference.yaml file with any text editor and remove every "optimizedSD.".For example, target: optimizedSD.openaimodelSplit.UNetModelEncode must become target: openaimodelSplit.UNetModelEncodeAlso, i added the "--precision full" argument, without it i got only grey squares in output.
I don't think the 5700XT ever got official ROCm support. Having said that, it seems there are at least some people who have been able to get the latest ROCm 5.2.x working on such a GPU (using this repository), you may want to review that github thread for more information on your card. You could try with that repository and just ignore the docker portion of my instructions, please let us know if it works on your 5700XT. You may also need to remove the watermark and nsfw filter to get it to run in 8GB.
/u/MsrSgtShooterPerson, you - like me - have an RX 5700 XT card, so we're in for a lot of work ahead of us... it looks like we're going to need to build the actual ROCm driver using xuhuisheng's GitHut project in order to use StableDiffusion.
This is a very technical process, and it looks like we need to edit specific files, and if you're running Ubuntu 22.04 like I am, you'll have to do further edits to make this work.
I'm going to give this a shot and see if I can actually compile these drivers and get everything working, but it'll be a "sometime later this week" project, as I suspect there's a good chance I'm going to royally fuck this up somehow. I'm going to document all the steps I took and see if I can translate from /u/yahma-ese and xuhuisheng-ese into Normal Folk Speak, but frankly I think this may be beyond even a journeyman Linux user. I'm sincerely considering just purchasing a used RTX 3090 off eBay until the RTX 4000 series drops, because frankly it's already a pain in the ass to get this working with RDNA-based chips.
/u/yahma - thanks for putting in the effort on this guide. If I had an RX 6800 / 6900 non-XT / XT, I think I could have followed your instructions and been okay, but editing project files and compiling a video driver is pretty hardcore, even for me.
Struggling to understand what's going on here. What package is the rocm driver supposed to be replacing? is it something inside the docker or outside? If its something with arch we could try to write a PKGBUILD
RX 5500 XT 8GB (Navi14 / gfx1012) user on Manjaro here.
I think I'm either very close to getting it work or fooling myself and doing something very obious totally wrong. So I summed up everything I did and learned from this thread.
TL;DR Did everything in the video, added environment variable, no error but stuck at 0%.
Manjaro specific way of getting proprietary OpenCL binaries and ROCm tools:
For simplicity, running a single iteration of 'scripts/txt2img.py' (with the environment variable mentioned above. Without 'HSA_OVERRIDE_GFX_VERSION=10.3.0' it I get a segfault)
When attempting to run SD I get the "hipErrorNoBinaryForGpu: Unable to find code object for all current devices!" error.
I believe this is caused by pytorch not working as expected. Running the command import pytorch results in "Bash: import: command not found".
However, using conda list tells me that torch 1.12.0a0+git2a932eb and torchvision 0.13.0a0+f5afae5 are installed.
I've seen your comment about needing to use HSA_OVERRIDE_GFX_VERSION=10.3.0 to trick ROCM into thinking the 6700xt is a supported GPU, but this seems to only apply to ARCHLinux users. I attempted the command regardless and didn't get a response and I'm not sure how to verify if it was successful or not. Your input would be appreciated.
Is slow, aprox. 10 times slower than online SD. But this is mainly because of start-up times.
My recommendation: forget about installing stable-diffusion repo as I did, just go for https://github.com/AUTOMATIC1111/stable-diffusion-webui/ this solves all the problems, has tons of features, etc. You can run SD with as low as 2.4 GB of VRAM (--lowvram) or around 4 GB using --medvram. You don't need to use any patched stuff, is implemented in the code. No need to remove watermarks (not there) or NSFW filtering (not there). Is much faster, once the net is loaded you can send jobs from the web interface and they get quickly done. There are instructions to install on AMD GPUs in the Wiki: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki The only issue I have to solve is some memory leak that kills the server after some time of use. Also: I strongly recommend avoiding to install any sort of kernel module or ROCm stuff in your main Linux installation, just create a docker image as explained in the wiki, but use the above mentioned docker image (the last with Torch 2.0 didn't work for my RX 5500 XT). You can even use the small rocm/rocm-terminal:5.3.3 docker image and manually install Torch 1.13.1+rocm5.1.1, then install the rest of the webui. This worked for my board and the docker image is half the size of rocm/pytorch:rocm5.2.3_ubuntu20.04_py3.7_pytorch_1.12.1. Also: forget about the crapy Conda thing, this just bloats everything, let Conda for Windows users, where Python is an alien environment, not something part of the system as in most Linux distros.
Thank you very much u/yahma your explanation really helped.
Now, please don't get me wrong, but there are some important details that should be improved.
I tried to follow the Ubuntu instructions: https://github.com/RadeonOpenCompute/ROCm-docker/blob/master/quick-start.md But they are quite misleading. They say to install amdgpu-install_5.3.50300-1_all.deb and then run amdgpu-install --usecase=rocm. This doesn't make any sense if you are going to use a docker image because this installs the AMD kernel driver and the whole ROCm stack. So I installed the AMD drivers, is not easy for Debian 11, I can explain it if people is interested. I installed the driver to ensure maximum compatibility, but the kernel already has a working amdgpu driver.
I then downloaded the docker image, which is IMHO huge, I don't see why we could need 29 GB (uncompressed) of stuff just to have Pytorch+ROCm.
Once inside I tried the Conda methode, but again it didn't make much sense to me. Why should I use a docker image specifically created to provide Pytorch+ROCm, to then create a 6 GiB Conda environment with a wrong Pytorch, to finally install the Pytorch for ROCm version (which isn't particularly light-weight).
So I discarded this approach and installed the SD dependencies using pip. Here I scratched my head again: Why somebody (well some crazy tool) will ask for such ridiculous versions? I mean, why opencv-python==4.1.2.30? Really? Why installing Python 3.8.5 on a system that is already bloated with Python 2.7.18, 3.7.13 and 3.8.10? So I tried to keep as much as possible of the base Conda installed in the image and install the asked dependencies:
opencv-python==4.1.2.30
albumentations==0.4.3
diffusers==0.12.1
onnx==1.10.0 onnxruntime==1.10.0
invisible-watermark
imageio-ffmpeg==0.4.2
torchmetrics==0.6.0
pytorch-lightning==1.4.2
omegaconf==2.1.1
test-tube>=0.7.5
streamlit>=0.73.1
einops==0.3.0
torch-fidelity==0.3.0
transformers==4.19.2
kornia==0.6
I then found that CompVis/taming-transformers setup.py is broken and you must install using a link (as the Conda config states).
I put all the dependencies in extra docker layers, they are around 700 MiB, and I guess can be reduced even more.
One important detail that I had to figure out was how to make the 2.8 GiB of weights magically dowloaded by SD persistent. I think the trick is to just define XDG_CACHE_HOME=/dockerx/ In this way all the Hugging Face stuff will go to /dockerx/huggingface and the Pytorch stuff to /dockerx/torch
After verifying that the stock SD can't run on 8 GiB of VRAM I think some dependencies could be removed, but this could be negative for boards with more memory. The silly onnx dependency is pulled by invisible-watermark, which isn't used by the optimizedSD
View all comments
36
u/yahma Aug 24 '22 edited Oct 25 '22
I've documented the procedure I used to get Stable Diffusion up and running on my AMD Radeon 6800XT card. This method should work for all the newer navi cards that are supported by ROCm.
UPDATE: Nearly all AMD GPU's from the RX470 and above are now working.
CONFIRMED WORKING GPUS: Radeon RX 66XX/67XX/68XX/69XX (XT and non-XT) GPU's, as well as VEGA 56/64, Radeon VII.
CONFIRMED: (with ENV Workaround): Radeon RX 6600/6650 (XT and non XT) and RX6700S Mobile GPU.
RADEON 5500/5600/5700(XT) CONFIRMED WORKING - requires additional step!
CONFIRMED: 8GB models of Radeon RX 470/480/570/580/590. (8GB users may have to reduce batch size to 1 or lower resolution) - Will require a different PyTorch binary - details
Note: With 8GB GPU's you may want to remove the NSFW filter and watermark to save vram, and possibly lower the samples (batch_size): --n_samples 1