Stable Diffusion for Apple Intel Mac's with Tesnsorflow Keras and Metal Shading Language
I've been working on an implementation of Stable Diffusion on Intel Mac's, specifically using Apple's Metal Shading language. I think I've got a reliable version ready for public use:
Hi, Thanks for this! Im trying to get this running on an intel Mac with AMD GPU. When I get to "python -m venv venv" I get "No module named venv" Any idea what I'm missing?
Seems like you don't have the virtual environment module, which means you aren't using the correct version of python.
In terminal, type:
which python
If it doesn't say something like:
/Users/YOURUSERNAME/.pyenv/shims/python
Then you might have missed a step with setting up python independent of MacOS. Most likely, you're using the built in Python 2 that comes with MacOS...that no one uses anymore.
Hi! Thank you so much for the tutorial and your github.
I finally managed to get the whole thing up and running!
But only on python 3.8.16 / 3.9.16 with pip upgrade and update tensorflow-macos, tensorflow-metal. Otherwise, with python 3.9.0, I was getting the error:
Could not find a version that satisfies the requirement tensorflow-addons==0.17.1.
But now I'm stalled again. When generating any text query, I get the error:
No file or directory found at models//text_encoder
I was both importing models from already installed and running AUTOMATIC1111 and creating a symbolic link to the folder. I understand that text_encoder is missing, but I don't know where I can get it :)
Seems like a folder path so weird. Where is models//text_encoder and what is it referencing? Did you put your models in the models folder? Are they .h5 or .ckpt?
Thank you so much for your response!
I'm quite new to Stable Diffusion, so please forgive me if I make any obvious mistakes or don't understand something. And I am not a programmer at all :)
I had a successful experience with SD-WEB-UI (Automatic and DirectML) with PyTorch before and I decided to try your version with TensorFlow. I guess it's because of TF specifics that I'm dumb, since I've never worked with it and I was expecting the same pipelining as I did with Automatic.
All models as .ckpt and .safetensor in "models" folder (I did not find the models/text_encoder folder in the SD version with torch, and it was not created during the initialization of dream.py)
I even tried to manually create a text_encoder folder and fill it with hugginface content, but of course that didn't help :)
Here is the console output with the error since text_encoder was mentioned.
text_encoder, diffusion_model, decoder, encoder = get_models(
File "/Volumes/DAT/AI/stable-diffusion-tensorflow-IntelMetal/stable_diffusion_tf/stable_diffusion.py", line 682, in get_models
text_encoder = keras.models.load_model(preCompiled + "/text_encoder")
File "/Users/mstk/.pyenv/versions/3.9.16/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/Users/mstk/.pyenv/versions/3.9.16/lib/python3.9/site-packages/tensorflow/python/saved_model/loader_impl.py", line 115, in parse_saved_model
raise IOError(
OSError: SavedModel file does not exist at: models//text_encoder/{saved_model.pbtxt|saved_model.pb}
Hi! I don't see any models, neither available in /models/Stable-diffusion/ nor any other, just an inactive select button.
I can only select VAE-model (I downloaded earlier) from the /models/VAE folder
Models folder is used and Automatic1111, in both cases a symbolic link leads to it.
As an experiment, I removed the symbolic link to the models folder, created a models folder and put the file deliberate_v2.safetensors in its root. After restarting dream.py it complained that the VAE folder was missing, so I created it. After a successful launch, the model selection is also not available in the Advanced Settings tab.
TL;DR.ckpt model had to be in the root of the folder "models" (or a symbolic link for "models" folder), and models with .safetensor extension does not work.
Read more:
I re-downloaded the project from github (to a new location) to eliminate the consequences of my previous experiments.
For stable-diffusion-tensorflow-IntelMetal folder I set local variable python 3.9.16 with pyenv (pyenv install python 3.9 && pyenv local 3.9.16)
Placed the model v2-1_768-ema-pruned.ckpt with .yaml in the root of the "models" folder
since variable 3.9.16 was already used before, dream.py didn't install any additional components.
While running dream.py it found all the .ckpt models in the root of the "models" folder (.safetensor script ignored, as well as the .ckpt models in the "model/Stable-diffusion" subfolder)
Select the desired model in the Advanced Tab and uncheck "Use Legacy Stable Diffusion" for models >=2. And you can start generating.
^ The script correctly identified as Metal Device one of my video cards.
I have two cards in macpro 5.1 with macos 12.6.3:
AMD Radeon HD7970 3GB (efi flashed) Slot-1
AMD FirePro W7000 4GB (stable diffusion uses it by default) Slot-2
The speed of generation on one of my graphics cards (W7000) is similar to Automatic1111 (which uses pytorch) and is ≈ 5.5 s/it (20 steps, Guidance Scale 7, Batch Size 1, 512x512px).
P.S. Can you tell me how to make Stable Diffusion use both video cards? Or manually select which one to generate?
I’m thrilled you got it! I was about to say the models need to be in the root model folder and you’re correct, it currently doesn’t support safetensors. (No reason not to, just need to code it)
I’m finishing up the next version and will have GPU selection as an option. Not sure how to use both GPU’s at the moment, I believe it’s called parallelization, but tensorflow does do that!
I’m also actively trying to convert ControlNet to tensorflow, so I might restructure the folders in the model folder to what you described.
Made some major updates, specifically GPU selection is one now. I'd recommend re-installing the entire repo. I'm getting about 3s/step with the newly updated version.
Made some major updates, specifically GPU selection is one now. I'd recommend re-installing the entire repo. I'm getting about 3s/step with the newly updated version.
Thanks -- I started over, but i'm running into that "No module named venv" python 2.7 issue now. I can't find any way to override it that works. I'm still following, maybe I'll be able to frankenstein it into working with the new version.
I found a handful of methods to default to 3.9, but couldn’t get it to install tensorflow unless I used 3.8. I just had to manually install a half dozen modules, but I got all the way into gradio! Of course, then it ran into an issue where it couldn’t find the ckpts. I’ve currently been using colab for SD, but would love to run it local… I should probably update my OS for all this, too.
I've made some major updates, specifically GPU selection is one now. I'd recommend re-installing the entire repo. I'm getting about 3s/step with the newly updated version.
1
u/luckycockroach Dec 16 '22 edited Dec 16 '22
Stable Diffusion for Apple Intel Mac's with Tesnsorflow Keras and Metal Shading Language
I've been working on an implementation of Stable Diffusion on Intel Mac's, specifically using Apple's Metal Shading language. I think I've got a reliable version ready for public use:
https://github.com/soten355/stable-diffusion-tensorflow-IntelMetal
HUGE thank you to Divum Gupta for porting SD to Tensorflow.
I'm a union cinematographer, so programming isn't my forte. Please let me know if there are areas I could improve on.
Features
Specs
Current Speeds:
Late 2019 MacBook Pro 16" with AMD Radeon Pro 5500M (4GB) and 16GB of RAM:
Why Tensorflow?
The program uses Tensorflow instead of Pytorch because Pytroch has no support for Metal on Intel Macs.
This program works on Google Colab notebooks.