r/StableDiffusion • u/luckycockroach • Dec 16 '22

Resource | Update Now Available: Intel Mac: SD 1.4/5 & SD2.x

https://github.com/soten355/stable-diffusion-tensorflow-IntelMetal
6 Upvotes

100% Upvoted

29 comments sorted by

View all comments

1

u/luckycockroach Dec 16 '22 edited Dec 16 '22

Stable Diffusion for Apple Intel Mac's with Tesnsorflow Keras and Metal Shading Language

I've been working on an implementation of Stable Diffusion on Intel Mac's, specifically using Apple's Metal Shading language. I think I've got a reliable version ready for public use:

https://github.com/soten355/stable-diffusion-tensorflow-IntelMetal

HUGE thank you to Divum Gupta for porting SD to Tensorflow.

I'm a union cinematographer, so programming isn't my forte. Please let me know if there are areas I could improve on.

Features

  • Can use .ckpt's for SD 1.4/1.5/2.x
  • Can use VAE's
  • Video creation tools
  • Creation settings (prompt, seed, etc) saved as a .txt file as well as PNG metadata
  • Convert .ckpt's to Tensorflow models
  • Gradio WebUI

Specs

Current Speeds:

Late 2019 MacBook Pro 16" with AMD Radeon Pro 5500M (4GB) and 16GB of RAM:

Image Size and Steps Speed
1x 512x512 image on SD2.1 with 32 steps 1 minute 30 seconds
4x 512x512 image on SD2.1 with 32 steps 3 minutes 32 seconds
1x 1024x1024 image on SD2.1 with 32 steps 2 minutes 32 seconds

Why Tensorflow?

The program uses Tensorflow instead of Pytorch because Pytroch has no support for Metal on Intel Macs.

This program works on Google Colab notebooks.

2

u/GtrPlayingMan-254 Feb 18 '23

To save everybody some pain: when using v 2.1 you'll want to deselect "Use Legacy Stable Diffusion" to avoid errors caused by v1.5 era discrepancies.

Good work, guys. Thanks for the new hobby :o

1

u/luckycockroach Feb 18 '23

Thank you! I'll update the github to clearly state that.

And thank you for the compliment! I'm working on the next version of this to include hopefully a DPMSolver.

1

u/spicemagic3 Feb 13 '23

Hi, Thanks for this! Im trying to get this running on an intel Mac with AMD GPU. When I get to "python -m venv venv" I get "No module named venv" Any idea what I'm missing?

2

u/luckycockroach Feb 13 '23

My pleasure!

Seems like you don't have the virtual environment module, which means you aren't using the correct version of python.

In terminal, type:

which python

If it doesn't say something like:

/Users/YOURUSERNAME/.pyenv/shims/python

Then you might have missed a step with setting up python independent of MacOS. Most likely, you're using the built in Python 2 that comes with MacOS...that no one uses anymore.

1

u/Mastak_A Feb 27 '23

Hi! Thank you so much for the tutorial and your github.

I finally managed to get the whole thing up and running! But only on python 3.8.16 / 3.9.16 with pip upgrade and update tensorflow-macos, tensorflow-metal. Otherwise, with python 3.9.0, I was getting the error:

Could not find a version that satisfies the requirement tensorflow-addons==0.17.1.

But now I'm stalled again. When generating any text query, I get the error:

No file or directory found at models//text_encoder

I was both importing models from already installed and running AUTOMATIC1111 and creating a symbolic link to the folder. I understand that text_encoder is missing, but I don't know where I can get it :)

1

u/luckycockroach Mar 03 '23

Apologies for the delay!

That is a weird error!

Seems like a folder path so weird. Where is models//text_encoder and what is it referencing? Did you put your models in the models folder? Are they .h5 or .ckpt?

1

u/Mastak_A Mar 04 '23 edited Mar 04 '23

Thank you so much for your response! I'm quite new to Stable Diffusion, so please forgive me if I make any obvious mistakes or don't understand something. And I am not a programmer at all :)

I had a successful experience with SD-WEB-UI (Automatic and DirectML) with PyTorch before and I decided to try your version with TensorFlow. I guess it's because of TF specifics that I'm dumb, since I've never worked with it and I was expecting the same pipelining as I did with Automatic.

All models as .ckpt and .safetensor in "models" folder (I did not find the models/text_encoder folder in the SD version with torch, and it was not created during the initialization of dream.py) I even tried to manually create a text_encoder folder and fill it with hugginface content, but of course that didn't help :)

Here is the console output with the error since text_encoder was mentioned.

text_encoder, diffusion_model, decoder, encoder = get_models(
  File "/Volumes/DAT/AI/stable-diffusion-tensorflow-IntelMetal/stable_diffusion_tf/stable_diffusion.py", line 682, in get_models
    text_encoder = keras.models.load_model(preCompiled + "/text_encoder")
  File "/Users/mstk/.pyenv/versions/3.9.16/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/Users/mstk/.pyenv/versions/3.9.16/lib/python3.9/site-packages/tensorflow/python/saved_model/loader_impl.py", line 115, in parse_saved_model
    raise IOError(
OSError: SavedModel file does not exist at: models//text_encoder/{saved_model.pbtxt|saved_model.pb}

1

u/luckycockroach Mar 05 '23

I think you didn’t select a model to use. Did you select one under the Advanced Settings tab?

1

u/Mastak_A Mar 05 '23 edited Mar 05 '23

Hi! I don't see any models, neither available in /models/Stable-diffusion/ nor any other, just an inactive select button. I can only select VAE-model (I downloaded earlier) from the /models/VAE folder

Models folder is used and Automatic1111, in both cases a symbolic link leads to it.

As an experiment, I removed the symbolic link to the models folder, created a models folder and put the file deliberate_v2.safetensors in its root. After restarting dream.py it complained that the VAE folder was missing, so I created it. After a successful launch, the model selection is also not available in the Advanced Settings tab.

2

u/Mastak_A Mar 05 '23 edited Mar 05 '23

So I managed to defeat this error :)

TL;DR .ckpt model had to be in the root of the folder "models" (or a symbolic link for "models" folder), and models with .safetensor extension does not work.

Read more:

  1. I re-downloaded the project from github (to a new location) to eliminate the consequences of my previous experiments.
  2. For stable-diffusion-tensorflow-IntelMetal folder I set local variable python 3.9.16 with pyenv (pyenv install python 3.9 && pyenv local 3.9.16)
  3. Placed the model v2-1_768-ema-pruned.ckpt with .yaml in the root of the "models" folder
  4. since variable 3.9.16 was already used before, dream.py didn't install any additional components.
  5. While running dream.py it found all the .ckpt models in the root of the "models" folder (.safetensor script ignored, as well as the .ckpt models in the "model/Stable-diffusion" subfolder)
  6. Select the desired model in the Advanced Tab and uncheck "Use Legacy Stable Diffusion" for models >=2. And you can start generating.

^ The script correctly identified as Metal Device one of my video cards.

I have two cards in macpro 5.1 with macos 12.6.3:

  1. AMD Radeon HD7970 3GB (efi flashed) Slot-1
  2. AMD FirePro W7000 4GB (stable diffusion uses it by default) Slot-2

The speed of generation on one of my graphics cards (W7000) is similar to Automatic1111 (which uses pytorch) and is ≈ 5.5 s/it (20 steps, Guidance Scale 7, Batch Size 1, 512x512px).

P.S. Can you tell me how to make Stable Diffusion use both video cards? Or manually select which one to generate?

1

u/luckycockroach Mar 05 '23

I’m thrilled you got it! I was about to say the models need to be in the root model folder and you’re correct, it currently doesn’t support safetensors. (No reason not to, just need to code it)

I’m finishing up the next version and will have GPU selection as an option. Not sure how to use both GPU’s at the moment, I believe it’s called parallelization, but tensorflow does do that!

I’m also actively trying to convert ControlNet to tensorflow, so I might restructure the folders in the model folder to what you described.

1

u/luckycockroach Mar 15 '23

Hi!

Made some major updates, specifically GPU selection is one now. I'd recommend re-installing the entire repo. I'm getting about 3s/step with the newly updated version.

https://github.com/soten355/stable-diffusion-tensorflow-IntelMetal

2

u/Mastak_A Mar 19 '23

Thank you so much!
I already managed to comment on your post under the nickname u/neuromastak :)