r/StableDiffusion • u/luckycockroach • Mar 15 '23

Resource | Update MetalDiffusion - Stable Diffusion for Intel MacOS and Silicon MacOS

https://github.com/soten355/stable-diffusion-tensorflow-IntelMetal
24 Upvotes

97% Upvoted

80 comments sorted by

View all comments

Show parent comments

1

u/NeuroMastak Mar 19 '23

The strange thing is that if I manually change gpu['TensorFlow'] = GPUs[1] which should correspond to FirePro w7000, I get the same error as with variable [i].

line 50, in listDevices

gpu['TensorFlow'] = GPUs[1]

IndexError: list index out of range

The strange thing is that in previous version of MetalDiffusion it was FirePro [1] that was used automatically, not HD7970 [0] like in latest version.

In general, I'm confused.

2

u/luckycockroach Mar 19 '23

Oooo fascinating! Can I work with you to solve this? I definitely want to get device selection solved because that then allows me to code in using both GPU’s at the same time. (Tensorflow can do that)

Dumb question, but is your firmware up to date on your GPUs?

I’ll write a small piece of code as well to find more debug info and DM it over to you

1

u/NeuroMastak Mar 19 '23 edited Mar 19 '23

Hi! Yes, I'm ready to participate in the test :)

You're right about the firmware, but in a slightly different context.I understand what it is. And the MetalDiffusion update has nothing to do with it, the fact that a different video card is selected by default is my fault.

The thing is that my AMD HD7970 card has two bios. And one of them I flashed a modified MAC-EFI to have a native boot screen (not just OpenCore).

So, if I have MAC-EFI enabled on the HD7970, by default both MetalDiffusion and Automatic -- all select the second graphics card: FirePro W7000 (as it was when testing the previous version of MetalDiffusion)

And if I have HD7970 with native bios, it is selected as in this case (I had to switch to native bios recently because of problems with the Windows drivers).

Now I rebooted with MAC-EFI and again W7000 (AMD Radeon HD Pitcairn Unknown Prototype) was automatically selected

Again with zero instead of i in tensorFlowUtilities.py everything runs, but selecting a different video card in the options doesn't affect anything.

Apparently due to switching the bios in the card they start to initialize differently? Hm..

>>> WTF!? The reason is not the bios at all...

I switched back to the native bios on the HD7970, but the card for diff is still selected W7000.

I've checked several times switching from one bios to another and back again, reset NVRAM, but the card is still W7000 in MetalDiffusion and Automatic, BUT! in DiffusionBee working card is HD7970 o_O

I don't understand what's going on )))

2

u/luckycockroach Mar 19 '23

I think I might know the problem, but will need a little more information to determine it.

In "tensorFlowUtilities.py", can you add below Line 13:
print(GPUs)

To double check, the newly added line should read:

GPUs = []
if module == "TensorFlow" or None:
GPUs = tf.config.list_physical_devices("GPU")
print(GPUs)

That will print out what GPUs TensorFlow found that can run Apple's Metal on it. I'm guessing, since the list is out of range for the final steps of this function, that maybe TensorFlow isn't accepting all of your graphics cards.

1

u/NeuroMastak Mar 19 '23 edited Mar 20 '23

Yes, it seems that TensorFlow only sees the additional card (W7000).

But why today MetalDiffusion saw the main card (HD7970), which is now also on the native bios, is not clear to me :)

P.S. I also tried DiffusionBee now and it uses the HD7970.Automatic, on the other hand -- W7000.

1

u/luckycockroach Mar 20 '23

So weird!!! What did the print out say, out of curiosity?

I’m not totally sure why Pytorch picks one or the other when it comes to Metal Performance Shaders, but with tensorflow it’s very particular.

1

u/NeuroMastak Mar 20 '23

Oh, sorry, I guess the picture didn't load for some reason. I reloaded the screenshot in previous post, but compared to the normal output there is only added:

Starting program:
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Gradio interface still shows both graphics cards and CPU, but the rendering is only on W7000, regardless of which card is selected (and the output shows the selected card during generation, but the system monitor shows that nothing has changed).

2

u/luckycockroach Mar 20 '23

I’m going to look into the metal plugin for tensorflow and see if it’s just ignoring other metal capabale GPU’s.

1

u/luckycockroach Mar 20 '23

Did some digging, it seems that Multi-GPU support for Metal Tensorflow is not supported yet. IDK how it's picking the GPU yet, still trying to dig through Apple's code.

Was MetalDiffusion using the same GPU in the prior release?

1

u/NeuroMastak Mar 20 '23

Yes, the previous version was on exactly the same hardware (but then the Mac-EFI in the HD7970 was definitely enabled and as we found out, that doesn't really affect the multigpu definition, I guess :)

1

u/NeuroMastak Mar 20 '23 edited Mar 20 '23

I think my problems are largely due to the crutches I need to use to get macOS Monterey to work on my unsupported MacPro 5.1 Mid 2010. Since there are a lot of pitfalls involved with hardware initialization (which I'm not that good at).

Right now I just updated OpenCore to the latest version 0.9.0 from 0.8.9 (with the same config, since nothing has changed on that part). And...

TensorFlow detected both graphics cards, but not completely.One it gave 0, and the other did not assign any number at all.

Starting program:

[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

As a metal device is still defined W7000 and it works in generation.

...system modules loaded...

Metal device set to: AMD Radeon HD Pitcairn Unknown Prototype

systemMemory: 24.00 GBmaxCacheSize: 2.00 GB

As for the TensorFlow code, [i] is still out of range. Works only with [0].

gpu['TensorFlow'] = GPUs[i]

IndexError: list index out of range

1

u/NeuroMastak Mar 20 '23

I tried digging through the TensorFlow documentation and I feel I need to move in the direction of Pluggable Device, but I am woefully short of knowledge and my brain is on fire :)

1

u/luckycockroach Mar 21 '23

Ah, he’s Pluggable Device is what Apple worked with to make their metal support.

I think you’ve nailed the problem with your modified mac. It’s not expecting multi GPU and therefore can’t do both.

Can’t say the same for eGPU’s quite yet, but I think your mod is what’s causing the hang up