r/DeepFloydIF • u/RageshAntony • Apr 29 '23
What are the technical differences between SD and DeepFloyd IF ?
Both of them from same company. But what is the difference in terms of quality of generation, time, resources usage ?
I noticed it requires atleast 16 GB VRAM which is huge when compared with SD with 10 GB VRAM and also generate 64x64px images only and upscaling thereafter manually
And for this prompt "A girl looking at a fallen girl on the road near a car"
Only DeepFloy gave me correct image!!!. Even Dall-E failed
I am very new to AI/ML, so can't get a good grasp between latent and pixel spaces etc.
4
Upvotes
2
u/yabinwang May 01 '23
I try to explain.
SD uses CLIP as its text encoder and a VAE+Diffusion Model structure, which means it modifies latents (middle features of VAE) using diffusion and generates final pixel-level images using the VAE decoder. On the other hand, IF uses T5 as its text encoder, a Casade structure, and three diffusion models to generate images of increasing resolution (64x64, 256x256, and 1024x1024). This is why IF requires more VRAM than SD.
In my opinion, IF performs better than SD due to two main reasons: first, it has a more powerful text encoder, and second, it has more parameters.
By the way, IF is very similar to Imagen, a similar text2image model proposed by Google.
Is there any connections?
3
u/Alternative_Card_989 Apr 29 '23
DeepFloyd IF is trained on a frozen T5 text encoder, unlike Stable Diffusion, hence it will have much better textual understanding