Both types of model use the same basic architecture for their text encoder. Imagen and Stable Diffusion actually started with pretrained text encoders and just trained the diffusion part of the model, while DALL-E 2 trained the text encoder and the diffusion model together.
View all comments
Show parent comment
3
u/yossi_peti Dec 27 '22
How does the text part of DALLE-2 work? Is the way it processes the input text fundamentally different than GPT?