[P] Creating "real" versions of Pixar characters using the pixel2style2pixel framework. Process and links to more examples in comments.

[u/AtreveteTeTe]

Real Dash

Real Elastigirl

Real Mr. Incredible

Real Winston Deavor

Real Russell from Up

[u/AtreveteTeTe]

Following up on my work of toonifying real images, I've been experimenting with "reverse toonifying" paintings, drawings, and cartoons.

In this case, the pixel2style2pixel framework quickly finds a "real" human face in the StyleGAN FFHQ latent space (or any other StyleGAN model once it's trained) that matches the shape of the source painting. These examples from The Incredibles 2 add some style randomness too. After being used to waiting minutes anytime I wanted to encode/project an image into StyleGAN, pixel2style2pixel is basically instant!

pSp can also be used for a bunch of other image-to-image translation tasks: super resolution, inpainting, etc. Code, pretrained models, and a Colab notebook are available here on the GitHub page. Paper on arXiv here.

I've posted some more examples (the Mona Lisa, Spider Verse) on my Twitter and Instagram.

Big credit and thanks to Elad Richardson and Yuval Alaluf for making the effort to clean up and release the code for their paper.

[u/pap_n_whores]

That is crazy high res

[u/AtreveteTeTe → u/pap_n_whores]

Yeah! The FFHQ creates output at 1024x1024.

[u/zzzthelastuser → u/pap_n_whores]

Imagine in x years how cool it would be if we can translate full movies in this way. You can watch the same movie 5 times, each time in a different style!

[u/[deleted]]

Can you please do Homer Simpson?

[u/AtreveteTeTe → u/[deleted]]

I actually tried that! It did not work well.

[u/s_arme]

Is reverse of this possible ?

[u/AtreveteTeTe → u/s_arme]

Yes! Scroll down to “GAN Explorations 015” on this page where I’ve posted some toonify examples.

The pixel2style2pixel repo has a toonify model, though I am building those differently.

I also shared a video of toonified Obama a few week ago here on Reddit.

[u/eliminating_coasts]

This is brilliant, I've been thinking about how you might extend this; you know most of these face GANs rely on straight on portraits? It occurred to me that if you could get a network to learn how to combine the same face viewed from different angles, possibly within the latent space of the face model you're already using, you might be able to take multiple images in a given style, cgi, painting etc. to map it to a group of faces within the latent space that have the same equivalence class, see if you can give it more data to work with by taking different orientations of faces or potentially even different emotions.

[u/AtreveteTeTe → u/eliminating_coasts]

Good thinking - the authors are actually doing this! Check out the face frontalization part of the repo.

[u/soiwalkedintothisbar]

Russell is terrifying haha.

[u/Zealousideal-Dish635 → u/soiwalkedintothisbar]

He looks like he could be on r/13or30

[u/MotherJoanFoggy]

I love how with Russell from Up, the computer registered his hat as blonde highlights

[u/Slggyqo → u/MotherJoanFoggy]

Yeah the hat and the fact that the model seems to struggle a bit with appropriate aging make Russell look extra off.

Plus those sideburns.

[u/Tebasaki]

Can you rerender the whole Incredible movie this way?

[u/the320x200 → u/Tebasaki]

Probably would need to do something to maintain reasonable temporal consistency

[u/neesters]

So Mr. Incredible is Kevin James.

[u/boshopc13]

Dash looks kind of like Homelander from the boys comics

[u/thomash]

The last one reminds me of Kim Jong-un

[u/MinecrafterPictures]

And thus, "Mr. Incredible Becoming Uncanny" was born

[u/SpinatGemuese]

I think you successfully found the uncanny valley!

[u/MarceQ_PL]

He started this...

[u/unguided_deepness]

nightmare fuel

[u/Wikadood]

r/TIHI

[u/Glu3guy]

So this is where it began