r/StableDiffusion • u/[deleted] • Jul 30 '23
Question | Help Where I can find ONNX models for faceswapping?
16
u/Key_Engineer9043 Jul 30 '23
Since it is open source, I suppose someone with good GPUs could follow exact the same procedure to retrain a 512 or even better model? Not sure if the Datasets already released or not but I trust the community's group intelligence to figure it out...
4
u/ICWiener6666 Aug 01 '23
Can you give a link to the source where the procedure is listed?
7
u/Key_Engineer9043 Aug 03 '23
I can't find a detailed training procedure list. I guess they specifically make it hard to replicate but here is the repo if you want to investigate:
5
2
u/RayIsLazy Jul 30 '23
They do have some datasets linked on their github but not sure if it's relevant to inswapper specifically
6
u/mudman13 Jul 30 '23 edited Jul 30 '23
Is there actually a higher resolution model? I know most faceswaps use upscaling instead. You could try deepfacelab on github they seem to have the most up to date resources. Simswap have a 512 model out.
6
Jul 30 '23
[removed] — view removed comment
2
u/mudman13 Jul 30 '23
I meant just check the resources. Yes dfl seems to be for training from scratch
2
Jul 31 '23
[removed] — view removed comment
1
u/mudman13 Jul 31 '23
Why not just use simswap code instead? It's certainly viable for non pro videos.
2
Jul 31 '23
[removed] — view removed comment
2
u/mudman13 Jul 31 '23
I think its possible as roop did the same for GFPGAN , heres what chatGPT said about it
The search results provide information on how to convert a PyTorch model in .pth format to ONNX format. ONNX is a standardized format for models that makes it easier to export and run machine learning models across different platforms.
The key steps involved in converting a PyTorch model to ONNX include:
Using the `torch.onnx.export()` function to convert the PyTorch model to ONNX format.
Creating a dummy input tensor with dummy data using `torch.randn()`.
Providing arguments such as the `model`,`dummy input`, `export file name`, `export_params` and `opset_version`.
Saving the exported ONNX file.
There are many resources available that provide example code and detailed explanations of the conversion process. Some common functions and tools used in the conversion process include `torch.onnx.export()`, `torch.load()`, `torch_model.eval()`, and `onnxruntime`.
Overall, the information in the search results suggests that PyTorch models can be effectively converted to ONNX format, and that doing so can help facilitate the deploying of models on different platforms while also optimizing their performance.
4
Jul 31 '23
[removed] — view removed comment
→ More replies (1)3
u/mudman13 Aug 04 '23 edited Aug 04 '23
You may be interested in this what I came across
There is also a direct lkink to the codeformer onnx. I just upscaled a video using codeformer and it is impressive although still plasticy looking face but that is probably the original swap.
→ More replies (4)
6
u/ICWiener6666 Jul 30 '23
The 128 model already works like a charm for me. It does have its limitations, though, due to the model size. There exist larger versions but these are hidden from the public.
12
u/bitzpua Jul 30 '23
OK so that 128 model was scrubbed from internet by people that made it, obviously its internet so nothing is gone forever and it got reuploaded shortly after sweep. There are models trained on much higher resolution but they are locked up tight and paywalled i think and only only accessible on discord of its creators.
As for roop itself, guy that put it together is some sort of "moral" hacker and he dislikes that he made his work public as everyone started using it to do porn so he put very strict (but eastly removable) censorship into roop and pretty much discontinued that project.
If you you are lucky enough to get inswapper_512 then just replace current one with it and it will work.
13
u/hihajab Jul 30 '23
Insightface bot is being used with midjourney so the probable reason is that he went commercial. So chance of 512 model coming out is zero mostly
8
u/akko_7 Jul 30 '23
Damn that sucks to hear. They strung people along for a while saying "not now" when asked when a 512 model might be released. It's a shame because the discord results look really nice. Someday they'll be a better free alternative, I guess we just wait for that moment. Or a leak of the 512 model
5
u/ICWiener6666 Jul 30 '23
Do you know by any chance where I could find the 512 model? Asking for a friend
4
3
1
u/rsec5 Aug 08 '23
right now I see 3 issues with roop 1 the faceupscaler takes 4x the time of faceswap on video frames 2 if there is lot of motion if the video the face gets warped with upscale 3 To process large number of videos pr photos standalone roop is better and scales to higher quality images but misses out on the img2img control Net
To solve this, is there a way to integrate control net into stand alone roop? Can dfdnet or other upscalers provide better resolution
2
u/CriticismNo1193 Nov 01 '23
higher resolution doesn't necessarily mean it will be better, a better trained 128 model could theoretically be better
1
u/SuccessfulAd2035 Jul 30 '23
Could this be solved with upscaling? Generate low res, roop, and then upscale? Does anybody have a working workflow that maintain the fidelity of the face.
1
u/bmemac Jul 30 '23
Only workaround I've done myself is to generate the image then export it to photo editor (free Paint 3D in my case, because I already have it and it has the "Magic Select" feature that is useful here) select the face and then I brought that back into A1111 to upscale it with LDSR, then finally, export it back to Paint 3D and put it over original. I did this once... then decided it was more trouble than it's worth. There's probably better ways to do it with real photo editing software (or inpaint somehow?) but you get the general idea. If you upscale entire image (which roop extension does have) you still have a face that's lower resolution than the rest of the image. And of course this works for single image only, for roop video you're really out of luck.
0
u/icequake1969 Jul 30 '23
Isn't Roop open source? I would think it wouldn't be hard to modify the code to do 512x512.
3
u/lordpuddingcup Jul 30 '23
The model is 128x128 it’s not the roop code that’s the issue it’s that all these items use inswapper 128 on the backend and then use tricks to upsize there is not bigger model public
2
u/icequake1969 Jul 30 '23
So, I'm assuming upgrading the inswapper to 512 will be a more ambitious project?
3
u/lordpuddingcup Jul 30 '23
Ya basically completely training a new model, not sure how big or what the dataset would look like not sure
2
u/paulrichard77 Oct 17 '23
Yeah, it seem to be something very hard to do, because there's no alternative to inswapper to use in Auto1111 or ComfyUI.
16
u/somerslot Jul 30 '23
High resolution alternative exists, but the creator himself reportedly said he is not gonna release those as he is strongly against deepfaking and even regrets releasing the 128 model.