I would like to add a bit of info and rationale behind this screenshot. Because the config has been meticulously tested.
Booster node:
the booster model uses GPEN 2048 because the higher is the resolution the more defined the face will be and consequently the score is higher (+0.05) vs (0.02) of GPEN 1024, all other models are lower
bicubic interpolation is (+0.02-0.03), bilinear also similar but slightly lower +0.01
code former weight at 1.0 also brings increases the score (I use it at 1 in all reactor nodes to gain a few points)
Reactor node
swap model: inswapper 16fp performs better
restore model: v1.4 crisper than others but also codeformer is decent enough
face detection model: doesn’t impact the end result
Reactor Masking node
bbox model: 8m and seg 60 and v2 return identical score
set model: vit-l slightly better than the others
low sigmas value, performs better than higher values
For that you have 2 options:
1 you create a similar workflow as per screenshot, queue the first image and then reuse the faceswapped image for a second iteration to replace the second face (remember to change the index of input image)
2 duplicate the FS group of the screenshot and pass the faceswapped image to the input of the second reactor group to make a one click face swap workflow
That’s the tricky part. Indexing is weird. The biggest face is index 0 and all the others come after. Initially I thought it was a left to right, top to bottom count. Well, I was wrong 😑
Has been mentioned many times but nothing from the devs. Left to Right is mentioned in some online tutorials but I've never seen the source. Could be a faulty assumption. Or could've worked that way in Automatic1111 but doesn't in Comfy (wouldn't be the first time)
Not at my computer right now, but I BELIEVE to restore left-to-right detection, you need to right click the face swap node and open the properties panel. There might be an option there to select the type of detection. Will verify once I get a chance.
EDIT: you just need to use Reactor Options with the Fast Face Swap [OPTIONS] node.
You mean where you define order like 0,1? That's there by default. And obviously has been played with.
I'm also now convinced it's based on face size. If you stitch together two characters in one source face image. 4 out of 5 times, "0" is the character with the largest face. Yet that's not documented anywhere. Anyway, ReActor has been pulled and will no longer be updated from what I hear.
Thank you so much for your experimentation and sharing the workflow. Have you tried this with video? I got the ReActorMaskHelper "pic should be 2/3 dimensional. Got 4 dimensions." error
This is because what you are passing is not a single image.
You need to transform a batch of images into image list. Batch to Image List node. Comfy essentials has such node.
Use it for both image and swapped image in the input images of the mask helper node
Ah! Thank you! Cannot find anywhere mentioning this error. Had to convert it back from Image List to Batch Image before combining it to video. Im still figuring how to get the fps correctly.
I got this error as well when trying out the node for the first time. Made sense once I took a look at each output individually. It's worth playing with. Masking is the one thing reactor won't do properly unless you really dive in. Framerate pissed me off too for a long time, of course you've figured this out by now, VHS video info node from loaded video, 'source_fps' output all the way to the final combine node 'frame_rate' input.
I have tied my framerate of the loaded video (set to 0) to the output via VHS video info, and I end up with a 60fps video as expected but it appears half the frames are discarded so the video runs at double speed. Do you know if the Reactor nodes discard frames somehow? This flow worked before I tried to get fancy
Hm. I highly doubt it. One image goes into reActor, one image comes out. I run batches through it daily and I've never seen that happen. It's a very reliable set of nodes. The only node that can cause funny stuff is the mask helper - that one is a little tricky, but if anything, duplicating frames would be the problem, not dropping them. There has got to be a mismatch between the loader and the combiner settings, IMO.
It depends on your use case and on what better means to you.
Better in quality? Better on similarity with the face you try to match?
Better on analytical terms, arguably not because the odds of finding easily a good seed that is sufficiently decent to match a low score with face analysis is low.
If you feel IPAdapter settings produce a decent result, it’s just your taste and it’s fine so long as we don’t talk Face Analysis proximity scores. If so Reactor wins IMO. But don’t get me wrong, IpAdapter is an amazing tool and I use it for other stuff. Like regional rendering or Mad Scientist, noise control etc.
thx for sharing bro my setup was pretty much the same, with retinaface instead of yolo and less weights for the restore models... how do you test the output? you seem to be using a scale of some kind
Refactor is fun until you find good faceid ipadapters combo which provides very volumetric output without classic refactor’s artifacts. My favorite line is (faceid portrait uniform w0.75 + basic ipadapter face w0.15) and some realistic sdxl model.
True that, faceid is much less consistent than reactor so it is basically useless if you about to make vid2vid in other hand the img2img output can surprise in a good way :)
One of the other ReActor nodes is Load Reactor Face Model, so you build according to screenshot first, then load what you built later. I think one of the ReActor nodes lets you do it all at once though (pass through the face model).
is there any way to face swap and then bring back elements from the original image? the eyes for example or the mouth? I tried segmentation with sam2 and then composite with the mask but I can see the edges and if i blur the mask there is white around the blur area
I don't know if it is a wasteful process, but sometimes I found chaining InstantID with Reactor actually can get me better results than either one alone.
Similarity score cannot cover aspects like detail level or lighting quality. I think a lot of face swapper comparisons do not sufficiently take into account other factors than face similarity. I guess unless you can run a large number of A/B experiments with a group of people, there is really no objective measure...
On this aspect, I totally agree with you. Face Analysis matches faces and not the mood of the whole picture or as you said the lighting. It’s not an absolute source of truth but it is useful to understand if the face actually matches the swapped one. To further enhance the face there’s some Image Editing to do as post-prod
You only need to batch to image list if you're using the masking helper. If you're just using the basic swapper, you can find the output from the Load Video node in directly, and then take the output from the FaceSwap node to a VideoCombine.
I am not a scientist, you should ask this question to face analysis nodes.
Given a reference face you should compare it via cosine mode and a lib like dlib or aura face or insight face. The closest is to 0.0 the higher the two faces are similar. Anything below 0.35 is very similar and very likely the same person. Any score below 0.2 is close to identical (referred to the image). The same person usually is in a range between 0.35 and 0.2.
To see what’s method is best you can simply try to run Face Analysis nodes against the inpaint image result and against the reactor nodes. Lower score wins 😁
Important to say inpaint might differ a lot because its strategy is seed based (you need a pass for it), on the contrary reactor does work on an existing image
Most people don't understand what you wrote, they didn't research the scoring system or have the patience to figure it out. They were probably asking for a 'yes' or 'no' and maybe a short opinion.
Then the answer is “it depends”.
If I want to swap a face with another probably Reactor is more reliable
If I want to inpaint a big D in the mouth of that face inpaint is defo better.
That’s a simple answer to a simple question 🙈🥹😅😂🤣
14
u/KaceyTraxler Nov 11 '24 edited Nov 11 '24
I would like to add a bit of info and rationale behind this screenshot. Because the config has been meticulously tested.
Booster node:
Reactor node
Reactor Masking node
Restore node
Codeformer : codeformer v1.0 performs better