Step 4-5: Extract Source Faceset and Step 5: Extract Destination Faceset-
Q1: You want to have a good variety with fewer images. Try to choose images that match the conditions of the destination video.
Q2: You should remove as many duplicates as possible. Best faces is not good on its own, better to go through the images using different sort methods, or another tool like Machine Video Editor.
Q3: A few bad images won’t hurt and may add some unintended variety to the training. Unless resolution is absolutely critical you shouldn’t worry about it.
Q4: The images in the aligned folder will be trained so those need to be cleaned up. Remove unwanted faces and use something like MVE to fix bad alignments. The dst frame images are only used in the extract and merge processes.
Step 5.3: XSeg Mask Labeling & XSeg Model Training
Q1: XSeg is not mandatory because the faces have a default mask. However in order to get the face proportions correct, and a better likeness, the mask needs to be fit to the actual faces. The main problem with the default mask is that the dst features (especially the forehead) will start to take over, and will crush the source face features. For WF and Head models it is HIGHLY recommended.
Q2: Generic is ok for easier videos. If you have fast movement, lots of shadows, obstructions, etc I would do a custom XSeg.
Step 6: Deepfake Model Training
Q1: For pretraining its better to have a variety of faces, but you could try pretraining on the source. You could just jump right into normal training. Pretrain images need to be packed using the utility, then placed in _internal/pretrain_faces. Yes, start with it on (Y) at the beginning, then turn it off (N) and don’t turn it back on. The pretrainer is more or less a specific set of settings and the pretrain faceset, AFAIK…
Q2: This might be due to the page file being too small. I believe there’s a link to info in the guide. Could also be a power issue. Try increasing Windows page file and if that doesn’t work maybe lower the model batch size.
Q3: GAN is tricky. I tend to avoid it after a few bad experiences. You can try it, just have to wait a while to see the results. Looks like crap at first.
Q4: Yeah you can reuse the model for another video. I would suggest going back to the warp phase for a while so the model can adapt to the new DST face. You might also delete one or both of the files labeled *_inter_AB and *_inter_B. These files (AFAIK ) basically hold the “intermediary” info that maps the SRC to DST or something like that. These 2 files are often deleted when reusing a model on a completely new SRC/DST, so try playing with those, after making a backup of course.
Q5: There’s not a set value because because the learning rate will be affected by whatever options you might have enabled at the time. You really do have to just look at it. Having said that, I try to go to below 0.1 under “normal” settings. But, for instance, you might decide to turn on color transfer at the end, in which case the learning rate would jump back up, but you might like the resulting look. The program has no idea what “looks good”.
Step 7: Merge Deepfake Model to Frame Images
Q1: Not sure what you mean here…
Q2: Yes, you can just set the first frame, then apply to the remaining frames (Shift + /) then process them (Shift + >). There’s no convenient way to skip around. I’ve been thinking of coding my own keyframe solution but I just don’t have the time or advanced python skills right now.
Q3: Again, not sure…
Start with a short video of just someone talking, like a dialogue clip from a movie. Something where the character is looking off the the side of the camera at another person. Not much movement, normal lighting, nothing in front of their face. This is probably the easiest angle to deepfake. Pick a similar looking face to swap as the source.