DFL – Sync image sequence FPS and final output

[LacidarionParticipant]

I must say that DeepFaceLive is an amazing bit of work! The Face Animator does wonders with a single photo and at 60 frames per second on my 3090 rig. Funny, I am getting more convincing output from this than with hours and hours of training with other systems. Also loads of fun pranking friends on Zoom meetings!

Currently I am using the app in FaceSwap (Insight) mode and once again… just amazing what it can do, even on a 1280 video and a 256 pixel single image.

I am trying to do something which I think would be easy, so perhaps I am just missing the proper settings, although I do find the app very easy to use and understand.
I am exporting a video as a series of 1920×1080 PNG frames and then using Insight FaceSwap to replace a face in the video. I Stream the output also a a series of frames.
The function works perfectly to the point that it will convince almost anyone, but where I am having trouble is the frame rate.

For some reason, no matter how I set the FileSource (auto / low FPS / matching FPS) the output module just spews out what seems to be 30FPS or more and of course skips frames.

What I want is quality over speed as this is not real-time. I don’t care if the conversion takes even a few seconds to process each frame, but I would expect the output image sequence to perfectly match the input sequence so when I use Media Encoder to stitch it back up it should play perfectly.

This never happens.

Is there a way to tell the app that it is to process all frames one at a time and then write them with no fills or gaps? Basically a batch processor.

Cheers!

[LacidarionParticipant]

I forgot to mention, the closest I have been able to get to what I want is to do the following…

– Set FileSource to a directory containing extracted video frames as PNG 1920×1080 files (could be hundreds).
– Set FileSource frame rate to 4 frames per second (enough time to process at massive quality wthout any frame drops).
– Use a very high image for FaceSwap, something at least 1024 wide.
– Use Insight Face marker as well as Insight FaceSwap *best results).
– Set Face aligner to 1024 resolution (dog slow for realtime, but super for quality).
– Set Stream output to udp://127.0.0.1

Then to capture (since writing frames from DFL does not work well), I use OBS on that stream port.

Now once the stream is recorded, I then have to use Media Encoder to write it at a multiple of the desired frame rate.
If I encode at 4 FPS, I then output as 225FPS to bring the resulting video back to the correct speed of 30 FPS.

Now if there was just a way to have DFL output one frame after the other as a sequence, I could drop a lot of extra steps!!

Cheers!

[Author]

Posts