More stable and precise version of the face transformation matrix.
Now full_faces are aligned with the upper and lateral boundaries of the frame,
result: fix of cutted mouth, increase area of the cheeks of side faces
before/after https://i.imgur.com/t9IyGZv.jpg
therefore, additional training is required for existing models.
Optionally, you can re-extract dst faces of your project, if they have problems with cutted mouth or cheeks.
improved model generalization, overall accuracy and sharpness
by using new 'Learning rate dropout' technique from paper https://arxiv.org/abs/1912.00144
An example of a loss histogram where this function is enabled after the red arrow:
https://i.imgur.com/3olskOd.jpg
This is sort method by absolute per pixel difference between all faces.
options:
Sort by similar? ( y/n ?:help skip:y ) :
if you choose 'n', then most dissimilar faces will be placed first.
now you have 3 ways:
1) define light directions manually (not for google colab)
watch demo https://youtu.be/79xz7yEO5Jw
2) relight faceset with one random direction
3) relight faceset with predefined 8 directions
Synthesize new faces from existing ones by relighting them using DeepPortraitRelighter network.
With the relighted faces neural network will better reproduce face shadows.
Therefore you can synthsize shadowed faces from fully lit faceset.
https://i.imgur.com/wxcmQoi.jpg
as a result, better fakes on dark faces:
https://i.imgur.com/5xXIbz5.jpg
in OpenCL build Relighter runs on CPU,
install pytorch directly via pip install, look at requirements
This is the fastest model for low-end cards.
Model has zero options and trains a 96pix fullface.
It is good for quick deepfake demo.
Example of the preview trained in 15 minutes on RTX2080Ti:
https://i.imgur.com/oRMvZFP.jpg
fixed crashes
removed useless 'ebs' color transfer
changed keys for color degrade
added image degrade via denoise - same as denoise extracted data_dst.bat ,
but you can control this option directly in the interactive converter
added image degrade via bicubic downscale and upscale
SAEHD: default ae_dims for df now 256.
removed option 'apply random ct'
added option
Color transfer mode apply to src faceset. ( none/rct/lct/mkl/idt, ?:help skip: none )
Change color distribution of src samples close to dst samples. Try all modes to find the best.
before was lct mode, but sometime it does not work properly for some facesets.
Random warp is required to generalize facial expressions of both faces. When the face is trained enough, you can disable it to get extra sharpness for less amount of iterations.
added SAEHD model ( High Definition Styled AutoEncoder )
This is a new heavyweight model for high-end cards to achieve maximum possible deepfake quality in 2020.
Differences from SAE:
+ new encoder produces more stable face and less scale jitter
before: https://i.imgur.com/4jUcol8.gifv
after: https://i.imgur.com/lyiax49.gifv - scale of the face is less changed within frame size
+ new decoder produces subpixel clear result
+ pixel loss and dssim loss are merged together to achieve both training speed and pixel trueness
+ by default networks will be initialized with CA weights, but only after first successful iteration
therefore you can test network size and batch size before weights initialization process
+ new neural network optimizer consumes less VRAM than before
+ added option <Enable 'true face' training>
The result face will be more like src and will get extra sharpness.
example: https://i.imgur.com/ME3A7dI.gifv
Enable it for last 15-30k iterations before conversion.
+ encoder and decoder dims are merged to one parameter encoder/decoder dims
+ added mid-full face, which covers 30% more area than half face.
removed TrueFace model.
added SAEv2 model. Differences from SAE:
+ default e_ch_dims is now 21
+ new encoder produces more stable face and less scale jitter
before: https://i.imgur.com/4jUcol8.gifv
after: https://i.imgur.com/lyiax49.gifv - scale of the face is less changed within frame size
+ decoder now has only 1 residual block instead of 2, result is same quality with less decoder size
+ added mid-full face, which covers 30% more area than half face.
+ added option " Enable 'true face' training "
Enable it only after 50k iters, when the face is sharp enough.
the result face will be more like src.
The most src-like face with 'true-face-training' you can achieve with DF architecture.
fixed model sizes from previous update.
avoided bug in ML framework(keras) that forces to train the model on random noise.
Converter: added blur on the same keys as sharpness
Added new model 'TrueFace'. This is a GAN model ported from https://github.com/NVlabs/FUNIT
Model produces near zero morphing and high detail face.
Model has higher failure rate than other models.
Keep src and dst faceset in same lighting conditions.
Session is now saved to the model folder.
blur and erode ranges are increased to -400+400
hist-match-bw is now replaced with seamless2 mode.
Added 'ebs' color transfer mode (works only on Windows).
FANSEG model (used in FAN-x mask modes) is retrained with new model configuration
and now produces better precision and less jitter
if input frames are changed (amount or filenames)
then interactive converter automatically starts a new session.
if model is more trained then all frames will be recomputed again with their saved configs.
ConvertAvatar: fix input image after fix landmarks face align,
VideoEd: video_from_sequence now uses pipe input to input any filenames instead of %.5d. formatted
Colombo
