CalvinBackup/iperov-DeepFaceLab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Added new face type : head

Browse Source 
Now you can replace the head.
Example: https://www.youtube.com/watch?v=xr5FHd0AdlQ
Requirements:
	Post processing skill in Adobe After Effects or Davinci Resolve.
Usage:
1)	Find suitable dst footage with the monotonous background behind head
2)	Use “extract head” script
3)	Gather rich src headset from only one scene (same color and haircut)
4)	Mask whole head for src and dst using XSeg editor
5)	Train XSeg
6)	Apply trained XSeg mask for src and dst headsets
7)	Train SAEHD using ‘head’ face_type as regular deepfake model with DF archi. You can use pretrained model for head. Minimum recommended resolution for head is 224.
8)	Extract multiple tracks, using Merger:
a.	Raw-rgb
b.	XSeg-prd mask
c.	XSeg-dst mask
9)	Using AAE or DavinciResolve, do:
a.	Hide source head using XSeg-prd mask: content-aware-fill, clone-stamp, background retraction, or other technique
b.	Overlay new head using XSeg-dst mask

Warning: Head faceset can be used for whole_face or less types of training only with XSeg masking.

XSegEditor: added button ‘view trained XSeg mask’, so you can see which frames should be masked to improve mask quality.
This commit is contained in:
Colombo
2020-04-04 09:28:06 +04:00
parent 383d4d3736
commit 2b7364005d
21 changed files with 506 additions and 413 deletions
Download Patch File Download Diff File Expand all files Collapse all files
merger/MergerConfig.py
+17 -17
View File
@@ -76,21 +76,21 @@ mode_dict = {0:'original',
2:'hist-match',
3:'seamless',
4:'seamless-hist-match',
5:'raw-rgb',}
5:'raw-rgb',
6:'raw-predict'}
mode_str_dict = {}
mode_str_dict = { mode_dict[key] : key for key in mode_dict.keys() }
for key in mode_dict.keys():
mode_str_dict[ mode_dict[key] ] = key
mask_mode_dict = {1:'learned',
2:'dst',
3:'XSeg-prd',
4:'XSeg-dst',
5:'XSeg-prd*XSeg-dst',
6:'learned*XSeg-prd*XSeg-dst'
mask_mode_dict = {1:'dst',
2:'learned-prd',
3:'learned-dst',
4:'learned-prd*learned-dst',
5:'XSeg-prd',
6:'XSeg-dst',
7:'XSeg-prd*XSeg-dst',
8:'learned-prd*learned-dst*XSeg-prd*XSeg-dst'
}
ctm_dict = { 0: "None", 1:"rct", 2:"lct", 3:"mkl", 4:"mkl-m", 5:"idt", 6:"idt-m", 7:"sot-m", 8:"mix-m" }
ctm_str_dict = {None:0, "rct":1, "lct":2, "mkl":3, "mkl-m":4, "idt":5, "idt-m":6, "sot-m":7, "mix-m":8 }
@@ -102,7 +102,7 @@ class MergerConfigMasked(MergerConfig):
mode='overlay',
masked_hist_match=True,
hist_match_threshold = 238,
mask_mode = 1,
mask_mode = 4,
erode_mask_modifier = 0,
blur_mask_modifier = 0,
motion_blur_power = 0,
@@ -118,7 +118,7 @@ class MergerConfigMasked(MergerConfig):
super().__init__(type=MergerConfig.TYPE_MASKED, **kwargs)
self.face_type = face_type
if self.face_type not in [FaceType.HALF, FaceType.MID_FULL, FaceType.FULL, FaceType.WHOLE_FACE ]:
if self.face_type not in [FaceType.HALF, FaceType.MID_FULL, FaceType.FULL, FaceType.WHOLE_FACE, FaceType.HEAD ]:
raise ValueError("MergerConfigMasked does not support this type of face.")
self.default_mode = default_mode
@@ -262,9 +262,9 @@ class MergerConfigMasked(MergerConfig):
if self.mode == 'hist-match' or self.mode == 'seamless-hist-match':
r += f"""hist_match_threshold: {self.hist_match_threshold}\n"""
r += f"""mask_mode: { mask_mode_dict[self.mask_mode] }\n"""
if 'raw' not in self.mode:
r += (f"""erode_mask_modifier: {self.erode_mask_modifier}\n"""
f"""blur_mask_modifier: {self.blur_mask_modifier}\n"""
@@ -274,8 +274,8 @@ class MergerConfigMasked(MergerConfig):
if 'raw' not in self.mode:
r += f"""color_transfer_mode: {ctm_dict[self.color_transfer_mode]}\n"""
r += super().to_string(filename)
r += super().to_string(filename)
r += f"""super_resolution_power: {self.super_resolution_power}\n"""
if 'raw' not in self.mode: