* mark as next, introduce dynamic scale for face debugger

* use latest onnxruntime

* update within Gradio 5

* Remove system memory limit (#986)

* remove system memory limit from ui

* remove system memory limit from args.py

* flatten the face store

* prevent countless importlib.import_module calls

* remove --onnxruntime from install.py

* remove --onnxruntime from install.py

* resolve static inference providers to fix macos (#1127)

* resolve static inference providers to fix macos

* fix lint

* restore old behaviour

* restore old behaviour

* handle ghost and uniface as well

* adjust condition for ghost and uniface

* fix Gradio gallery styles

* remove face store (#1132)

* fix dataflow in streamer

* Face selector auto mode (#1137)

* introduce face selector auto mode

* introduce face selector auto mode

* introduce face selector auto mode

* correct way is to pass source_vision_frames

* make the world a better place

* fix dataflow in faceswapper, no read of files withing inner methods (#1148)

* fix dataflow in faceswapper, no read of files withing inner methods

* fix lint

* adjust code more

* adjust code more

* bring back the face store but for source and reference only (#1149)

* bring back the face store but for source and reference only

* fix ci

* minor improvement

* guard for tobytes()

* drop condition in select_faces()

* Replace CONFIG_PARSER global with @lru_cache (#1147)

* remove global config_parser

* fix import order

* remove lambda

* remove unused block

* optimize app context detection

* decouple common modules from core (#1152)

* decouple common modules from core

* remove that nonsense

* remove that nonsense

* minor adjustment to workflows

* Tag HEVC output as hvc1 and move moov atom to the front (#1153)

* Tag HEVC output as hvc1 and move moov atom to the front

ffmpeg defaults HEVC in MP4 to the 'hev1' sample entry and leaves the moov
atom at the tail. Apple players (QuickTime, Finder QuickLook) refuse to decode
'hev1' and stall reading a tail-placed moov on large files, so hevc_nvenc /
libx265 renders cannot be previewed on macOS.

- add ffmpeg_builder.set_video_tag(): emit `-tag:v hvc1` for every HEVC
  encoder (libx265, hevc_nvenc, hevc_amf, hevc_qsv, hevc_videotoolbox).
  Applied in merge_video where the encoder is known; `-c:v copy` in the audio
  mux / concat steps preserves the tag.
- add ffmpeg_builder.set_faststart(): emit `-movflags +faststart`, applied in
  restore_audio / replace_audio / concat_video which write the final output.

H.264 and other codecs are left untouched. Verified on a real hevc_nvenc
render: hev1 hung QuickLook (no thumbnail); after the patch the file is hvc1
with a front-placed moov and QuickLook generates a thumbnail.

* Restrict hvc1 tag and faststart to quicktime containers

Gate set_video_tag / set_faststart on the output container format
(m4v, mov, mp4) via get_file_format(), so non-quicktime muxers no longer
receive -tag:v hvc1 / -movflags +faststart. Trim test_set_video_tag to a
single positive and negative assertion.

Addresses review on #1153.

* Move hvc1 tag and faststart gates into ffmpeg_builder

Rename set_video_tag / set_faststart to conditional_* and push the
container-format gate (m4v, mov, mp4) inside the builders, keeping
ffmpeg.py free of inline conditionals. Matches the set_image_quality
pattern. Addresses review on #1153.

* post cleanup after merge

* Pack target frames (#1158)

* pack target frames

* add todos

* add todos, resolve todos

* resolve todos

* change names

* revert to single target frame for select faces

* fix lint

* return empty frame

* get() have no default

* Fix trim (#1162)

* fix trim

* fix trim

* rename ffmpeg builder method

* rename to temp_frame_set and temp_frame_pattern

---------

Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com>
Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>

* Implement face tracker (#1163)

* add face tracker

* change get_nearest_track_face -> get_nearest_track_index

* create face_creator.py and move methods around

* add type FaceTrack

* naming

* remove iou test, don't belong there

* fix spaces

* rename to interpolate_points

* rename to find_best_face_track

* just track_faces

* cleanp

* previous next naming

* remove >= and >=

* rename

* remove helper from test and use face from source.jpg

* make get_anchor_indices more readable

* track_faces() call before and is forwarded to select_faces

* change to interpolate_faces

* rename methods

* rename methods

* rename variables

* remove dtype

* move face_anlyser -> face_creator

* claenup face_creator.py

* move tests to dedicated test face detector

* move tracking inside select_faces

* simplify face_tracker (#1165)

* minor renaming

* improve face_tracker test (#1166)

* improve face_tracker test

* cleanup

* Add target frame amount (#1167)

* introduce --target-frame-amount

* add ui

* make track_faces conditional

* update choices.py

* fix []

* rename component file to frame_process.py

* fix track preview (#1168)

* introduce face origin (#1169)

* add guard to prevent failure

* show and hide voice extractor according to lip syncer

* rename average_face_coordinates to average_face_geometry

* use static faces for select_faces()

* face store with lock (#1171)

* face store with lock

* face store with lock

* remove refill color from bbox

* adjust tests and handle frame_position proper way

* enforce similar naming

* introduce face tracker score

* introduce face tracker score

* fix/audio-trim-alignment (#1173)

* fix audio offset

* fix audio offset

* remove reference_frame_number check

---------

Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com>

* reduce face tracker score from 0 to 0.5

* mark as 3.7.0

* make face tracker stateless

---------

Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>
Co-authored-by: kazuki nakai <kazuki.nakai@agiletec.net>
Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com>
This commit is contained in:
Henry Ruhs
2026-06-30 15:00:02 +02:00
committed by GitHub
parent 5b7d145aa7
commit a2cbfd73b1
76 changed files with 1281 additions and 586 deletions
+2 -2
View File
@@ -33,7 +33,7 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.12'
- run: python install.py --onnxruntime default --skip-conda
- run: python install.py default --skip-conda
- run: pip install pytest
- run: pytest
report:
@@ -48,7 +48,7 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.12'
- run: python install.py --onnxruntime default --skip-conda
- run: python install.py default --skip-conda
- run: pip install coveralls
- run: pip install pytest
- run: pip install pytest-cov
+6 -1
View File
@@ -32,6 +32,9 @@ reference_face_position =
reference_face_distance =
reference_frame_number =
[face_tracker]
face_tracker_score =
[face_masker]
face_occluder_model =
face_parser_model =
@@ -50,6 +53,9 @@ trim_frame_end =
temp_frame_format =
keep_temp =
[frame_process]
target_frame_amount =
[output_creation]
output_image_quality =
output_image_scale =
@@ -125,7 +131,6 @@ execution_thread_count =
[memory]
video_memory_strategy =
system_memory_limit =
[misc]
log_level =
+4 -2
View File
@@ -5,12 +5,14 @@ from facefusion.types import AppContext
def detect_app_context() -> AppContext:
jobs_path = os.path.join('facefusion', 'jobs')
uis_path = os.path.join('facefusion', 'uis')
frame = sys._getframe(1)
while frame:
if os.path.join('facefusion', 'jobs') in frame.f_code.co_filename:
if jobs_path in frame.f_code.co_filename:
return 'cli'
if os.path.join('facefusion', 'uis') in frame.f_code.co_filename:
if uis_path in frame.f_code.co_filename:
return 'ui'
frame = frame.f_back
return 'cli'
+2 -1
View File
@@ -33,6 +33,7 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('reference_face_position', args.get('reference_face_position'))
apply_state_item('reference_face_distance', args.get('reference_face_distance'))
apply_state_item('reference_frame_number', args.get('reference_frame_number'))
apply_state_item('face_tracker_score', args.get('face_tracker_score'))
apply_state_item('face_occluder_model', args.get('face_occluder_model'))
apply_state_item('face_parser_model', args.get('face_parser_model'))
apply_state_item('face_mask_types', args.get('face_mask_types'))
@@ -45,6 +46,7 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('trim_frame_end', args.get('trim_frame_end'))
apply_state_item('temp_frame_format', args.get('temp_frame_format'))
apply_state_item('keep_temp', args.get('keep_temp'))
apply_state_item('target_frame_amount', args.get('target_frame_amount'))
apply_state_item('output_image_quality', args.get('output_image_quality'))
apply_state_item('output_image_scale', args.get('output_image_scale'))
apply_state_item('output_audio_encoder', args.get('output_audio_encoder'))
@@ -77,7 +79,6 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('benchmark_resolutions', args.get('benchmark_resolutions'))
apply_state_item('benchmark_cycle_count', args.get('benchmark_cycle_count'))
apply_state_item('video_memory_strategy', args.get('video_memory_strategy'))
apply_state_item('system_memory_limit', args.get('system_memory_limit'))
apply_state_item('log_level', args.get('log_level'))
apply_state_item('halt_on_error', args.get('halt_on_error'))
apply_state_item('job_id', args.get('job_id'))
+2 -2
View File
@@ -9,7 +9,7 @@ import facefusion.choices
from facefusion import content_analyser, core, state_manager
from facefusion.cli_helper import render_table
from facefusion.download import conditional_download, resolve_download_url
from facefusion.face_store import clear_static_faces
from facefusion.face_store import clear_faces
from facefusion.filesystem import get_file_extension
from facefusion.types import BenchmarkCycleSet
from facefusion.vision import count_video_frame_total, detect_video_fps
@@ -64,7 +64,7 @@ def cycle(cycle_count : int) -> BenchmarkCycleSet:
if state_manager.get_item('benchmark_mode') == 'cold':
content_analyser.analyse_image.cache_clear()
content_analyser.analyse_video.cache_clear()
clear_static_faces()
clear_faces()
start_time = perf_counter()
core.conditional_process()
+7 -4
View File
@@ -2,7 +2,7 @@ import logging
from typing import List, Sequence, get_args
from facefusion.common_helper import create_float_range, create_int_range
from facefusion.types import Angle, AudioEncoder, AudioFormat, AudioTypeSet, BenchmarkMode, BenchmarkResolution, BenchmarkSet, DownloadProvider, DownloadProviderSet, DownloadScope, EncoderSet, ExecutionProvider, ExecutionProviderSet, FaceDetectorModel, FaceDetectorSet, FaceLandmarkerModel, FaceMaskArea, FaceMaskAreaSet, FaceMaskRegion, FaceMaskRegionSet, FaceMaskType, FaceOccluderModel, FaceParserModel, FaceSelectorMode, FaceSelectorOrder, Gender, ImageFormat, ImageTypeSet, JobStatus, LogLevel, LogLevelSet, Race, Score, TempFrameFormat, UiWorkflow, VideoEncoder, VideoFormat, VideoMemoryStrategy, VideoPreset, VideoTypeSet, VoiceExtractorModel
from facefusion.types import Angle, AudioEncoder, AudioFormat, AudioTypeSet, BenchmarkMode, BenchmarkResolution, BenchmarkSet, DownloadProvider, DownloadProviderSet, DownloadScope, EncoderSet, ExecutionProvider, ExecutionProviderSet, FaceDetectorModel, FaceDetectorSet, FaceLandmarkerModel, FaceMaskArea, FaceMaskAreaSet, FaceMaskRegion, FaceMaskRegionSet, FaceMaskType, FaceOccluderModel, FaceParserModel, FaceSelectorGender, FaceSelectorMode, FaceSelectorOrder, FaceSelectorRace, Gender, ImageFormat, ImageTypeSet, JobStatus, LogLevel, LogLevelSet, Race, Score, TempFrameFormat, UiWorkflow, VideoEncoder, VideoFormat, VideoMemoryStrategy, VideoPreset, VideoTypeSet, VoiceExtractorModel
face_detector_set : FaceDetectorSet =\
{
@@ -16,8 +16,10 @@ face_detector_models : List[FaceDetectorModel] = list(get_args(FaceDetectorModel
face_landmarker_models : List[FaceLandmarkerModel] = list(get_args(FaceLandmarkerModel))
face_selector_modes : List[FaceSelectorMode] = list(get_args(FaceSelectorMode))
face_selector_orders : List[FaceSelectorOrder] = list(get_args(FaceSelectorOrder))
face_selector_genders : List[Gender] = list(get_args(Gender))
face_selector_races : List[Race] = list(get_args(Race))
genders : List[Gender] = list(get_args(Gender))
races : List[Race] = list(get_args(Race))
face_selector_genders : List[FaceSelectorGender] = list(get_args(FaceSelectorGender))
face_selector_races : List[FaceSelectorRace] = list(get_args(FaceSelectorRace))
face_occluder_models : List[FaceOccluderModel] = list(get_args(FaceOccluderModel))
face_parser_models : List[FaceParserModel] = list(get_args(FaceParserModel))
face_mask_types : List[FaceMaskType] = list(get_args(FaceMaskType))
@@ -153,7 +155,6 @@ job_statuses : List[JobStatus] = list(get_args(JobStatus))
benchmark_cycle_count_range : Sequence[int] = create_int_range(1, 10, 1)
execution_thread_count_range : Sequence[int] = create_int_range(1, 32, 1)
system_memory_limit_range : Sequence[int] = create_int_range(0, 128, 4)
face_detector_margin_range : Sequence[int] = create_int_range(0, 100, 1)
face_detector_angles : Sequence[Angle] = create_int_range(0, 270, 90)
face_detector_score_range : Sequence[Score] = create_float_range(0.0, 1.0, 0.05)
@@ -162,6 +163,8 @@ face_mask_blur_range : Sequence[float] = create_float_range(0.0, 1.0, 0.05)
face_mask_padding_range : Sequence[int] = create_int_range(0, 100, 1)
face_selector_age_range : Sequence[int] = create_int_range(0, 100, 1)
reference_face_distance_range : Sequence[float] = create_float_range(0.0, 1.0, 0.05)
face_tracker_score_range : Sequence[Score] = create_float_range(0.0, 0.5, 0.05)
target_frame_amount_range : Sequence[int] = create_int_range(0, 10, 1)
output_image_quality_range : Sequence[int] = create_int_range(0, 100, 1)
output_image_scale_range : Sequence[float] = create_float_range(0.25, 8.0, 0.25)
output_audio_quality_range : Sequence[int] = create_int_range(0, 100, 1)
+6
View File
@@ -78,6 +78,12 @@ def get_first(__list__ : Any) -> Any:
return None
def get_middle(__list__ : Any) -> Any:
if isinstance(__list__, Sequence) and __list__:
return __list__[len(__list__) // 2]
return None
def get_last(__list__ : Any) -> Any:
if isinstance(__list__, Reversible):
return next(reversed(__list__), None)
+12 -21
View File
@@ -1,29 +1,20 @@
from configparser import ConfigParser
from functools import lru_cache
from typing import List, Optional
from facefusion import state_manager
from facefusion.common_helper import cast_bool, cast_float, cast_int
CONFIG_PARSER = None
def get_config_parser() -> ConfigParser:
global CONFIG_PARSER
if CONFIG_PARSER is None:
CONFIG_PARSER = ConfigParser()
CONFIG_PARSER.read(state_manager.get_item('config_path'), encoding = 'utf-8')
return CONFIG_PARSER
def clear_config_parser() -> None:
global CONFIG_PARSER
CONFIG_PARSER = None
@lru_cache
def get_static_config_parser() -> ConfigParser:
config_parser = ConfigParser()
config_parser.read(state_manager.get_item('config_path'), encoding = 'utf-8')
return config_parser
def get_str_value(section : str, option : str, fallback : Optional[str] = None) -> Optional[str]:
config_parser = get_config_parser()
config_parser = get_static_config_parser()
if config_parser.has_option(section, option) and config_parser.get(section, option).strip():
return config_parser.get(section, option)
@@ -31,7 +22,7 @@ def get_str_value(section : str, option : str, fallback : Optional[str] = None)
def get_int_value(section : str, option : str, fallback : Optional[str] = None) -> Optional[int]:
config_parser = get_config_parser()
config_parser = get_static_config_parser()
if config_parser.has_option(section, option) and config_parser.get(section, option).strip():
return config_parser.getint(section, option)
@@ -39,7 +30,7 @@ def get_int_value(section : str, option : str, fallback : Optional[str] = None)
def get_float_value(section : str, option : str, fallback : Optional[str] = None) -> Optional[float]:
config_parser = get_config_parser()
config_parser = get_static_config_parser()
if config_parser.has_option(section, option) and config_parser.get(section, option).strip():
return config_parser.getfloat(section, option)
@@ -47,7 +38,7 @@ def get_float_value(section : str, option : str, fallback : Optional[str] = None
def get_bool_value(section : str, option : str, fallback : Optional[str] = None) -> Optional[bool]:
config_parser = get_config_parser()
config_parser = get_static_config_parser()
if config_parser.has_option(section, option) and config_parser.get(section, option).strip():
return config_parser.getboolean(section, option)
@@ -55,7 +46,7 @@ def get_bool_value(section : str, option : str, fallback : Optional[str] = None)
def get_str_list(section : str, option : str, fallback : Optional[str] = None) -> Optional[List[str]]:
config_parser = get_config_parser()
config_parser = get_static_config_parser()
if config_parser.has_option(section, option) and config_parser.get(section, option).strip():
return config_parser.get(section, option).split()
@@ -65,7 +56,7 @@ def get_str_list(section : str, option : str, fallback : Optional[str] = None) -
def get_int_list(section : str, option : str, fallback : Optional[str] = None) -> Optional[List[int]]:
config_parser = get_config_parser()
config_parser = get_static_config_parser()
if config_parser.has_option(section, option) and config_parser.get(section, option).strip():
return list(map(int, config_parser.get(section, option).split()))
+7 -13
View File
@@ -1,16 +1,14 @@
from functools import lru_cache
from typing import List, Tuple
from typing import Tuple
import numpy
from tqdm import tqdm
from facefusion import inference_manager, state_manager, translator
from facefusion.common_helper import is_macos
from facefusion.download import conditional_download_hashes, conditional_download_sources, resolve_download_url
from facefusion.execution import has_execution_provider
from facefusion.filesystem import resolve_relative_path
from facefusion.thread_helper import conditional_thread_semaphore
from facefusion.types import Detection, DownloadScope, DownloadSet, ExecutionProvider, Fps, InferencePool, ModelSet, VisionFrame
from facefusion.types import Detection, DownloadScope, DownloadSet, Fps, InferencePool, ModelSet, VisionFrame
from facefusion.vision import detect_video_fps, fit_contain_frame, read_image, read_video_frame
STREAM_COUNTER = 0
@@ -119,12 +117,6 @@ def clear_inference_pool() -> None:
inference_manager.clear_inference_pool(__name__, model_names)
def resolve_execution_providers() -> List[ExecutionProvider]:
if is_macos() and has_execution_provider('coreml'):
return [ 'cpu' ]
return state_manager.get_item('execution_providers')
def collect_model_downloads() -> Tuple[DownloadSet, DownloadSet]:
model_set = create_static_model_set('full')
model_hash_set = {}
@@ -175,10 +167,12 @@ def analyse_video(video_path : str, trim_frame_start : int, trim_frame_end : int
for frame_number in frame_range:
if frame_number % int(video_fps) == 0:
vision_frame = read_video_frame(video_path, frame_number)
total += 1
if analyse_frame(vision_frame):
counter += 1
if numpy.any(vision_frame):
total += 1
if analyse_frame(vision_frame):
counter += 1
if counter > 0 and total > 0:
rate = counter / total * 100
+10 -31
View File
@@ -5,14 +5,13 @@ import signal
import sys
from time import time
from facefusion import benchmarker, cli_helper, content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer, hash_helper, logger, state_manager, translator, voice_extractor
from facefusion import benchmarker, cli_helper, content_analyser, hash_helper, logger, state_manager, translator
from facefusion.args import apply_args, collect_job_args, reduce_job_args, reduce_step_args
from facefusion.download import conditional_download_hashes, conditional_download_sources
from facefusion.exit_helper import hard_exit, signal_exit
from facefusion.filesystem import get_file_extension, get_file_name, is_image, is_video, resolve_file_paths, resolve_file_pattern
from facefusion.jobs import job_helper, job_manager, job_runner
from facefusion.jobs.job_list import compose_job_list
from facefusion.memory import limit_system_memory
from facefusion.processors.core import get_processors_modules
from facefusion.program import create_program
from facefusion.program_helper import validate_args
@@ -41,11 +40,6 @@ def cli() -> None:
def route(args : Args) -> None:
system_memory_limit = state_manager.get_item('system_memory_limit')
if system_memory_limit and system_memory_limit > 0:
limit_system_memory(system_memory_limit)
if state_manager.get_item('command') == 'force-download':
error_code = force_download()
hard_exit(error_code)
@@ -107,21 +101,9 @@ def pre_check() -> bool:
def common_pre_check() -> bool:
common_modules =\
[
content_analyser,
face_classifier,
face_detector,
face_landmarker,
face_masker,
face_recognizer,
voice_extractor
]
content_analyser_content = inspect.getsource(content_analyser).encode()
content_analyser_hash = hash_helper.create_hash(content_analyser_content)
return all(module.pre_check() for module in common_modules) and content_analyser_hash == 'b14e7b92'
return hash_helper.create_hash(content_analyser_content) == '975d67d6'
def processors_pre_check() -> bool:
@@ -132,22 +114,19 @@ def processors_pre_check() -> bool:
def force_download() -> ErrorCode:
common_modules =\
[
content_analyser,
face_classifier,
face_detector,
face_landmarker,
face_masker,
face_recognizer,
voice_extractor
]
download_scope = state_manager.get_item('download_scope')
available_processors = [ get_file_name(file_path) for file_path in resolve_file_paths('facefusion/processors/modules') ]
processor_modules = get_processors_modules(available_processors)
common_modules = []
for processor_module in processor_modules:
for common_module in processor_module.get_common_modules():
if common_module not in common_modules:
common_modules.append(common_module)
for module in common_modules + processor_modules:
if hasattr(module, 'create_static_model_set'):
for model in module.create_static_model_set(state_manager.get_item('download_scope')).values():
for model in module.create_static_model_set(download_scope).values():
model_hash_set = model.get('hashes')
model_source_set = model.get('sources')
@@ -2,14 +2,13 @@ from typing import List, Optional
import numpy
from facefusion import state_manager
from facefusion.common_helper import get_first
from facefusion import face_store, state_manager
from facefusion.common_helper import get_first, get_middle
from facefusion.face_classifier import classify_face
from facefusion.face_detector import detect_faces, detect_faces_by_angle
from facefusion.face_helper import apply_nms, convert_to_face_landmark_5, estimate_face_angle, get_nms_threshold
from facefusion.face_helper import apply_nms, average_points, convert_to_face_landmark_5, estimate_face_angle, get_nms_threshold
from facefusion.face_landmarker import detect_face_landmark, estimate_face_landmark_68_5
from facefusion.face_recognizer import calculate_face_embedding
from facefusion.face_store import get_static_faces, set_static_faces
from facefusion.types import BoundingBox, Face, FaceLandmark5, FaceLandmarkSet, FaceScoreSet, Score, VisionFrame
@@ -47,7 +46,9 @@ def create_faces(vision_frame : VisionFrame, bounding_boxes : List[BoundingBox],
}
face_embedding, face_embedding_norm = calculate_face_embedding(vision_frame, face_landmark_set.get('5/68'))
gender, age, race = classify_face(vision_frame, face_landmark_set.get('5/68'))
faces.append(Face(
origin = 'detect',
bounding_box = bounding_box,
score_set = face_score_set,
landmark_set = face_landmark_set,
@@ -68,7 +69,102 @@ def get_one_face(faces : List[Face], position : int = 0) -> Optional[Face]:
return None
def get_average_face(faces : List[Face]) -> Optional[Face]:
def get_many_faces(vision_frames : List[VisionFrame]) -> List[Face]:
many_faces : List[Face] = []
for vision_frame in vision_frames:
if numpy.any(vision_frame):
all_bounding_boxes = []
all_face_scores = []
all_face_landmarks_5 = []
for face_detector_angle in state_manager.get_item('face_detector_angles'):
if face_detector_angle == 0:
bounding_boxes, face_scores, face_landmarks_5 = detect_faces(vision_frame)
else:
bounding_boxes, face_scores, face_landmarks_5 = detect_faces_by_angle(vision_frame, face_detector_angle)
all_bounding_boxes.extend(bounding_boxes)
all_face_scores.extend(face_scores)
all_face_landmarks_5.extend(face_landmarks_5)
if all_bounding_boxes and all_face_scores and all_face_landmarks_5 and state_manager.get_item('face_detector_score') > 0:
faces = create_faces(vision_frame, all_bounding_boxes, all_face_scores, all_face_landmarks_5)
if faces:
many_faces.extend(faces)
return many_faces
def get_static_faces(vision_frames : List[VisionFrame]) -> List[Face]:
many_faces : List[Face] = []
for vision_frame in vision_frames:
faces = face_store.get_faces(vision_frame)
if not faces:
with face_store.resolve_lock(vision_frame):
faces = face_store.get_faces(vision_frame)
if not faces:
faces = get_many_faces([ vision_frame ])
if faces:
face_store.set_faces(vision_frame, faces)
many_faces.extend(faces)
return many_faces
def refill_faces(faces : List[Optional[Face]]) -> List[Face]:
fill_faces = []
anchor_index_previous = -1
for index, face in enumerate(faces):
if face:
for gap_index in range(anchor_index_previous + 1, index):
average_factor = (gap_index - anchor_index_previous) / (index - anchor_index_previous)
average_face = average_face_geometry([faces[anchor_index_previous], face], average_factor)
fill_faces.append(average_face)
fill_faces.append(face)
anchor_index_previous = index
return fill_faces
def average_face_geometry(faces : List[Face], average_factor : float) -> Face:
face_first = get_first(faces)
face_middle = get_middle(faces)
face_anchor = face_middle
if average_factor < 0.5:
face_anchor = face_first
landmark_set : FaceLandmarkSet =\
{
'5': average_points(face_first.landmark_set.get('5'), face_middle.landmark_set.get('5'), average_factor),
'5/68': average_points(face_first.landmark_set.get('5/68'), face_middle.landmark_set.get('5/68'), average_factor),
'68': average_points(face_first.landmark_set.get('68'), face_middle.landmark_set.get('68'), average_factor),
'68/5': average_points(face_first.landmark_set.get('68/5'), face_middle.landmark_set.get('68/5'), average_factor)
}
return Face(
origin = 'refill',
bounding_box = average_points(face_first.bounding_box, face_middle.bounding_box, average_factor),
score_set = face_anchor.score_set,
landmark_set = landmark_set,
angle = estimate_face_angle(landmark_set.get('68/5')),
embedding = face_anchor.embedding,
embedding_norm = face_anchor.embedding_norm,
gender = face_anchor.gender,
age = face_anchor.age,
race = face_anchor.race
)
def average_face_identity(faces : List[Face]) -> Optional[Face]:
face_embeddings = []
face_embeddings_norm = []
@@ -80,6 +176,7 @@ def get_average_face(faces : List[Face]) -> Optional[Face]:
face_embeddings_norm.append(face.embedding_norm)
return Face(
origin = first_face.origin,
bounding_box = first_face.bounding_box,
score_set = first_face.score_set,
landmark_set = first_face.landmark_set,
@@ -93,37 +190,6 @@ def get_average_face(faces : List[Face]) -> Optional[Face]:
return None
def get_many_faces(vision_frames : List[VisionFrame]) -> List[Face]:
many_faces : List[Face] = []
for vision_frame in vision_frames:
if numpy.any(vision_frame):
static_faces = get_static_faces(vision_frame)
if static_faces:
many_faces.extend(static_faces)
else:
all_bounding_boxes = []
all_face_scores = []
all_face_landmarks_5 = []
for face_detector_angle in state_manager.get_item('face_detector_angles'):
if face_detector_angle == 0:
bounding_boxes, face_scores, face_landmarks_5 = detect_faces(vision_frame)
else:
bounding_boxes, face_scores, face_landmarks_5 = detect_faces_by_angle(vision_frame, face_detector_angle)
all_bounding_boxes.extend(bounding_boxes)
all_face_scores.extend(face_scores)
all_face_landmarks_5.extend(face_landmarks_5)
if all_bounding_boxes and all_face_scores and all_face_landmarks_5 and state_manager.get_item('face_detector_score') > 0:
faces = create_faces(vision_frame, all_bounding_boxes, all_face_scores, all_face_landmarks_5)
if faces:
many_faces.extend(faces)
set_static_faces(vision_frame, faces)
return many_faces
def scale_face(target_face : Face, target_vision_frame : VisionFrame, temp_vision_frame : VisionFrame) -> Face:
scale_x = temp_vision_frame.shape[1] / target_vision_frame.shape[1]
scale_y = temp_vision_frame.shape[0] / target_vision_frame.shape[0]
+20
View File
@@ -254,3 +254,23 @@ def merge_matrix(temp_matrices : List[Matrix]) -> Matrix:
matrix = numpy.dot(temp_matrix, matrix)
return matrix[:2, :]
def calculate_bounding_box_overlap(bounding_box_a : BoundingBox, bounding_box_b : BoundingBox) -> float:
intersection_x1 = max(bounding_box_a[0], bounding_box_b[0])
intersection_y1 = max(bounding_box_a[1], bounding_box_b[1])
intersection_x2 = min(bounding_box_a[2], bounding_box_b[2])
intersection_y2 = min(bounding_box_a[3], bounding_box_b[3])
intersection = max(0, intersection_x2 - intersection_x1) * max(0, intersection_y2 - intersection_y1)
bounding_box_area = (bounding_box_a[2] - bounding_box_a[0]) * (bounding_box_a[3] - bounding_box_a[1])
reference_bounding_box_area = (bounding_box_b[2] - bounding_box_b[0]) * (bounding_box_b[3] - bounding_box_b[1])
union = bounding_box_area + reference_bounding_box_area - intersection
if union > 0:
return intersection / union
return 0.0
def average_points(points_previous : Points, points_next : Points, average_factor : float) -> Points:
return points_previous * (1 - average_factor) + points_next * average_factor
+41 -16
View File
@@ -2,26 +2,35 @@ from typing import List
import numpy
import facefusion.choices
from facefusion import state_manager
from facefusion.face_analyser import get_many_faces, get_one_face
from facefusion.common_helper import get_first, get_middle
from facefusion.face_creator import get_one_face, get_static_faces
from facefusion.face_tracker import track_faces
from facefusion.types import Face, FaceSelectorOrder, Gender, Race, Score, VisionFrame
def select_faces(reference_vision_frame : VisionFrame, target_vision_frame : VisionFrame) -> List[Face]:
target_faces = get_many_faces([ target_vision_frame ])
def select_faces(reference_vision_frame : VisionFrame, source_vision_frames : List[VisionFrame], target_vision_frames : List[VisionFrame]) -> List[Face]:
source_faces = get_static_faces(source_vision_frames)
if state_manager.get_item('face_tracker_score') > 0:
target_faces = track_faces(target_vision_frames, state_manager.get_item('face_tracker_score'))
else:
target_faces = get_static_faces([ get_middle(target_vision_frames) ])
if state_manager.get_item('face_selector_mode') == 'many':
return sort_and_filter_faces(target_faces)
return sort_and_filter_faces(source_faces, target_faces)
if state_manager.get_item('face_selector_mode') == 'one':
target_face = get_one_face(sort_and_filter_faces(target_faces))
target_face = get_one_face(sort_and_filter_faces(source_faces, target_faces))
if target_face:
return [ target_face ]
if state_manager.get_item('face_selector_mode') == 'reference':
reference_faces = get_many_faces([ reference_vision_frame ])
reference_faces = sort_and_filter_faces(reference_faces)
reference_faces = get_static_faces([ reference_vision_frame ])
reference_faces = sort_and_filter_faces(source_faces, reference_faces)
reference_face = get_one_face(reference_faces, state_manager.get_item('reference_face_position'))
if reference_face:
match_faces = find_match_faces([ reference_face ], target_faces, state_manager.get_item('reference_face_distance'))
return match_faces
@@ -53,17 +62,33 @@ def calculate_face_distance(face : Face, reference_face : Face) -> float:
return 0
def sort_and_filter_faces(faces : List[Face]) -> List[Face]:
if faces:
def sort_and_filter_faces(source_faces : List[Face], target_faces : List[Face]) -> List[Face]:
if target_faces:
if state_manager.get_item('face_selector_order'):
faces = sort_faces_by_order(faces, state_manager.get_item('face_selector_order'))
if state_manager.get_item('face_selector_gender'):
faces = filter_faces_by_gender(faces, state_manager.get_item('face_selector_gender'))
if state_manager.get_item('face_selector_race'):
faces = filter_faces_by_race(faces, state_manager.get_item('face_selector_race'))
target_faces = sort_faces_by_order(target_faces, state_manager.get_item('face_selector_order'))
face_selector_gender = state_manager.get_item('face_selector_gender')
face_selector_race = state_manager.get_item('face_selector_race')
if source_faces and face_selector_gender == 'auto' or face_selector_race == 'auto':
source_face = get_first(sort_faces_by_order(source_faces, 'large-small'))
if source_face:
if face_selector_gender == 'auto':
face_selector_gender = source_face.gender
if face_selector_race == 'auto':
face_selector_race = source_face.race
if face_selector_gender in facefusion.choices.genders:
target_faces = filter_faces_by_gender(target_faces, face_selector_gender)
if face_selector_race in facefusion.choices.races:
target_faces = filter_faces_by_race(target_faces, face_selector_race)
if state_manager.get_item('face_selector_age_start') or state_manager.get_item('face_selector_age_end'):
faces = filter_faces_by_age(faces, state_manager.get_item('face_selector_age_start'), state_manager.get_item('face_selector_age_end'))
return faces
target_faces = filter_faces_by_age(target_faces, state_manager.get_item('face_selector_age_start'), state_manager.get_item('face_selector_age_end'))
return target_faces
def sort_faces_by_order(faces : List[Face], order : FaceSelectorOrder) -> List[Face]:
+29 -15
View File
@@ -1,28 +1,42 @@
import threading
from typing import List, Optional
import numpy
from facefusion.hash_helper import create_hash
from facefusion.types import Face, FaceStore, VisionFrame
FACE_STORE : FaceStore =\
{
'static_faces': {}
}
FACE_STORE : FaceStore = {}
def get_face_store() -> FaceStore:
return FACE_STORE
def get_faces(vision_frame : VisionFrame) -> Optional[List[Face]]:
if numpy.any(vision_frame):
vision_hash = create_hash(vision_frame.tobytes())
if FACE_STORE.get(vision_hash):
return FACE_STORE.get(vision_hash).get('faces')
return None
def get_static_faces(vision_frame : VisionFrame) -> Optional[List[Face]]:
vision_hash = create_hash(vision_frame.tobytes())
return FACE_STORE.get('static_faces').get(vision_hash)
def set_faces(vision_frame : VisionFrame, faces : List[Face]) -> None:
if numpy.any(vision_frame):
vision_hash = create_hash(vision_frame.tobytes())
FACE_STORE.setdefault(vision_hash,
{
'lock': threading.Lock()
})['faces'] = faces
def set_static_faces(vision_frame : VisionFrame, faces : List[Face]) -> None:
vision_hash = create_hash(vision_frame.tobytes())
if vision_hash:
FACE_STORE['static_faces'][vision_hash] = faces
def resolve_lock(vision_frame : VisionFrame) -> threading.Lock:
if numpy.any(vision_frame):
vision_hash = create_hash(vision_frame.tobytes())
return FACE_STORE.setdefault(vision_hash,
{
'lock': threading.Lock()
}).get('lock')
return threading.Lock()
def clear_static_faces() -> None:
FACE_STORE['static_faces'].clear()
def clear_faces() -> None:
FACE_STORE.clear()
+61
View File
@@ -0,0 +1,61 @@
from typing import List
from facefusion.common_helper import get_first, get_last
from facefusion.face_creator import get_static_faces, refill_faces
from facefusion.face_helper import calculate_bounding_box_overlap
from facefusion.types import Face, FaceTrack, Score, VisionFrame
def track_faces(vision_frames : List[VisionFrame], score : Score) -> List[Face]:
target_index = len(vision_frames) // 2
face_tracks = create_face_tracks(vision_frames, score)
temp_faces = []
for face_track in face_tracks:
track_indices = sorted(face_track)
track_index_first = get_first(track_indices)
track_index_last = get_last(track_indices)
track_range = range(track_index_first, track_index_last + 1)
if target_index in track_range:
fill_faces = []
for index in track_range:
fill_faces.append(face_track.get(index))
temp_faces.append(refill_faces(fill_faces)[target_index - track_index_first])
return temp_faces
def create_face_tracks(vision_frames : List[VisionFrame], score : Score) -> List[FaceTrack]:
face_tracks : List[FaceTrack] = []
for frame_index, vision_frame in enumerate(vision_frames):
for face in get_static_faces([ vision_frame ]):
face_track = select_face_track(face_tracks, face, score)
if face_track:
face_track[frame_index] = face
else:
face_tracks.append(
{
frame_index : face
})
return face_tracks
def select_face_track(face_tracks : List[FaceTrack], face : Face, score : Score) -> FaceTrack:
select_track : FaceTrack = {}
select_score = score
for face_track in face_tracks:
track_face = face_track.get(get_last(face_track))
track_score = calculate_bounding_box_overlap(face.bounding_box, track_face.bounding_box)
if track_score > select_score:
select_score = track_score
select_track = face_track
return select_track
+14 -5
View File
@@ -9,7 +9,7 @@ from tqdm import tqdm
import facefusion.choices
from facefusion import ffmpeg_builder, logger, process_manager, state_manager, translator
from facefusion.filesystem import get_file_format, remove_file
from facefusion.temp_helper import get_temp_file_path, get_temp_frames_pattern
from facefusion.temp_helper import get_temp_file_path, get_temp_frame_pattern
from facefusion.types import AudioBuffer, AudioEncoder, Command, EncoderSet, Fps, Resolution, UpdateProgress, VideoEncoder, VideoFormat
from facefusion.vision import detect_video_duration, detect_video_fps, pack_resolution, predict_video_frame_total
@@ -109,7 +109,7 @@ def get_available_encoder_set() -> EncoderSet:
def extract_frames(target_path : str, temp_video_resolution : Resolution, temp_video_fps : Fps, trim_frame_start : int, trim_frame_end : int) -> bool:
extract_frame_total = predict_video_frame_total(target_path, temp_video_fps, trim_frame_start, trim_frame_end)
temp_frames_pattern = get_temp_frames_pattern(target_path, '%08d')
temp_frame_pattern = get_temp_frame_pattern(target_path, '%08d')
commands = ffmpeg_builder.chain(
ffmpeg_builder.set_input(target_path),
ffmpeg_builder.set_media_resolution(pack_resolution(temp_video_resolution)),
@@ -117,7 +117,8 @@ def extract_frames(target_path : str, temp_video_resolution : Resolution, temp_v
ffmpeg_builder.enforce_pixel_format('rgb24'),
ffmpeg_builder.select_frame_range(trim_frame_start, trim_frame_end, temp_video_fps),
ffmpeg_builder.prevent_frame_drop(),
ffmpeg_builder.set_output(temp_frames_pattern)
ffmpeg_builder.set_start_number(trim_frame_start),
ffmpeg_builder.set_output(temp_frame_pattern)
)
with tqdm(total = extract_frame_total, desc = translator.get('extracting'), unit = 'frame', ascii = ' =', disable = state_manager.get_item('log_level') in [ 'warn', 'error' ]) as progress:
@@ -173,6 +174,7 @@ def restore_audio(target_path : str, output_path : str, trim_frame_start : int,
temp_video_path = get_temp_file_path(target_path)
temp_video_format = cast(VideoFormat, get_file_format(temp_video_path))
temp_video_duration = detect_video_duration(temp_video_path)
output_video_format = cast(VideoFormat, get_file_format(output_path))
output_audio_encoder = fix_audio_encoder(temp_video_format, output_audio_encoder)
commands = ffmpeg_builder.chain(
@@ -186,6 +188,7 @@ def restore_audio(target_path : str, output_path : str, trim_frame_start : int,
ffmpeg_builder.select_media_stream('0:v:0'),
ffmpeg_builder.select_media_stream('1:a:0'),
ffmpeg_builder.set_video_duration(temp_video_duration),
ffmpeg_builder.set_faststart(output_video_format),
ffmpeg_builder.force_output(output_path)
)
return run_ffmpeg(commands).returncode == 0
@@ -198,6 +201,7 @@ def replace_audio(target_path : str, audio_path : str, output_path : str) -> boo
temp_video_path = get_temp_file_path(target_path)
temp_video_format = cast(VideoFormat, get_file_format(temp_video_path))
temp_video_duration = detect_video_duration(temp_video_path)
output_video_format = cast(VideoFormat, get_file_format(output_path))
output_audio_encoder = fix_audio_encoder(temp_video_format, output_audio_encoder)
commands = ffmpeg_builder.chain(
@@ -208,6 +212,7 @@ def replace_audio(target_path : str, audio_path : str, output_path : str) -> boo
ffmpeg_builder.set_audio_quality(output_audio_encoder, output_audio_quality),
ffmpeg_builder.set_audio_volume(output_audio_volume),
ffmpeg_builder.set_video_duration(temp_video_duration),
ffmpeg_builder.set_faststart(output_video_format),
ffmpeg_builder.force_output(output_path)
)
return run_ffmpeg(commands).returncode == 0
@@ -220,14 +225,16 @@ def merge_video(target_path : str, temp_video_fps : Fps, output_video_resolution
merge_frame_total = predict_video_frame_total(target_path, output_video_fps, trim_frame_start, trim_frame_end)
temp_video_path = get_temp_file_path(target_path)
temp_video_format = cast(VideoFormat, get_file_format(temp_video_path))
temp_frames_pattern = get_temp_frames_pattern(target_path, '%08d')
temp_frame_pattern = get_temp_frame_pattern(target_path, '%08d')
output_video_encoder = fix_video_encoder(temp_video_format, output_video_encoder)
commands = ffmpeg_builder.chain(
ffmpeg_builder.set_input_fps(temp_video_fps),
ffmpeg_builder.set_input(temp_frames_pattern),
ffmpeg_builder.set_start_number(trim_frame_start),
ffmpeg_builder.set_input(temp_frame_pattern),
ffmpeg_builder.set_media_resolution(pack_resolution(output_video_resolution)),
ffmpeg_builder.set_video_encoder(output_video_encoder),
ffmpeg_builder.set_video_tag(output_video_encoder, temp_video_format),
ffmpeg_builder.set_video_quality(output_video_encoder, output_video_quality),
ffmpeg_builder.set_video_preset(output_video_encoder, output_video_preset),
ffmpeg_builder.concat(
@@ -254,11 +261,13 @@ def concat_video(output_path : str, temp_output_paths : List[str]) -> bool:
concat_video_file.close()
output_path = os.path.abspath(output_path)
output_video_format = cast(VideoFormat, get_file_format(output_path))
commands = ffmpeg_builder.chain(
ffmpeg_builder.unsafe_concat(),
ffmpeg_builder.set_input(concat_video_file.name),
ffmpeg_builder.copy_video_encoder(),
ffmpeg_builder.copy_audio_encoder(),
ffmpeg_builder.set_faststart(output_video_format),
ffmpeg_builder.force_output(output_path)
)
process = run_ffmpeg(commands)
+17 -1
View File
@@ -5,7 +5,7 @@ from typing import List, Optional
import numpy
from facefusion.filesystem import get_file_format
from facefusion.types import AudioEncoder, Command, CommandSet, Duration, Fps, StreamMode, VideoEncoder, VideoPreset
from facefusion.types import AudioEncoder, Command, CommandSet, Duration, Fps, StreamMode, VideoEncoder, VideoFormat, VideoPreset
def run(commands : List[Command]) -> List[Command]:
@@ -51,6 +51,10 @@ def set_input_fps(input_fps : Fps) -> List[Command]:
return [ '-r', str(input_fps) ]
def set_start_number(frame_number : int) -> List[Command]:
return [ '-start_number', str(frame_number) ]
def set_output(output_path : str) -> List[Command]:
return [ output_path ]
@@ -187,6 +191,18 @@ def copy_video_encoder() -> List[Command]:
return set_video_encoder('copy')
def set_faststart(video_format : VideoFormat) -> List[Command]:
if video_format in [ 'm4v', 'mov', 'mp4' ]:
return [ '-movflags', '+faststart' ]
return []
def set_video_tag(video_encoder : VideoEncoder, video_format : VideoFormat) -> List[Command]:
if video_format in [ 'm4v', 'mov', 'mp4' ] and video_encoder in [ 'libx265', 'hevc_nvenc', 'hevc_amf', 'hevc_qsv', 'hevc_videotoolbox' ]:
return [ '-tag:v', 'hvc1' ]
return []
def set_video_quality(video_encoder : VideoEncoder, video_quality : int) -> List[Command]:
if video_encoder in [ 'libx264', 'libx264rgb', 'libx265' ]:
video_compression = numpy.round(numpy.interp(video_quality, [ 0, 100 ], [ 51, 0 ])).astype(int).item()
+19 -12
View File
@@ -1,5 +1,6 @@
import importlib
import random
from functools import lru_cache
from time import sleep, time
from typing import List
@@ -12,7 +13,7 @@ from facefusion.execution import create_inference_providers, has_execution_provi
from facefusion.exit_helper import fatal_exit
from facefusion.filesystem import get_file_name, is_file
from facefusion.time_helper import calculate_end_time
from facefusion.types import DownloadSet, ExecutionProvider, InferencePool, InferencePoolSet
from facefusion.types import DownloadSet, ExecutionProvider, InferencePool, InferencePoolSet, InferenceProvider
INFERENCE_POOL_SET : InferencePoolSet =\
{
@@ -25,7 +26,7 @@ def get_inference_pool(module_name : str, model_names : List[str], model_source_
while process_manager.is_checking():
sleep(0.5)
execution_device_ids = state_manager.get_item('execution_device_ids')
execution_providers = resolve_execution_providers(module_name)
execution_providers = state_manager.get_item('execution_providers')
app_context = detect_app_context()
for execution_device_id in execution_device_ids:
@@ -36,26 +37,27 @@ def get_inference_pool(module_name : str, model_names : List[str], model_source_
if app_context == 'ui' and INFERENCE_POOL_SET.get('cli').get(inference_context):
INFERENCE_POOL_SET['ui'][inference_context] = INFERENCE_POOL_SET.get('cli').get(inference_context)
if not INFERENCE_POOL_SET.get(app_context).get(inference_context):
INFERENCE_POOL_SET[app_context][inference_context] = create_inference_pool(model_source_set, execution_device_id, execution_providers)
inference_providers = resolve_static_inference_providers(module_name, execution_device_id)
INFERENCE_POOL_SET[app_context][inference_context] = create_inference_pool(model_source_set, inference_providers)
current_inference_context = get_inference_context(module_name, model_names, random.choice(execution_device_ids), execution_providers)
return INFERENCE_POOL_SET.get(app_context).get(current_inference_context)
def create_inference_pool(model_source_set : DownloadSet, execution_device_id : int, execution_providers : List[ExecutionProvider]) -> InferencePool:
def create_inference_pool(model_source_set : DownloadSet, inference_providers : List[InferenceProvider]) -> InferencePool:
inference_pool : InferencePool = {}
for model_name in model_source_set.keys():
model_path = model_source_set.get(model_name).get('path')
if is_file(model_path):
inference_pool[model_name] = create_inference_session(model_path, execution_device_id, execution_providers)
inference_pool[model_name] = create_inference_session(model_path, inference_providers)
return inference_pool
def clear_inference_pool(module_name : str, model_names : List[str]) -> None:
execution_device_ids = state_manager.get_item('execution_device_ids')
execution_providers = resolve_execution_providers(module_name)
execution_providers = state_manager.get_item('execution_providers')
app_context = detect_app_context()
if is_windows() and has_execution_provider('directml'):
@@ -67,12 +69,11 @@ def clear_inference_pool(module_name : str, model_names : List[str]) -> None:
del INFERENCE_POOL_SET[app_context][inference_context]
def create_inference_session(model_path : str, execution_device_id : int, execution_providers : List[ExecutionProvider]) -> InferenceSession:
def create_inference_session(model_path : str, inference_providers : List[InferenceProvider]) -> InferenceSession:
model_file_name = get_file_name(model_path)
start_time = time()
try:
inference_providers = create_inference_providers(execution_device_id, execution_providers)
inference_session = InferenceSession(model_path, providers = inference_providers)
logger.debug(translator.get('loading_model_succeeded').format(model_name = model_file_name, seconds = calculate_end_time(start_time)), __name__)
return inference_session
@@ -87,9 +88,15 @@ def get_inference_context(module_name : str, model_names : List[str], execution_
return inference_context
def resolve_execution_providers(module_name : str) -> List[ExecutionProvider]:
@lru_cache()
def resolve_static_inference_providers(module_name : str, execution_device_id : int) -> List[InferenceProvider]:
module = importlib.import_module(module_name)
execution_providers = state_manager.get_item('execution_providers')
if hasattr(module, 'resolve_execution_providers'):
return getattr(module, 'resolve_execution_providers')()
return state_manager.get_item('execution_providers')
if hasattr(module, 'resolve_inference_providers'):
inference_providers = getattr(module, 'resolve_inference_providers')()
if inference_providers:
return inference_providers
return create_inference_providers(execution_device_id, execution_providers)
+4 -4
View File
@@ -19,23 +19,23 @@ LOCALES =\
}
ONNXRUNTIME_SET =\
{
'default': ('onnxruntime', '1.24.4')
'default': ('onnxruntime', '1.26.0')
}
if is_windows() or is_linux():
ONNXRUNTIME_SET['cuda'] = ('onnxruntime-gpu', '1.24.4')
ONNXRUNTIME_SET['cuda'] = ('onnxruntime-gpu', '1.26.0')
ONNXRUNTIME_SET['openvino'] = ('onnxruntime-openvino', '1.24.1')
if is_windows():
ONNXRUNTIME_SET['directml'] = ('onnxruntime-directml', '1.24.4')
ONNXRUNTIME_SET['qnn'] = ('onnxruntime-qnn', '1.24.4')
if is_linux():
ONNXRUNTIME_SET['migraphx'] = ('onnxruntime-migraphx', '1.24.2')
ONNXRUNTIME_SET['migraphx'] = ('onnxruntime-migraphx', '1.25.0')
ONNXRUNTIME_SET['rocm'] = ('onnxruntime-rocm', '1.22.2.post1')
def cli() -> None:
signal.signal(signal.SIGINT, signal_exit)
program = ArgumentParser(formatter_class = partial(HelpFormatter, max_help_position = 50))
program.add_argument('--onnxruntime', help = LOCALES.get('install_dependency').format(dependency = 'onnxruntime'), choices = ONNXRUNTIME_SET.keys(), required = True)
program.add_argument('onnxruntime', help = LOCALES.get('install_dependency').format(dependency = 'onnxruntime'), choices = ONNXRUNTIME_SET.keys())
program.add_argument('--force-reinstall', help = LOCALES.get('force_reinstall'), action = 'store_true')
program.add_argument('--skip-conda', help = LOCALES.get('skip_conda'), action = 'store_true')
program.add_argument('-v', '--version', version = metadata.get('name') + ' ' + metadata.get('version'), action = 'version')
+2
View File
@@ -6,6 +6,7 @@ import facefusion.choices
from facefusion.filesystem import create_directory, get_file_name, is_directory, is_file, move_file, remove_directory, remove_file, resolve_file_pattern
from facefusion.jobs.job_helper import get_step_output_path
from facefusion.json import read_json, write_json
from facefusion.sanitizer import sanitize_job_id
from facefusion.time_helper import get_current_date_time
from facefusion.types import Args, Job, JobSet, JobStatus, JobStep, JobStepStatus
@@ -261,5 +262,6 @@ def find_job_path(job_id : str) -> Optional[str]:
def get_job_file_name(job_id : str) -> Optional[str]:
if job_id:
job_id = sanitize_job_id(job_id)
return job_id + '.json'
return None
+3 -2
View File
@@ -124,6 +124,7 @@ LOCALES : Locales =\
'reference_face_position': 'specify the position used to create the reference face',
'reference_face_distance': 'specify the similarity between the reference face and target face',
'reference_frame_number': 'specify the frame used to create the reference face',
'face_tracker_score': 'specify the minimum score to track a face',
'face_occluder_model': 'choose the model responsible for the occlusion mask',
'face_parser_model': 'choose the model responsible for the region mask',
'face_mask_types': 'mix and match different face mask types (choices: {choices})',
@@ -136,6 +137,7 @@ LOCALES : Locales =\
'trim_frame_end': 'specify the ending frame of the target video',
'temp_frame_format': 'specify the temporary resources format',
'keep_temp': 'keep the temporary resources after processing',
'target_frame_amount': 'specify the amount of frames around the target frame',
'output_image_quality': 'specify the image quality which translates to the image compression',
'output_image_scale': 'specify the image scale based on the target image',
'output_audio_encoder': 'specify the encoder used for the audio',
@@ -161,7 +163,6 @@ LOCALES : Locales =\
'execution_providers': 'inference using different providers (choices: {choices}, ...)',
'execution_thread_count': 'specify the amount of parallel threads while processing',
'video_memory_strategy': 'balance fast processing and low VRAM usage',
'system_memory_limit': 'limit the available RAM that can be used while processing',
'log_level': 'adjust the message severity displayed in the terminal',
'halt_on_error': 'halt the program once an error occurred',
'run': 'run the program',
@@ -224,6 +225,7 @@ LOCALES : Locales =\
'face_selector_mode_dropdown': 'FACE SELECTOR MODE',
'face_selector_order_dropdown': 'FACE SELECTOR ORDER',
'face_selector_race_dropdown': 'FACE SELECTOR RACE',
'face_tracker_score_slider': 'FACE TRACKER SCORE',
'face_occluder_model_dropdown': 'FACE OCCLUDER MODEL',
'face_parser_model_dropdown': 'FACE PARSER MODEL',
'voice_extractor_model_dropdown': 'VOICE EXTRACTOR MODEL',
@@ -257,7 +259,6 @@ LOCALES : Locales =\
'source_file': 'SOURCE',
'start_button': 'START',
'stop_button': 'STOP',
'system_memory_limit_slider': 'SYSTEM MEMORY LIMIT',
'target_file': 'TARGET',
'temp_frame_format_dropdown': 'TEMP FRAME FORMAT',
'terminal_textbox': 'TERMINAL',
-21
View File
@@ -1,21 +0,0 @@
from facefusion.common_helper import is_macos, is_windows
if is_windows():
import ctypes
else:
import resource
def limit_system_memory(system_memory_limit : int = 1) -> bool:
if is_macos():
system_memory_limit = system_memory_limit * (1024 ** 6)
else:
system_memory_limit = system_memory_limit * (1024 ** 3)
try:
if is_windows():
ctypes.windll.kernel32.SetProcessWorkingSetSize(-1, ctypes.c_size_t(system_memory_limit), ctypes.c_size_t(system_memory_limit)) #type:ignore[attr-defined]
else:
resource.setrlimit(resource.RLIMIT_DATA, (system_memory_limit, system_memory_limit))
return True
except Exception:
return False
+1 -1
View File
@@ -4,7 +4,7 @@ METADATA =\
{
'name': 'FaceFusion',
'description': 'Industry leading face manipulation platform',
'version': '3.6.1',
'version': '3.7.0',
'license': 'OpenRAIL-AS',
'author': 'Henry Ruhs',
'url': 'https://facefusion.io'
+1
View File
@@ -12,6 +12,7 @@ PROCESSORS_METHODS =\
'clear_inference_pool',
'register_args',
'apply_args',
'get_common_modules',
'pre_check',
'pre_process',
'post_process',
@@ -1,5 +1,7 @@
from argparse import ArgumentParser
from functools import lru_cache
from types import ModuleType
from typing import List
import cv2
import numpy
@@ -8,10 +10,9 @@ import facefusion.choices
import facefusion.jobs.job_manager
import facefusion.jobs.job_store
from facefusion import config, content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer, inference_manager, logger, state_manager, translator, video_manager
from facefusion.common_helper import create_int_metavar, is_macos
from facefusion.common_helper import create_int_metavar, get_middle
from facefusion.download import conditional_download_hashes, conditional_download_sources, resolve_download_url
from facefusion.execution import has_execution_provider
from facefusion.face_analyser import scale_face
from facefusion.face_creator import scale_face
from facefusion.face_helper import merge_matrix, paste_back, scale_face_landmark_5, warp_face_by_face_landmark_5
from facefusion.face_masker import create_box_mask, create_occlusion_mask
from facefusion.face_selector import select_faces
@@ -22,7 +23,7 @@ from facefusion.processors.types import ProcessorOutputs
from facefusion.program_helper import find_argument_group
from facefusion.thread_helper import thread_semaphore
from facefusion.types import ApplyStateItem, Args, DownloadScope, Face, InferencePool, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import match_frame_color, read_static_image, read_static_video_frame
from facefusion.vision import match_frame_color, read_static_image, read_static_video_chunk, read_static_video_frame
@lru_cache()
@@ -134,10 +135,18 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('age_modifier_direction', args.get('age_modifier_direction'))
def get_common_modules() -> List[ModuleType]:
return [ content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer ]
def pre_check() -> bool:
model_hash_set = get_model_options().get('hashes')
model_source_set = get_model_options().get('sources')
for common_module in get_common_modules():
if not common_module.pre_check():
return False
return conditional_download_hashes(model_hash_set) and conditional_download_sources(model_source_set)
@@ -157,16 +166,15 @@ def pre_process(mode : ProcessMode) -> bool:
def post_process() -> None:
read_static_image.cache_clear()
read_static_video_frame.cache_clear()
read_static_video_chunk.cache_clear()
video_manager.clear_video_pool()
if state_manager.get_item('video_memory_strategy') in [ 'strict', 'moderate' ]:
clear_inference_pool()
if state_manager.get_item('video_memory_strategy') == 'strict':
content_analyser.clear_inference_pool()
face_classifier.clear_inference_pool()
face_detector.clear_inference_pool()
face_landmarker.clear_inference_pool()
face_masker.clear_inference_pool()
face_recognizer.clear_inference_pool()
for common_module in get_common_modules():
common_module.clear_inference_pool()
def modify_age(target_face : Face, temp_vision_frame : VisionFrame) -> VisionFrame:
@@ -231,9 +239,6 @@ def forward(crop_vision_frame : VisionFrame, extend_vision_frame : VisionFrame,
age_modifier = get_inference_pool().get('age_modifier')
age_modifier_inputs = {}
if is_macos() and has_execution_provider('coreml'):
age_modifier.set_providers([ facefusion.choices.execution_provider_set.get('cpu') ])
for age_modifier_input in age_modifier.get_inputs():
if age_modifier_input.name == 'target':
age_modifier_inputs[age_modifier_input.name] = crop_vision_frame
@@ -280,10 +285,13 @@ def normalize_extend_frame(extend_vision_frame : VisionFrame) -> VisionFrame:
def process_frame(inputs : AgeModifierInputs) -> ProcessorOutputs:
reference_vision_frame = inputs.get('reference_vision_frame')
target_vision_frame = inputs.get('target_vision_frame')
source_vision_frames = inputs.get('source_vision_frames')
target_vision_frames = inputs.get('target_vision_frames')
temp_vision_frame = inputs.get('temp_vision_frame')
temp_vision_mask = inputs.get('temp_vision_mask')
target_faces = select_faces(reference_vision_frame, target_vision_frame)
target_vision_frame = get_middle(target_vision_frames)
target_faces = select_faces(reference_vision_frame, source_vision_frames, target_vision_frames)
if target_faces:
for target_face in target_faces:
@@ -1,4 +1,4 @@
from typing import Any, Literal, TypeAlias, TypedDict
from typing import Any, List, Literal, TypeAlias, TypedDict
from numpy.typing import NDArray
@@ -7,7 +7,8 @@ from facefusion.types import Mask, VisionFrame
AgeModifierInputs = TypedDict('AgeModifierInputs',
{
'reference_vision_frame' : VisionFrame,
'target_vision_frame' : VisionFrame,
'source_vision_frames' : List[VisionFrame],
'target_vision_frames' : List[VisionFrame],
'temp_vision_frame' : VisionFrame,
'temp_vision_mask' : Mask
})
@@ -1,10 +1,12 @@
from argparse import ArgumentParser
from functools import lru_cache, partial
from types import ModuleType
from typing import List, Tuple
import cv2
import numpy
import facefusion.choices
import facefusion.jobs.job_manager
import facefusion.jobs.job_store
from facefusion import config, content_analyser, inference_manager, logger, state_manager, translator, video_manager
@@ -19,8 +21,8 @@ from facefusion.processors.types import ProcessorOutputs
from facefusion.program_helper import find_argument_group
from facefusion.sanitizer import sanitize_int_range
from facefusion.thread_helper import thread_semaphore
from facefusion.types import ApplyStateItem, Args, DownloadScope, ExecutionProvider, InferencePool, Mask, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import read_static_image, read_static_video_frame
from facefusion.types import ApplyStateItem, Args, DownloadScope, InferencePool, InferenceProvider, Mask, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import read_static_image, read_static_video_chunk, read_static_video_frame
@lru_cache()
@@ -477,12 +479,13 @@ def clear_inference_pool() -> None:
inference_manager.clear_inference_pool(__name__, model_names)
def resolve_execution_providers() -> List[ExecutionProvider]:
def resolve_inference_providers() -> List[InferenceProvider]:
model_type = get_model_options().get('type')
if is_macos() and has_execution_provider('coreml') or is_windows() and has_execution_provider('directml') and model_type == 'corridor_key':
return [ 'cpu' ]
return state_manager.get_item('execution_providers')
return [ facefusion.choices.execution_provider_set.get('cpu') ]
return []
def get_model_options() -> ModelOptions:
@@ -505,10 +508,18 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('background_remover_despill_color', normalize_color(args.get('background_remover_despill_color')))
def get_common_modules() -> List[ModuleType]:
return [ content_analyser ]
def pre_check() -> bool:
model_hash_set = get_model_options().get('hashes')
model_source_set = get_model_options().get('sources')
for common_module in get_common_modules():
if not common_module.pre_check():
return False
return conditional_download_hashes(model_hash_set) and conditional_download_sources(model_source_set)
@@ -528,11 +539,15 @@ def pre_process(mode : ProcessMode) -> bool:
def post_process() -> None:
read_static_image.cache_clear()
read_static_video_frame.cache_clear()
read_static_video_chunk.cache_clear()
video_manager.clear_video_pool()
if state_manager.get_item('video_memory_strategy') in [ 'strict', 'moderate' ]:
clear_inference_pool()
if state_manager.get_item('video_memory_strategy') == 'strict':
content_analyser.clear_inference_pool()
for common_module in get_common_modules():
common_module.clear_inference_pool()
def remove_background(temp_vision_frame : VisionFrame) -> Tuple[VisionFrame, Mask]:
@@ -1,10 +1,10 @@
from typing import Literal, TypedDict
from typing import List, Literal, TypedDict
from facefusion.types import Mask, VisionFrame
BackgroundRemoverInputs = TypedDict('BackgroundRemoverInputs',
{
'target_vision_frame' : VisionFrame,
'target_vision_frames' : List[VisionFrame],
'temp_vision_frame' : VisionFrame,
'temp_vision_mask' : Mask
})
@@ -1,6 +1,7 @@
from argparse import ArgumentParser
from functools import lru_cache
from typing import Tuple
from types import ModuleType
from typing import List, Tuple
import cv2
import numpy
@@ -9,9 +10,9 @@ from cv2.typing import Size
import facefusion.jobs.job_manager
import facefusion.jobs.job_store
from facefusion import config, content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer, inference_manager, logger, state_manager, translator, video_manager
from facefusion.common_helper import create_int_metavar
from facefusion.common_helper import create_int_metavar, get_middle
from facefusion.download import conditional_download_hashes, conditional_download_sources, resolve_download_url_by_provider
from facefusion.face_analyser import scale_face
from facefusion.face_creator import scale_face
from facefusion.face_helper import paste_back, warp_face_by_face_landmark_5
from facefusion.face_masker import create_area_mask, create_box_mask, create_occlusion_mask, create_region_mask
from facefusion.face_selector import select_faces
@@ -22,7 +23,7 @@ from facefusion.processors.types import ProcessorOutputs
from facefusion.program_helper import find_argument_group
from facefusion.thread_helper import thread_semaphore
from facefusion.types import ApplyStateItem, Args, DownloadScope, Face, InferencePool, Mask, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import conditional_match_frame_color, read_static_image, read_static_video_frame
from facefusion.vision import conditional_match_frame_color, read_static_image, read_static_video_chunk, read_static_video_frame
@lru_cache()
@@ -286,10 +287,18 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('deep_swapper_morph', args.get('deep_swapper_morph'))
def get_common_modules() -> List[ModuleType]:
return [ content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer ]
def pre_check() -> bool:
model_hash_set = get_model_options().get('hashes')
model_source_set = get_model_options().get('sources')
for common_module in get_common_modules():
if not common_module.pre_check():
return False
if model_hash_set and model_source_set:
return conditional_download_hashes(model_hash_set) and conditional_download_sources(model_source_set)
return True
@@ -311,16 +320,15 @@ def pre_process(mode : ProcessMode) -> bool:
def post_process() -> None:
read_static_image.cache_clear()
read_static_video_frame.cache_clear()
read_static_video_chunk.cache_clear()
video_manager.clear_video_pool()
if state_manager.get_item('video_memory_strategy') in [ 'strict', 'moderate' ]:
clear_inference_pool()
if state_manager.get_item('video_memory_strategy') == 'strict':
content_analyser.clear_inference_pool()
face_classifier.clear_inference_pool()
face_detector.clear_inference_pool()
face_landmarker.clear_inference_pool()
face_masker.clear_inference_pool()
face_recognizer.clear_inference_pool()
for common_module in get_common_modules():
common_module.clear_inference_pool()
def swap_face(target_face : Face, temp_vision_frame : VisionFrame) -> VisionFrame:
@@ -411,10 +419,13 @@ def prepare_crop_mask(crop_source_mask : Mask, crop_target_mask : Mask) -> Mask:
def process_frame(inputs : DeepSwapperInputs) -> ProcessorOutputs:
reference_vision_frame = inputs.get('reference_vision_frame')
target_vision_frame = inputs.get('target_vision_frame')
source_vision_frames = inputs.get('source_vision_frames')
target_vision_frames = inputs.get('target_vision_frames')
temp_vision_frame = inputs.get('temp_vision_frame')
temp_vision_mask = inputs.get('temp_vision_mask')
target_faces = select_faces(reference_vision_frame, target_vision_frame)
target_vision_frame = get_middle(target_vision_frames)
target_faces = select_faces(reference_vision_frame, source_vision_frames, target_vision_frames)
if target_faces:
for target_face in target_faces:
@@ -1,4 +1,4 @@
from typing import Any, TypeAlias, TypedDict
from typing import Any, List, TypeAlias, TypedDict
from numpy.typing import NDArray
@@ -7,7 +7,8 @@ from facefusion.types import Mask, VisionFrame
DeepSwapperInputs = TypedDict('DeepSwapperInputs',
{
'reference_vision_frame' : VisionFrame,
'target_vision_frame' : VisionFrame,
'source_vision_frames' : List[VisionFrame],
'target_vision_frames' : List[VisionFrame],
'temp_vision_frame' : VisionFrame,
'temp_vision_mask' : Mask
})
@@ -1,6 +1,7 @@
from argparse import ArgumentParser
from functools import lru_cache
from typing import Tuple
from types import ModuleType
from typing import List, Tuple
import cv2
import numpy
@@ -8,9 +9,9 @@ import numpy
import facefusion.jobs.job_manager
import facefusion.jobs.job_store
from facefusion import config, content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer, inference_manager, logger, state_manager, translator, video_manager
from facefusion.common_helper import create_int_metavar
from facefusion.common_helper import create_int_metavar, get_middle
from facefusion.download import conditional_download_hashes, conditional_download_sources, resolve_download_url
from facefusion.face_analyser import scale_face
from facefusion.face_creator import scale_face
from facefusion.face_helper import paste_back, warp_face_by_face_landmark_5
from facefusion.face_masker import create_box_mask, create_occlusion_mask
from facefusion.face_selector import select_faces
@@ -22,7 +23,7 @@ from facefusion.processors.types import LivePortraitExpression, LivePortraitFeat
from facefusion.program_helper import find_argument_group
from facefusion.thread_helper import conditional_thread_semaphore, thread_semaphore
from facefusion.types import ApplyStateItem, Args, DownloadScope, Face, InferencePool, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import read_static_image, read_static_video_frame
from facefusion.vision import read_static_image, read_static_video_chunk, read_static_video_frame
@lru_cache()
@@ -111,10 +112,18 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('expression_restorer_areas', args.get('expression_restorer_areas'))
def get_common_modules() -> List[ModuleType]:
return [ content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer ]
def pre_check() -> bool:
model_hash_set = get_model_options().get('hashes')
model_source_set = get_model_options().get('sources')
for common_module in get_common_modules():
if not common_module.pre_check():
return False
return conditional_download_hashes(model_hash_set) and conditional_download_sources(model_source_set)
@@ -137,16 +146,15 @@ def pre_process(mode : ProcessMode) -> bool:
def post_process() -> None:
read_static_image.cache_clear()
read_static_video_frame.cache_clear()
read_static_video_chunk.cache_clear()
video_manager.clear_video_pool()
if state_manager.get_item('video_memory_strategy') in [ 'strict', 'moderate' ]:
clear_inference_pool()
if state_manager.get_item('video_memory_strategy') == 'strict':
content_analyser.clear_inference_pool()
face_classifier.clear_inference_pool()
face_detector.clear_inference_pool()
face_landmarker.clear_inference_pool()
face_masker.clear_inference_pool()
face_recognizer.clear_inference_pool()
for common_module in get_common_modules():
common_module.clear_inference_pool()
def restore_expression(target_face : Face, target_vision_frame : VisionFrame, temp_vision_frame : VisionFrame) -> VisionFrame:
@@ -257,10 +265,13 @@ def normalize_crop_frame(crop_vision_frame : VisionFrame) -> VisionFrame:
def process_frame(inputs : ExpressionRestorerInputs) -> ProcessorOutputs:
reference_vision_frame = inputs.get('reference_vision_frame')
target_vision_frame = inputs.get('target_vision_frame')
source_vision_frames = inputs.get('source_vision_frames')
target_vision_frames = inputs.get('target_vision_frames')
temp_vision_frame = inputs.get('temp_vision_frame')
temp_vision_mask = inputs.get('temp_vision_mask')
target_faces = select_faces(reference_vision_frame, target_vision_frame)
target_vision_frame = get_middle(target_vision_frames)
target_faces = select_faces(reference_vision_frame, source_vision_frames, target_vision_frames)
if target_faces:
for target_face in target_faces:
@@ -6,7 +6,7 @@ ExpressionRestorerInputs = TypedDict('ExpressionRestorerInputs',
{
'reference_vision_frame' : VisionFrame,
'source_vision_frames' : List[VisionFrame],
'target_vision_frame' : VisionFrame,
'target_vision_frames' : List[VisionFrame],
'temp_vision_frame' : VisionFrame,
'temp_vision_mask' : Mask
})
@@ -1,4 +1,6 @@
from argparse import ArgumentParser
from types import ModuleType
from typing import List
import cv2
import numpy
@@ -6,7 +8,8 @@ import numpy
import facefusion.jobs.job_manager
import facefusion.jobs.job_store
from facefusion import config, content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer, logger, state_manager, translator, video_manager
from facefusion.face_analyser import scale_face
from facefusion.common_helper import get_middle
from facefusion.face_creator import scale_face
from facefusion.face_helper import warp_face_by_face_landmark_5
from facefusion.face_masker import create_area_mask, create_box_mask, create_occlusion_mask, create_region_mask
from facefusion.face_selector import select_faces
@@ -16,7 +19,7 @@ from facefusion.processors.modules.face_debugger.types import FaceDebuggerInputs
from facefusion.processors.types import ProcessorOutputs
from facefusion.program_helper import find_argument_group
from facefusion.types import ApplyStateItem, Args, Face, InferencePool, ProcessMode, VisionFrame
from facefusion.vision import read_static_image, read_static_video_frame
from facefusion.vision import read_static_image, read_static_video_chunk, read_static_video_frame
def get_inference_pool() -> InferencePool:
@@ -38,7 +41,14 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('face_debugger_items', args.get('face_debugger_items'))
def get_common_modules() -> List[ModuleType]:
return [ content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer ]
def pre_check() -> bool:
for common_module in get_common_modules():
if not common_module.pre_check():
return False
return True
@@ -58,14 +68,12 @@ def pre_process(mode : ProcessMode) -> bool:
def post_process() -> None:
read_static_image.cache_clear()
read_static_video_frame.cache_clear()
read_static_video_chunk.cache_clear()
video_manager.clear_video_pool()
if state_manager.get_item('video_memory_strategy') == 'strict':
content_analyser.clear_inference_pool()
face_classifier.clear_inference_pool()
face_detector.clear_inference_pool()
face_landmarker.clear_inference_pool()
face_masker.clear_inference_pool()
face_recognizer.clear_inference_pool()
for common_module in get_common_modules():
common_module.clear_inference_pool()
def debug_face(target_face : Face, temp_vision_frame : VisionFrame) -> VisionFrame:
@@ -94,21 +102,22 @@ def debug_face(target_face : Face, temp_vision_frame : VisionFrame) -> VisionFra
def draw_bounding_box(target_face : Face, temp_vision_frame : VisionFrame) -> VisionFrame:
temp_vision_frame = numpy.ascontiguousarray(temp_vision_frame)
box_color = 0, 0, 255
border_color = 100, 100, 255
bounding_box = target_face.bounding_box.astype(numpy.int32)
x1, y1, x2, y2 = bounding_box
box_color = 0, 0, 255
border_scale = calculate_scale(temp_vision_frame)
border_color = 100, 100, 255
cv2.rectangle(temp_vision_frame, (x1, y1), (x2, y2), box_color, 2)
cv2.rectangle(temp_vision_frame, (x1, y1), (x2, y2), box_color, border_scale)
if target_face.angle == 0:
cv2.line(temp_vision_frame, (x1, y1), (x2, y1), border_color, 3)
cv2.line(temp_vision_frame, (x1, y1), (x2, y1), border_color, border_scale + 1)
if target_face.angle == 180:
cv2.line(temp_vision_frame, (x1, y2), (x2, y2), border_color, 3)
cv2.line(temp_vision_frame, (x1, y2), (x2, y2), border_color, border_scale + 1)
if target_face.angle == 90:
cv2.line(temp_vision_frame, (x2, y1), (x2, y2), border_color, 3)
cv2.line(temp_vision_frame, (x2, y1), (x2, y2), border_color, border_scale + 1)
if target_face.angle == 270:
cv2.line(temp_vision_frame, (x1, y1), (x1, y2), border_color, 3)
cv2.line(temp_vision_frame, (x1, y1), (x1, y2), border_color, border_scale + 1)
return temp_vision_frame
@@ -122,11 +131,15 @@ def draw_face_mask(target_face : Face, temp_vision_frame : VisionFrame) -> Visio
crop_vision_frame, affine_matrix = warp_face_by_face_landmark_5(temp_vision_frame, face_landmark_5_68, 'arcface_128', (512, 512))
inverse_matrix = cv2.invertAffineTransform(affine_matrix)
temp_size = temp_vision_frame.shape[:2][::-1]
mask_scale = calculate_scale(temp_vision_frame)
mask_color = 0, 255, 0
if numpy.array_equal(face_landmark_5, face_landmark_5_68):
mask_color = 255, 255, 0
if target_face.origin == 'refill':
mask_color = 0, 165, 255
if 'box' in state_manager.get_item('face_mask_types'):
box_mask = create_box_mask(crop_vision_frame, 0, state_manager.get_item('face_mask_padding'))
crop_masks.append(box_mask)
@@ -149,7 +162,7 @@ def draw_face_mask(target_face : Face, temp_vision_frame : VisionFrame) -> Visio
inverse_vision_frame = cv2.warpAffine(crop_mask, inverse_matrix, temp_size)
inverse_vision_frame = cv2.threshold(inverse_vision_frame, 100, 255, cv2.THRESH_BINARY)[1]
inverse_contours, _ = cv2.findContours(inverse_vision_frame, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
cv2.drawContours(temp_vision_frame, inverse_contours, -1, mask_color, 2)
cv2.drawContours(temp_vision_frame, inverse_contours, -1, mask_color, mask_scale)
return temp_vision_frame
@@ -157,13 +170,17 @@ def draw_face_mask(target_face : Face, temp_vision_frame : VisionFrame) -> Visio
def draw_face_landmark_5(target_face : Face, temp_vision_frame : VisionFrame) -> VisionFrame:
temp_vision_frame = numpy.ascontiguousarray(temp_vision_frame)
face_landmark_5 = target_face.landmark_set.get('5')
point_scale = calculate_scale(temp_vision_frame)
point_color = 0, 0, 255
if target_face.origin == 'refill':
point_color = 0, 165, 255
if numpy.any(face_landmark_5):
face_landmark_5 = face_landmark_5.astype(numpy.int32)
for point in face_landmark_5:
cv2.circle(temp_vision_frame, tuple(point), 3, point_color, -1)
cv2.circle(temp_vision_frame, tuple(point), point_scale, point_color, -1)
return temp_vision_frame
@@ -172,16 +189,20 @@ def draw_face_landmark_5_68(target_face : Face, temp_vision_frame : VisionFrame)
temp_vision_frame = numpy.ascontiguousarray(temp_vision_frame)
face_landmark_5 = target_face.landmark_set.get('5')
face_landmark_5_68 = target_face.landmark_set.get('5/68')
point_scale = calculate_scale(temp_vision_frame)
point_color = 0, 255, 0
if numpy.array_equal(face_landmark_5, face_landmark_5_68):
point_color = 255, 255, 0
if target_face.origin == 'refill':
point_color = 0, 165, 255
if numpy.any(face_landmark_5_68):
face_landmark_5_68 = face_landmark_5_68.astype(numpy.int32)
for point in face_landmark_5_68:
cv2.circle(temp_vision_frame, tuple(point), 3, point_color, -1)
cv2.circle(temp_vision_frame, tuple(point), point_scale, point_color, -1)
return temp_vision_frame
@@ -190,16 +211,20 @@ def draw_face_landmark_68(target_face : Face, temp_vision_frame : VisionFrame) -
temp_vision_frame = numpy.ascontiguousarray(temp_vision_frame)
face_landmark_68 = target_face.landmark_set.get('68')
face_landmark_68_5 = target_face.landmark_set.get('68/5')
point_scale = calculate_scale(temp_vision_frame)
point_color = 0, 255, 0
if numpy.array_equal(face_landmark_68, face_landmark_68_5):
point_color = 255, 255, 0
if target_face.origin == 'refill':
point_color = 0, 165, 255
if numpy.any(face_landmark_68):
face_landmark_68 = face_landmark_68.astype(numpy.int32)
for point in face_landmark_68:
cv2.circle(temp_vision_frame, tuple(point), 3, point_color, -1)
cv2.circle(temp_vision_frame, tuple(point), point_scale, point_color, -1)
return temp_vision_frame
@@ -207,23 +232,36 @@ def draw_face_landmark_68(target_face : Face, temp_vision_frame : VisionFrame) -
def draw_face_landmark_68_5(target_face : Face, temp_vision_frame : VisionFrame) -> VisionFrame:
temp_vision_frame = numpy.ascontiguousarray(temp_vision_frame)
face_landmark_68_5 = target_face.landmark_set.get('68/5')
point_scale = calculate_scale(temp_vision_frame)
point_color = 255, 255, 0
if target_face.origin == 'refill':
point_color = 0, 165, 255
if numpy.any(face_landmark_68_5):
face_landmark_68_5 = face_landmark_68_5.astype(numpy.int32)
for point in face_landmark_68_5:
cv2.circle(temp_vision_frame, tuple(point), 3, point_color, -1)
cv2.circle(temp_vision_frame, tuple(point), point_scale, point_color, -1)
return temp_vision_frame
def calculate_scale(temp_vision_frame : VisionFrame) -> int:
frame_height, _ = temp_vision_frame.shape[:2]
frame_scale = round(frame_height / 270)
return max(1, min(10, frame_scale))
def process_frame(inputs : FaceDebuggerInputs) -> ProcessorOutputs:
reference_vision_frame = inputs.get('reference_vision_frame')
target_vision_frame = inputs.get('target_vision_frame')
source_vision_frames = inputs.get('source_vision_frames')
target_vision_frames = inputs.get('target_vision_frames')
temp_vision_frame = inputs.get('temp_vision_frame')
temp_vision_mask = inputs.get('temp_vision_mask')
target_faces = select_faces(reference_vision_frame, target_vision_frame)
target_vision_frame = get_middle(target_vision_frames)
target_faces = select_faces(reference_vision_frame, source_vision_frames, target_vision_frames)
if target_faces:
for target_face in target_faces:
@@ -231,5 +269,3 @@ def process_frame(inputs : FaceDebuggerInputs) -> ProcessorOutputs:
temp_vision_frame = debug_face(target_face, temp_vision_frame)
return temp_vision_frame, temp_vision_mask
@@ -1,11 +1,12 @@
from typing import Literal, TypedDict
from typing import List, Literal, TypedDict
from facefusion.types import Mask, VisionFrame
FaceDebuggerInputs = TypedDict('FaceDebuggerInputs',
{
'reference_vision_frame' : VisionFrame,
'target_vision_frame' : VisionFrame,
'source_vision_frames' : List[VisionFrame],
'target_vision_frames' : List[VisionFrame],
'temp_vision_frame' : VisionFrame,
'temp_vision_mask' : Mask
})
@@ -1,6 +1,7 @@
from argparse import ArgumentParser
from functools import lru_cache
from typing import Tuple
from types import ModuleType
from typing import List, Tuple
import cv2
import numpy
@@ -8,9 +9,9 @@ import numpy
import facefusion.jobs.job_manager
import facefusion.jobs.job_store
from facefusion import config, content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer, inference_manager, logger, state_manager, translator, video_manager
from facefusion.common_helper import create_float_metavar
from facefusion.common_helper import create_float_metavar, get_middle
from facefusion.download import conditional_download_hashes, conditional_download_sources, resolve_download_url
from facefusion.face_analyser import scale_face
from facefusion.face_creator import scale_face
from facefusion.face_helper import paste_back, scale_face_landmark_5, warp_face_by_face_landmark_5
from facefusion.face_masker import create_box_mask
from facefusion.face_selector import select_faces
@@ -22,7 +23,7 @@ from facefusion.processors.types import LivePortraitExpression, LivePortraitFeat
from facefusion.program_helper import find_argument_group
from facefusion.thread_helper import conditional_thread_semaphore, thread_semaphore
from facefusion.types import ApplyStateItem, Args, DownloadScope, Face, FaceLandmark68, InferencePool, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import read_static_image, read_static_video_frame
from facefusion.vision import read_static_image, read_static_video_chunk, read_static_video_frame
@lru_cache()
@@ -165,10 +166,18 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('face_editor_head_roll', args.get('face_editor_head_roll'))
def get_common_modules() -> List[ModuleType]:
return [ content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer ]
def pre_check() -> bool:
model_hash_set = get_model_options().get('hashes')
model_source_set = get_model_options().get('sources')
for common_module in get_common_modules():
if not common_module.pre_check():
return False
return conditional_download_hashes(model_hash_set) and conditional_download_sources(model_source_set)
@@ -188,16 +197,15 @@ def pre_process(mode : ProcessMode) -> bool:
def post_process() -> None:
read_static_image.cache_clear()
read_static_video_frame.cache_clear()
read_static_video_chunk.cache_clear()
video_manager.clear_video_pool()
if state_manager.get_item('video_memory_strategy') in [ 'strict', 'moderate' ]:
clear_inference_pool()
if state_manager.get_item('video_memory_strategy') == 'strict':
content_analyser.clear_inference_pool()
face_classifier.clear_inference_pool()
face_detector.clear_inference_pool()
face_landmarker.clear_inference_pool()
face_masker.clear_inference_pool()
face_recognizer.clear_inference_pool()
for common_module in get_common_modules():
common_module.clear_inference_pool()
def edit_face(target_face : Face, temp_vision_frame : VisionFrame) -> VisionFrame:
@@ -486,10 +494,13 @@ def normalize_crop_frame(crop_vision_frame : VisionFrame) -> VisionFrame:
def process_frame(inputs : FaceEditorInputs) -> ProcessorOutputs:
reference_vision_frame = inputs.get('reference_vision_frame')
target_vision_frame = inputs.get('target_vision_frame')
source_vision_frames = inputs.get('source_vision_frames')
target_vision_frames = inputs.get('target_vision_frames')
temp_vision_frame = inputs.get('temp_vision_frame')
temp_vision_mask = inputs.get('temp_vision_mask')
target_faces = select_faces(reference_vision_frame, target_vision_frame)
target_vision_frame = get_middle(target_vision_frames)
target_faces = select_faces(reference_vision_frame, source_vision_frames, target_vision_frames)
if target_faces:
for target_face in target_faces:
@@ -1,11 +1,12 @@
from typing import Literal, TypedDict
from typing import List, Literal, TypedDict
from facefusion.types import Mask, VisionFrame
FaceEditorInputs = TypedDict('FaceEditorInputs',
{
'reference_vision_frame' : VisionFrame,
'target_vision_frame' : VisionFrame,
'source_vision_frames' : List[VisionFrame],
'target_vision_frames' : List[VisionFrame],
'temp_vision_frame' : VisionFrame,
'temp_vision_mask' : Mask
})
@@ -1,14 +1,16 @@
from argparse import ArgumentParser
from functools import lru_cache
from types import ModuleType
from typing import List
import numpy
import facefusion.jobs.job_manager
import facefusion.jobs.job_store
from facefusion import config, content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer, inference_manager, logger, state_manager, translator, video_manager
from facefusion.common_helper import create_float_metavar, create_int_metavar
from facefusion.common_helper import create_float_metavar, create_int_metavar, get_middle
from facefusion.download import conditional_download_hashes, conditional_download_sources, resolve_download_url
from facefusion.face_analyser import scale_face
from facefusion.face_creator import scale_face
from facefusion.face_helper import paste_back, warp_face_by_face_landmark_5
from facefusion.face_masker import create_box_mask, create_occlusion_mask
from facefusion.face_selector import select_faces
@@ -19,7 +21,7 @@ from facefusion.processors.types import ProcessorOutputs
from facefusion.program_helper import find_argument_group
from facefusion.thread_helper import thread_semaphore
from facefusion.types import ApplyStateItem, Args, DownloadScope, Face, InferencePool, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import blend_frame, read_static_image, read_static_video_frame
from facefusion.vision import blend_frame, read_static_image, read_static_video_chunk, read_static_video_frame
@lru_cache()
@@ -304,10 +306,18 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('face_enhancer_weight', args.get('face_enhancer_weight'))
def get_common_modules() -> List[ModuleType]:
return [ content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer ]
def pre_check() -> bool:
model_hash_set = get_model_options().get('hashes')
model_source_set = get_model_options().get('sources')
for common_module in get_common_modules():
if not common_module.pre_check():
return False
return conditional_download_hashes(model_hash_set) and conditional_download_sources(model_source_set)
@@ -327,16 +337,15 @@ def pre_process(mode : ProcessMode) -> bool:
def post_process() -> None:
read_static_image.cache_clear()
read_static_video_frame.cache_clear()
read_static_video_chunk.cache_clear()
video_manager.clear_video_pool()
if state_manager.get_item('video_memory_strategy') in [ 'strict', 'moderate' ]:
clear_inference_pool()
if state_manager.get_item('video_memory_strategy') == 'strict':
content_analyser.clear_inference_pool()
face_classifier.clear_inference_pool()
face_detector.clear_inference_pool()
face_landmarker.clear_inference_pool()
face_masker.clear_inference_pool()
face_recognizer.clear_inference_pool()
for common_module in get_common_modules():
common_module.clear_inference_pool()
def enhance_face(target_face : Face, temp_vision_frame : VisionFrame) -> VisionFrame:
@@ -413,10 +422,13 @@ def blend_paste_frame(temp_vision_frame : VisionFrame, paste_vision_frame : Visi
def process_frame(inputs : FaceEnhancerInputs) -> ProcessorOutputs:
reference_vision_frame = inputs.get('reference_vision_frame')
target_vision_frame = inputs.get('target_vision_frame')
source_vision_frames = inputs.get('source_vision_frames')
target_vision_frames = inputs.get('target_vision_frames')
temp_vision_frame = inputs.get('temp_vision_frame')
temp_vision_mask = inputs.get('temp_vision_mask')
target_faces = select_faces(reference_vision_frame, target_vision_frame)
target_vision_frame = get_middle(target_vision_frames)
target_faces = select_faces(reference_vision_frame, source_vision_frames, target_vision_frames)
if target_faces:
for target_face in target_faces:
@@ -1,4 +1,4 @@
from typing import Any, Literal, TypeAlias, TypedDict
from typing import Any, List, Literal, TypeAlias, TypedDict
from numpy.typing import NDArray
@@ -7,7 +7,8 @@ from facefusion.types import Mask, VisionFrame
FaceEnhancerInputs = TypedDict('FaceEnhancerInputs',
{
'reference_vision_frame' : VisionFrame,
'target_vision_frame' : VisionFrame,
'source_vision_frames' : List[VisionFrame],
'target_vision_frames' : List[VisionFrame],
'temp_vision_frame' : VisionFrame,
'temp_vision_mask' : Mask
})
@@ -1,5 +1,6 @@
from argparse import ArgumentParser
from functools import lru_cache
from types import ModuleType
from typing import List, Optional, Tuple
import cv2
@@ -9,10 +10,10 @@ import facefusion.choices
import facefusion.jobs.job_manager
import facefusion.jobs.job_store
from facefusion import config, content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer, inference_manager, logger, state_manager, translator, video_manager
from facefusion.common_helper import get_first, is_macos
from facefusion.common_helper import get_first, get_middle, is_macos
from facefusion.download import conditional_download_hashes, conditional_download_sources, resolve_download_url
from facefusion.execution import has_execution_provider
from facefusion.face_analyser import get_average_face, get_many_faces, get_one_face, scale_face
from facefusion.face_creator import average_face_identity, get_one_face, get_static_faces, scale_face
from facefusion.face_helper import paste_back, warp_face_by_face_landmark_5
from facefusion.face_masker import create_area_mask, create_box_mask, create_occlusion_mask, create_region_mask
from facefusion.face_selector import select_faces, sort_faces_by_order
@@ -24,8 +25,8 @@ from facefusion.processors.pixel_boost import explode_pixel_boost, implode_pixel
from facefusion.processors.types import ProcessorOutputs
from facefusion.program_helper import find_argument_group
from facefusion.thread_helper import conditional_thread_semaphore
from facefusion.types import ApplyStateItem, Args, DownloadScope, Embedding, Face, InferencePool, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import read_static_image, read_static_images, read_static_video_frame, unpack_resolution
from facefusion.types import ApplyStateItem, Args, DownloadScope, Embedding, Face, InferencePool, InferenceProvider, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import read_static_image, read_static_images, read_static_video_chunk, read_static_video_frame, unpack_resolution
@lru_cache()
@@ -246,6 +247,7 @@ def create_static_model_set(download_scope : DownloadScope) -> ModelSet:
'path': resolve_relative_path('../.assets/models/hyperswap_1a_256.onnx')
}
},
'precision': 'fp16',
'type': 'hyperswap',
'template': 'arcface_128',
'size': (256, 256),
@@ -276,6 +278,7 @@ def create_static_model_set(download_scope : DownloadScope) -> ModelSet:
'path': resolve_relative_path('../.assets/models/hyperswap_1b_256.onnx')
}
},
'precision': 'fp16',
'type': 'hyperswap',
'template': 'arcface_128',
'size': (256, 256),
@@ -306,6 +309,7 @@ def create_static_model_set(download_scope : DownloadScope) -> ModelSet:
'path': resolve_relative_path('../.assets/models/hyperswap_1c_256.onnx')
}
},
'precision': 'fp16',
'type': 'hyperswap',
'template': 'arcface_128',
'size': (256, 256),
@@ -366,6 +370,7 @@ def create_static_model_set(download_scope : DownloadScope) -> ModelSet:
'path': resolve_relative_path('../.assets/models/inswapper_128_fp16.onnx')
}
},
'precision': 'fp16',
'type': 'inswapper',
'template': 'arcface_128',
'size': (128, 128),
@@ -486,28 +491,38 @@ def create_static_model_set(download_scope : DownloadScope) -> ModelSet:
def get_inference_pool() -> InferencePool:
model_names = [ get_model_name() ]
model_names = [ state_manager.get_item('face_swapper_model') ]
model_source_set = get_model_options().get('sources')
return inference_manager.get_inference_pool(__name__, model_names, model_source_set)
def clear_inference_pool() -> None:
model_names = [ get_model_name() ]
model_names = [ state_manager.get_item('face_swapper_model') ]
inference_manager.clear_inference_pool(__name__, model_names)
def resolve_inference_providers() -> List[InferenceProvider]:
model_precision = get_model_options().get('precision')
model_type = get_model_options().get('type')
if is_macos() and has_execution_provider('coreml'):
if model_type in [ 'ghost', 'uniface' ] or model_precision == 'fp16':
return\
[
(facefusion.choices.execution_provider_set.get('coreml'),
{
'ModelFormat': 'MLProgram',
'SpecializationStrategy': 'FastPrediction'
})
]
return []
def get_model_options() -> ModelOptions:
model_name = get_model_name()
return create_static_model_set('full').get(model_name)
def get_model_name() -> str:
model_name = state_manager.get_item('face_swapper_model')
if is_macos() and has_execution_provider('coreml') and model_name == 'inswapper_128_fp16':
return 'inswapper_128'
return model_name
return create_static_model_set('full').get(model_name)
def register_args(program : ArgumentParser) -> None:
@@ -527,10 +542,18 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('face_swapper_weight', args.get('face_swapper_weight'))
def get_common_modules() -> List[ModuleType]:
return [ content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer ]
def pre_check() -> bool:
model_hash_set = get_model_options().get('hashes')
model_source_set = get_model_options().get('sources')
for common_module in get_common_modules():
if not common_module.pre_check():
return False
return conditional_download_hashes(model_hash_set) and conditional_download_sources(model_source_set)
@@ -541,7 +564,7 @@ def pre_process(mode : ProcessMode) -> bool:
source_image_paths = filter_image_paths(state_manager.get_item('source_paths'))
source_vision_frames = read_static_images(source_image_paths)
source_faces = get_many_faces(source_vision_frames)
source_faces = get_static_faces(source_vision_frames)
if not get_one_face(source_faces):
logger.error(translator.get('no_source_face_detected') + translator.get('exclamation_mark'), __name__)
@@ -565,20 +588,19 @@ def pre_process(mode : ProcessMode) -> bool:
def post_process() -> None:
read_static_image.cache_clear()
read_static_video_frame.cache_clear()
read_static_video_chunk.cache_clear()
video_manager.clear_video_pool()
if state_manager.get_item('video_memory_strategy') in [ 'strict', 'moderate' ]:
get_static_model_initializer.cache_clear()
clear_inference_pool()
if state_manager.get_item('video_memory_strategy') == 'strict':
content_analyser.clear_inference_pool()
face_classifier.clear_inference_pool()
face_detector.clear_inference_pool()
face_landmarker.clear_inference_pool()
face_masker.clear_inference_pool()
face_recognizer.clear_inference_pool()
for common_module in get_common_modules():
common_module.clear_inference_pool()
def swap_face(source_face : Face, target_face : Face, temp_vision_frame : VisionFrame) -> VisionFrame:
def swap_face(source_face : Face, target_face : Face, source_vision_frame : VisionFrame, temp_vision_frame : VisionFrame) -> VisionFrame:
model_template = get_model_options().get('template')
model_size = get_model_options().get('size')
pixel_boost_size = unpack_resolution(state_manager.get_item('face_swapper_pixel_boost'))
@@ -598,7 +620,7 @@ def swap_face(source_face : Face, target_face : Face, temp_vision_frame : Vision
pixel_boost_vision_frames = implode_pixel_boost(crop_vision_frame, pixel_boost_total, model_size)
for pixel_boost_vision_frame in pixel_boost_vision_frames:
pixel_boost_vision_frame = prepare_crop_frame(pixel_boost_vision_frame)
pixel_boost_vision_frame = forward_swap_face(source_face, target_face, pixel_boost_vision_frame)
pixel_boost_vision_frame = forward_swap_face(source_face, target_face, source_vision_frame, pixel_boost_vision_frame)
pixel_boost_vision_frame = normalize_crop_frame(pixel_boost_vision_frame)
temp_vision_frames.append(pixel_boost_vision_frame)
crop_vision_frame = explode_pixel_boost(temp_vision_frames, pixel_boost_total, model_size, pixel_boost_size)
@@ -617,18 +639,15 @@ def swap_face(source_face : Face, target_face : Face, temp_vision_frame : Vision
return paste_vision_frame
def forward_swap_face(source_face : Face, target_face : Face, crop_vision_frame : VisionFrame) -> VisionFrame:
def forward_swap_face(source_face : Face, target_face : Face, source_vision_frame : VisionFrame, crop_vision_frame : VisionFrame) -> VisionFrame:
face_swapper = get_inference_pool().get('face_swapper')
model_type = get_model_options().get('type')
face_swapper_inputs = {}
if is_macos() and has_execution_provider('coreml') and model_type in [ 'ghost', 'uniface' ]:
face_swapper.set_providers([ facefusion.choices.execution_provider_set.get('cpu') ])
for face_swapper_input in face_swapper.get_inputs():
if face_swapper_input.name == 'source':
if model_type in [ 'blendswap', 'uniface' ]:
face_swapper_inputs[face_swapper_input.name] = prepare_source_frame(source_face)
face_swapper_inputs[face_swapper_input.name] = prepare_source_frame(source_face, source_vision_frame)
else:
source_embedding = prepare_source_embedding(source_face)
source_embedding = balance_source_embedding(source_embedding, target_face.embedding)
@@ -654,9 +673,8 @@ def forward_convert_embedding(face_embedding : Embedding) -> Embedding:
return face_embedding
def prepare_source_frame(source_face : Face) -> VisionFrame:
def prepare_source_frame(source_face : Face, source_vision_frame : VisionFrame) -> VisionFrame:
model_type = get_model_options().get('type')
source_vision_frame = read_static_image(get_first(state_manager.get_item('source_paths')))
if model_type == 'blendswap':
source_vision_frame, _ = warp_face_by_face_landmark_5(source_vision_frame, source_face.landmark_set.get('5/68'), 'arcface_112_v2', (112, 112))
@@ -748,27 +766,31 @@ def extract_source_face(source_vision_frames : List[VisionFrame]) -> Optional[Fa
if source_vision_frames:
for source_vision_frame in source_vision_frames:
temp_faces = get_many_faces([source_vision_frame])
temp_faces = get_static_faces([ source_vision_frame ])
temp_faces = sort_faces_by_order(temp_faces, 'large-small')
if temp_faces:
source_faces.append(get_first(temp_faces))
return get_average_face(source_faces)
return average_face_identity(source_faces)
def process_frame(inputs : FaceSwapperInputs) -> ProcessorOutputs:
reference_vision_frame = inputs.get('reference_vision_frame')
source_vision_frames = inputs.get('source_vision_frames')
target_vision_frame = inputs.get('target_vision_frame')
target_vision_frames = inputs.get('target_vision_frames')
temp_vision_frame = inputs.get('temp_vision_frame')
temp_vision_mask = inputs.get('temp_vision_mask')
target_vision_frame = get_middle(target_vision_frames)
source_face = extract_source_face(source_vision_frames)
target_faces = select_faces(reference_vision_frame, target_vision_frame)
target_faces = select_faces(reference_vision_frame, source_vision_frames, target_vision_frames)
if source_face and target_faces:
source_vision_frame = get_first(source_vision_frames)
for target_face in target_faces:
target_face = scale_face(target_face, target_vision_frame, temp_vision_frame)
temp_vision_frame = swap_face(source_face, target_face, temp_vision_frame)
temp_vision_frame = swap_face(source_face, target_face, source_vision_frame, temp_vision_frame)
return temp_vision_frame, temp_vision_mask
@@ -6,7 +6,7 @@ FaceSwapperInputs = TypedDict('FaceSwapperInputs',
{
'reference_vision_frame' : VisionFrame,
'source_vision_frames' : List[VisionFrame],
'target_vision_frame' : VisionFrame,
'target_vision_frames' : List[VisionFrame],
'temp_vision_frame' : VisionFrame,
'temp_vision_mask' : Mask
})
@@ -1,10 +1,12 @@
from argparse import ArgumentParser
from functools import lru_cache
from types import ModuleType
from typing import List
import cv2
import numpy
import facefusion.choices
import facefusion.jobs.job_manager
import facefusion.jobs.job_store
from facefusion import config, content_analyser, inference_manager, logger, state_manager, translator, video_manager
@@ -17,8 +19,8 @@ from facefusion.processors.modules.frame_colorizer.types import FrameColorizerIn
from facefusion.processors.types import ProcessorOutputs
from facefusion.program_helper import find_argument_group
from facefusion.thread_helper import thread_semaphore
from facefusion.types import ApplyStateItem, Args, DownloadScope, ExecutionProvider, InferencePool, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import blend_frame, read_static_image, read_static_video_frame, unpack_resolution
from facefusion.types import ApplyStateItem, Args, DownloadScope, InferencePool, InferenceProvider, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import blend_frame, read_static_image, read_static_video_chunk, read_static_video_frame, unpack_resolution
@lru_cache()
@@ -170,10 +172,11 @@ def clear_inference_pool() -> None:
inference_manager.clear_inference_pool(__name__, model_names)
def resolve_execution_providers() -> List[ExecutionProvider]:
def resolve_inference_providers() -> List[InferenceProvider]:
if is_macos() and has_execution_provider('coreml'):
return [ 'cpu' ]
return state_manager.get_item('execution_providers')
return [ facefusion.choices.execution_provider_set.get('cpu') ]
return []
def get_model_options() -> ModelOptions:
@@ -196,10 +199,18 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('frame_colorizer_size', args.get('frame_colorizer_size'))
def get_common_modules() -> List[ModuleType]:
return [ content_analyser ]
def pre_check() -> bool:
model_hash_set = get_model_options().get('hashes')
model_source_set = get_model_options().get('sources')
for common_module in get_common_modules():
if not common_module.pre_check():
return False
return conditional_download_hashes(model_hash_set) and conditional_download_sources(model_source_set)
@@ -219,11 +230,15 @@ def pre_process(mode : ProcessMode) -> bool:
def post_process() -> None:
read_static_image.cache_clear()
read_static_video_frame.cache_clear()
read_static_video_chunk.cache_clear()
video_manager.clear_video_pool()
if state_manager.get_item('video_memory_strategy') in [ 'strict', 'moderate' ]:
clear_inference_pool()
if state_manager.get_item('video_memory_strategy') == 'strict':
content_analyser.clear_inference_pool()
for common_module in get_common_modules():
common_module.clear_inference_pool()
def colorize_frame(temp_vision_frame : VisionFrame) -> VisionFrame:
@@ -1,10 +1,10 @@
from typing import Literal, TypedDict
from typing import List, Literal, TypedDict
from facefusion.types import Mask, VisionFrame
FrameColorizerInputs = TypedDict('FrameColorizerInputs',
{
'target_vision_frame' : VisionFrame,
'target_vision_frames' : List[VisionFrame],
'temp_vision_frame' : VisionFrame,
'temp_vision_mask' : Mask
})
@@ -1,9 +1,12 @@
from argparse import ArgumentParser
from functools import lru_cache
from types import ModuleType
from typing import List
import cv2
import numpy
import facefusion.choices
import facefusion.jobs.job_manager
import facefusion.jobs.job_store
from facefusion import config, content_analyser, inference_manager, logger, state_manager, translator, video_manager
@@ -16,8 +19,8 @@ from facefusion.processors.modules.frame_enhancer.types import FrameEnhancerInpu
from facefusion.processors.types import ProcessorOutputs
from facefusion.program_helper import find_argument_group
from facefusion.thread_helper import conditional_thread_semaphore
from facefusion.types import ApplyStateItem, Args, DownloadScope, InferencePool, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import blend_frame, create_tile_frames, merge_tile_frames, read_static_image, read_static_video_frame
from facefusion.types import ApplyStateItem, Args, DownloadScope, InferencePool, InferenceProvider, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import blend_frame, create_tile_frames, merge_tile_frames, read_static_image, read_static_video_chunk, read_static_video_frame
@lru_cache()
@@ -156,6 +159,7 @@ def create_static_model_set(download_scope : DownloadScope) -> ModelSet:
'path': resolve_relative_path('../.assets/models/real_esrgan_x2_fp16.onnx')
}
},
'precision': 'fp16',
'size': (256, 16, 8),
'scale': 2
},
@@ -210,6 +214,7 @@ def create_static_model_set(download_scope : DownloadScope) -> ModelSet:
'path': resolve_relative_path('../.assets/models/real_esrgan_x4_fp16.onnx')
}
},
'precision': 'fp16',
'size': (256, 16, 8),
'scale': 4
},
@@ -264,6 +269,7 @@ def create_static_model_set(download_scope : DownloadScope) -> ModelSet:
'path': resolve_relative_path('../.assets/models/real_esrgan_x8_fp16.onnx')
}
},
'precision': 'fp16',
'size': (256, 16, 8),
'scale': 8
},
@@ -541,35 +547,38 @@ def create_static_model_set(download_scope : DownloadScope) -> ModelSet:
def get_inference_pool() -> InferencePool:
model_names = [ get_frame_enhancer_model() ]
model_names = [ state_manager.get_item('frame_enhancer_model') ]
model_source_set = get_model_options().get('sources')
return inference_manager.get_inference_pool(__name__, model_names, model_source_set)
def clear_inference_pool() -> None:
model_names = [ get_frame_enhancer_model() ]
model_names = [ state_manager.get_item('frame_enhancer_model') ]
inference_manager.clear_inference_pool(__name__, model_names)
def resolve_inference_providers() -> List[InferenceProvider]:
model_precision = get_model_options().get('precision')
if is_macos() and has_execution_provider('coreml') and model_precision == 'fp16':
return\
[
(facefusion.choices.execution_provider_set.get('coreml'),
{
'ModelFormat': 'MLProgram',
'SpecializationStrategy': 'FastPrediction'
})
]
return []
def get_model_options() -> ModelOptions:
model_name = get_frame_enhancer_model()
model_name = state_manager.get_item('frame_enhancer_model')
return create_static_model_set('full').get(model_name)
def get_frame_enhancer_model() -> str:
frame_enhancer_model = state_manager.get_item('frame_enhancer_model')
if is_macos() and has_execution_provider('coreml'):
if frame_enhancer_model == 'real_esrgan_x2_fp16':
return 'real_esrgan_x2'
if frame_enhancer_model == 'real_esrgan_x4_fp16':
return 'real_esrgan_x4'
if frame_enhancer_model == 'real_esrgan_x8_fp16':
return 'real_esrgan_x8'
return frame_enhancer_model
def register_args(program : ArgumentParser) -> None:
group_processors = find_argument_group(program, 'processors')
if group_processors:
@@ -583,10 +592,18 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('frame_enhancer_blend', args.get('frame_enhancer_blend'))
def get_common_modules() -> List[ModuleType]:
return [ content_analyser ]
def pre_check() -> bool:
model_hash_set = get_model_options().get('hashes')
model_source_set = get_model_options().get('sources')
for common_module in get_common_modules():
if not common_module.pre_check():
return False
return conditional_download_hashes(model_hash_set) and conditional_download_sources(model_source_set)
@@ -606,11 +623,15 @@ def pre_process(mode : ProcessMode) -> bool:
def post_process() -> None:
read_static_image.cache_clear()
read_static_video_frame.cache_clear()
read_static_video_chunk.cache_clear()
video_manager.clear_video_pool()
if state_manager.get_item('video_memory_strategy') in [ 'strict', 'moderate' ]:
clear_inference_pool()
if state_manager.get_item('video_memory_strategy') == 'strict':
content_analyser.clear_inference_pool()
for common_module in get_common_modules():
common_module.clear_inference_pool()
def enhance_frame(temp_vision_frame : VisionFrame) -> VisionFrame:
@@ -1,10 +1,10 @@
from typing import Literal, TypedDict
from typing import List, Literal, TypedDict
from facefusion.types import Mask, VisionFrame
FrameEnhancerInputs = TypedDict('FrameEnhancerInputs',
{
'target_vision_frame' : VisionFrame,
'target_vision_frames' : List[VisionFrame],
'temp_vision_frame' : VisionFrame,
'temp_vision_mask' : Mask
})
@@ -1,5 +1,7 @@
from argparse import ArgumentParser
from functools import lru_cache
from types import ModuleType
from typing import List
import cv2
import numpy
@@ -8,9 +10,9 @@ import facefusion.jobs.job_manager
import facefusion.jobs.job_store
from facefusion import config, content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer, inference_manager, logger, state_manager, translator, video_manager, voice_extractor
from facefusion.audio import read_static_voice
from facefusion.common_helper import create_float_metavar
from facefusion.common_helper import create_float_metavar, get_middle
from facefusion.download import conditional_download_hashes, conditional_download_sources, resolve_download_url
from facefusion.face_analyser import scale_face
from facefusion.face_creator import scale_face
from facefusion.face_helper import create_bounding_box, paste_back, warp_face_by_bounding_box, warp_face_by_face_landmark_5
from facefusion.face_masker import create_area_mask, create_box_mask, create_occlusion_mask
from facefusion.face_selector import select_faces
@@ -21,7 +23,7 @@ from facefusion.processors.types import ProcessorOutputs
from facefusion.program_helper import find_argument_group
from facefusion.thread_helper import conditional_thread_semaphore
from facefusion.types import ApplyStateItem, Args, AudioFrame, DownloadScope, Face, InferencePool, ModelOptions, ModelSet, ProcessMode, VisionFrame
from facefusion.vision import read_static_image, read_static_video_frame
from facefusion.vision import read_static_image, read_static_video_chunk, read_static_video_frame
@lru_cache()
@@ -142,10 +144,18 @@ def apply_args(args : Args, apply_state_item : ApplyStateItem) -> None:
apply_state_item('lip_syncer_weight', args.get('lip_syncer_weight'))
def get_common_modules() -> List[ModuleType]:
return [ content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer, voice_extractor ]
def pre_check() -> bool:
model_hash_set = get_model_options().get('hashes')
model_source_set = get_model_options().get('sources')
for common_module in get_common_modules():
if not common_module.pre_check():
return False
return conditional_download_hashes(model_hash_set) and conditional_download_sources(model_source_set)
@@ -159,18 +169,16 @@ def pre_process(mode : ProcessMode) -> bool:
def post_process() -> None:
read_static_image.cache_clear()
read_static_video_frame.cache_clear()
read_static_video_chunk.cache_clear()
read_static_voice.cache_clear()
video_manager.clear_video_pool()
if state_manager.get_item('video_memory_strategy') in [ 'strict', 'moderate' ]:
clear_inference_pool()
if state_manager.get_item('video_memory_strategy') == 'strict':
content_analyser.clear_inference_pool()
face_classifier.clear_inference_pool()
face_detector.clear_inference_pool()
face_landmarker.clear_inference_pool()
face_masker.clear_inference_pool()
face_recognizer.clear_inference_pool()
voice_extractor.clear_inference_pool()
for common_module in get_common_modules():
common_module.clear_inference_pool()
def sync_lip(target_face : Face, source_voice_frame : AudioFrame, temp_vision_frame : VisionFrame) -> VisionFrame:
@@ -282,11 +290,14 @@ def normalize_crop_frame(crop_vision_frame : VisionFrame) -> VisionFrame:
def process_frame(inputs : LipSyncerInputs) -> ProcessorOutputs:
reference_vision_frame = inputs.get('reference_vision_frame')
source_vision_frames = inputs.get('source_vision_frames')
source_voice_frame = inputs.get('source_voice_frame')
target_vision_frame = inputs.get('target_vision_frame')
target_vision_frames = inputs.get('target_vision_frames')
temp_vision_frame = inputs.get('temp_vision_frame')
temp_vision_mask = inputs.get('temp_vision_mask')
target_faces = select_faces(reference_vision_frame, target_vision_frame)
target_vision_frame = get_middle(target_vision_frames)
target_faces = select_faces(reference_vision_frame, source_vision_frames, target_vision_frames)
if target_faces:
for target_face in target_faces:
@@ -1,4 +1,4 @@
from typing import Any, Literal, TypeAlias, TypedDict
from typing import Any, List, Literal, TypeAlias, TypedDict
from numpy.typing import NDArray
@@ -7,8 +7,9 @@ from facefusion.types import AudioFrame, Mask, VisionFrame
LipSyncerInputs = TypedDict('LipSyncerInputs',
{
'reference_vision_frame' : VisionFrame,
'source_vision_frames' : List[VisionFrame],
'source_voice_frame' : AudioFrame,
'target_vision_frame' : VisionFrame,
'target_vision_frames' : List[VisionFrame],
'temp_vision_frame' : VisionFrame,
'temp_vision_mask' : Mask
})
+18 -3
View File
@@ -133,6 +133,14 @@ def create_face_selector_program() -> ArgumentParser:
return program
def create_face_tracker_program() -> ArgumentParser:
program = ArgumentParser(add_help = False)
group_face_tracker = program.add_argument_group('face tracker')
group_face_tracker.add_argument('--face-tracker-score', help = translator.get('help.face_tracker_score'), type = float, default = config.get_float_value('face_tracker', 'face_tracker_score', '0.0'), choices = facefusion.choices.face_tracker_score_range, metavar = create_float_metavar(facefusion.choices.face_tracker_score_range))
job_store.register_step_keys([ 'face_tracker_score' ])
return program
def create_face_masker_program() -> ArgumentParser:
program = ArgumentParser(add_help = False)
group_face_masker = program.add_argument_group('face masker')
@@ -166,6 +174,14 @@ def create_frame_extraction_program() -> ArgumentParser:
return program
def create_frame_process_program() -> ArgumentParser:
program = ArgumentParser(add_help = False)
group_frame_process = program.add_argument_group('frame process')
group_frame_process.add_argument('--target-frame-amount', help = translator.get('help.target_frame_amount'), type = int, default = config.get_int_value('frame_process', 'target_frame_amount', '5'), choices = facefusion.choices.target_frame_amount_range, metavar = create_int_metavar(facefusion.choices.target_frame_amount_range))
job_store.register_step_keys([ 'target_frame_amount' ])
return program
def create_output_creation_program() -> ArgumentParser:
program = ArgumentParser(add_help = False)
available_encoder_set = get_available_encoder_set()
@@ -245,8 +261,7 @@ def create_memory_program() -> ArgumentParser:
program = ArgumentParser(add_help = False)
group_memory = program.add_argument_group('memory')
group_memory.add_argument('--video-memory-strategy', help = translator.get('help.video_memory_strategy'), default = config.get_str_value('memory', 'video_memory_strategy', 'strict'), choices = facefusion.choices.video_memory_strategies)
group_memory.add_argument('--system-memory-limit', help = translator.get('help.system_memory_limit'), type = int, default = config.get_int_value('memory', 'system_memory_limit', '0'), choices = facefusion.choices.system_memory_limit_range, metavar = create_int_metavar(facefusion.choices.system_memory_limit_range))
job_store.register_job_keys([ 'video_memory_strategy', 'system_memory_limit' ])
job_store.register_job_keys([ 'video_memory_strategy' ])
return program
@@ -285,7 +300,7 @@ def create_step_index_program() -> ArgumentParser:
def collect_step_program() -> ArgumentParser:
return ArgumentParser(parents = [ create_face_detector_program(), create_face_landmarker_program(), create_face_selector_program(), create_face_masker_program(), create_voice_extractor_program(), create_frame_extraction_program(), create_output_creation_program(), create_processors_program() ], add_help = False)
return ArgumentParser(parents = [ create_face_detector_program(), create_face_landmarker_program(), create_face_selector_program(), create_face_tracker_program(), create_face_masker_program(), create_voice_extractor_program(), create_frame_extraction_program(), create_frame_process_program(), create_output_creation_program(), create_processors_program() ], add_help = False)
def collect_job_program() -> ArgumentParser:
+1
View File
@@ -9,6 +9,7 @@ def sanitize_job_id(job_id : str) -> str:
if __job_id__.isalnum():
return job_id
return hashlib.sha1(job_id.encode()).hexdigest()
+5 -5
View File
@@ -2,7 +2,7 @@ import os
import subprocess
from collections import deque
from concurrent.futures import ThreadPoolExecutor
from typing import Deque, Iterator
from typing import Deque, Iterator, List
import cv2
import numpy
@@ -20,6 +20,7 @@ from facefusion.vision import extract_vision_mask, read_static_images
def multi_process_capture(camera_capture : cv2.VideoCapture, camera_fps : Fps) -> Iterator[VisionFrame]:
capture_deque : Deque[VisionFrame] = deque()
source_vision_frames = read_static_images(state_manager.get_item('source_paths'))
with tqdm(desc = translator.get('streaming'), unit = 'frame', disable = state_manager.get_item('log_level') in [ 'warn', 'error' ]) as progress:
with ThreadPoolExecutor(max_workers = state_manager.get_item('execution_thread_count')) as executor:
@@ -31,7 +32,7 @@ def multi_process_capture(camera_capture : cv2.VideoCapture, camera_fps : Fps) -
camera_capture.release()
if numpy.any(capture_vision_frame):
future = executor.submit(process_stream_frame, capture_vision_frame)
future = executor.submit(process_stream_frame, source_vision_frames, capture_vision_frame)
futures.append(future)
for future_done in [ future for future in futures if future.done() ]:
@@ -44,8 +45,7 @@ def multi_process_capture(camera_capture : cv2.VideoCapture, camera_fps : Fps) -
yield capture_deque.popleft()
def process_stream_frame(target_vision_frame : VisionFrame) -> VisionFrame:
source_vision_frames = read_static_images(state_manager.get_item('source_paths'))
def process_stream_frame(source_vision_frames : List[VisionFrame], target_vision_frame : VisionFrame) -> VisionFrame:
source_audio_frame = create_empty_audio_frame()
source_voice_frame = create_empty_audio_frame()
temp_vision_frame = target_vision_frame.copy()
@@ -60,7 +60,7 @@ def process_stream_frame(target_vision_frame : VisionFrame) -> VisionFrame:
'source_vision_frames': source_vision_frames,
'source_audio_frame': source_audio_frame,
'source_voice_frame': source_voice_frame,
'target_vision_frame': target_vision_frame,
'target_vision_frames': [ target_vision_frame ],
'temp_vision_frame': temp_vision_frame,
'temp_vision_mask': temp_vision_mask
})
+11 -5
View File
@@ -1,8 +1,8 @@
import os
from typing import List
from facefusion import state_manager
from facefusion.filesystem import create_directory, get_file_extension, get_file_name, move_file, remove_directory, resolve_file_pattern
from facefusion.types import FrameSet
def get_temp_file_path(file_path : str) -> str:
@@ -16,12 +16,18 @@ def move_temp_file(file_path : str, move_path : str) -> bool:
return move_file(temp_file_path, move_path)
def resolve_temp_frame_paths(target_path : str) -> List[str]:
temp_frames_pattern = get_temp_frames_pattern(target_path, '*')
return resolve_file_pattern(temp_frames_pattern)
def resolve_temp_frame_set(target_path : str) -> FrameSet:
temp_frame_pattern = get_temp_frame_pattern(target_path, '*')
temp_frame_set = {}
for temp_frame_path in resolve_file_pattern(temp_frame_pattern):
frame_number = int(get_file_name(temp_frame_path))
temp_frame_set[frame_number] = temp_frame_path
return temp_frame_set
def get_temp_frames_pattern(target_path : str, temp_frame_prefix : str) -> str:
def get_temp_frame_pattern(target_path : str, temp_frame_prefix : str) -> str:
temp_directory_path = get_temp_directory_path(target_path)
return os.path.join(temp_directory_path, temp_frame_prefix + '.' + state_manager.get_item('temp_frame_format'))
+28 -15
View File
@@ -1,5 +1,6 @@
from collections import namedtuple
from typing import Any, Callable, Dict, List, Literal, Optional, Tuple, TypeAlias, TypedDict
from threading import Lock
from typing import Any, Callable, Dict, List, Literal, NotRequired, Optional, Tuple, TypeAlias, TypedDict
import cv2
import numpy
@@ -29,26 +30,34 @@ FaceScoreSet = TypedDict('FaceScoreSet',
'landmarker' : Score
})
Embedding : TypeAlias = NDArray[numpy.float64]
Gender = Literal['female', 'male']
Age : TypeAlias = range
Gender = Literal['female', 'male']
Race = Literal['white', 'black', 'latino', 'asian', 'indian', 'arabic']
FaceSelectorGender = Literal['auto', 'female', 'male']
FaceSelectorRace = Literal['auto', 'white', 'black', 'latino', 'asian', 'indian', 'arabic']
Face = namedtuple('Face',
[
'origin',
'bounding_box',
'score_set',
'landmark_set',
'angle',
'embedding',
'embedding_norm',
'gender',
'age',
'gender',
'race'
])
FaceSet : TypeAlias = Dict[str, List[Face]]
FaceStore = TypedDict('FaceStore',
FaceSet = TypedDict('FaceSet',
{
'static_faces' : FaceSet
'lock': Lock,
'faces': NotRequired[List[Face]]
})
FaceStore : TypeAlias = Dict[str, FaceSet]
FaceTrack : TypeAlias = Dict[int, Face]
Language = Literal['en']
Locales : TypeAlias = Dict[Language, Dict[str, Any]]
@@ -59,12 +68,12 @@ VideoWriterSet : TypeAlias = Dict[str, cv2.VideoWriter]
CameraCaptureSet : TypeAlias = Dict[str, cv2.VideoCapture]
VideoPoolSet = TypedDict('VideoPoolSet',
{
'capture': VideoCaptureSet,
'writer': VideoWriterSet
'capture' : VideoCaptureSet,
'writer' : VideoWriterSet
})
CameraPoolSet = TypedDict('CameraPoolSet',
{
'capture': CameraCaptureSet
'capture' : CameraCaptureSet
})
ColorMode = Literal['rgb', 'rgba']
@@ -138,6 +147,8 @@ AudioTypeSet : TypeAlias = Dict[AudioFormat, str]
ImageTypeSet : TypeAlias = Dict[ImageFormat, str]
VideoTypeSet : TypeAlias = Dict[VideoFormat, str]
FrameSet : TypeAlias = Dict[int, str]
AudioEncoder = Literal['flac', 'aac', 'libmp3lame', 'libopus', 'libvorbis', 'pcm_s16le', 'pcm_s32le']
VideoEncoder = Literal['libx264', 'libx264rgb', 'libx265', 'libvpx-vp9', 'h264_nvenc', 'hevc_nvenc', 'h264_amf', 'hevc_amf', 'h264_qsv', 'hevc_qsv', 'h264_videotoolbox', 'hevc_videotoolbox', 'rawvideo']
EncoderSet = TypedDict('EncoderSet',
@@ -290,6 +301,7 @@ StateKey = Literal\
'reference_face_position',
'reference_face_distance',
'reference_frame_number',
'face_tracker_score',
'face_occluder_model',
'face_parser_model',
'face_mask_types',
@@ -302,6 +314,7 @@ StateKey = Literal\
'trim_frame_end',
'temp_frame_format',
'keep_temp',
'target_frame_amount',
'output_image_quality',
'output_image_scale',
'output_audio_encoder',
@@ -320,7 +333,6 @@ StateKey = Literal\
'execution_providers',
'execution_thread_count',
'video_memory_strategy',
'system_memory_limit',
'log_level',
'halt_on_error',
'job_id',
@@ -346,20 +358,21 @@ State = TypedDict('State',
'benchmark_cycle_count' : int,
'face_detector_model' : FaceDetectorModel,
'face_detector_size' : str,
'face_detector_margin': Margin,
'face_detector_margin' : Margin,
'face_detector_angles' : List[Angle],
'face_detector_score' : Score,
'face_landmarker_model' : FaceLandmarkerModel,
'face_landmarker_score' : Score,
'face_selector_mode' : FaceSelectorMode,
'face_selector_order' : FaceSelectorOrder,
'face_selector_race' : Race,
'face_selector_gender' : Gender,
'face_selector_race' : FaceSelectorRace,
'face_selector_gender' : FaceSelectorGender,
'face_selector_age_start' : int,
'face_selector_age_end' : int,
'reference_face_position' : int,
'reference_face_distance' : float,
'reference_frame_number' : int,
'face_tracker_score' : Score,
'face_occluder_model' : FaceOccluderModel,
'face_parser_model' : FaceParserModel,
'face_mask_types' : List[FaceMaskType],
@@ -367,11 +380,12 @@ State = TypedDict('State',
'face_mask_regions' : List[FaceMaskRegion],
'face_mask_blur' : float,
'face_mask_padding' : Padding,
'voice_extractor_model': VoiceExtractorModel,
'voice_extractor_model' : VoiceExtractorModel,
'trim_frame_start' : int,
'trim_frame_end' : int,
'temp_frame_format' : TempFrameFormat,
'keep_temp' : bool,
'target_frame_amount' : int,
'output_image_quality' : int,
'output_image_scale' : Scale,
'output_audio_encoder' : AudioEncoder,
@@ -390,7 +404,6 @@ State = TypedDict('State',
'execution_providers' : List[ExecutionProvider],
'execution_thread_count' : int,
'video_memory_strategy' : VideoMemoryStrategy,
'system_memory_limit' : int,
'log_level' : LogLevel,
'halt_on_error' : bool,
'job_id' : str,
+6 -2
View File
@@ -62,10 +62,14 @@
width: 1.125rem;
}
:root:root:root:root .thumbnail-item
:root:root:root:root .gallery-container .thumbnail-item
{
border: unset;
box-shadow: unset;
}
:root:root:root:root .gallery-container .grid-container
{
grid-template-columns: repeat(7, 1fr);
}
:root:root:root:root .grid-wrap.fixed-height
+23 -18
View File
@@ -7,15 +7,15 @@ from gradio_rangeslider import RangeSlider
import facefusion.choices
from facefusion import state_manager, translator
from facefusion.common_helper import calculate_float_step, calculate_int_step
from facefusion.face_analyser import get_many_faces
from facefusion.face_creator import get_many_faces
from facefusion.face_selector import sort_and_filter_faces
from facefusion.face_store import clear_static_faces
from facefusion.filesystem import is_image, is_video
from facefusion.types import FaceSelectorMode, FaceSelectorOrder, Gender, Race, VisionFrame
from facefusion.face_store import clear_faces
from facefusion.filesystem import filter_image_paths, is_image, is_video
from facefusion.types import FaceSelectorGender, FaceSelectorMode, FaceSelectorOrder, FaceSelectorRace, VisionFrame
from facefusion.uis.core import get_ui_component, get_ui_components, register_ui_component
from facefusion.uis.types import ComponentOptions
from facefusion.uis.ui_helper import convert_str_none
from facefusion.vision import fit_cover_frame, read_static_image, read_video_frame
from facefusion.vision import fit_cover_frame, read_static_image, read_static_images, read_video_frame
FACE_SELECTOR_MODE_DROPDOWN : Optional[gradio.Dropdown] = None
FACE_SELECTOR_ORDER_DROPDOWN : Optional[gradio.Dropdown] = None
@@ -39,17 +39,18 @@ def render() -> None:
{
'label': translator.get('uis.reference_face_gallery'),
'object_fit': 'cover',
'columns': 7,
'allow_preview': False,
'elem_classes': 'box-face-selector',
'visible': 'reference' in state_manager.get_item('face_selector_mode')
}
source_vision_frames = read_static_images(filter_image_paths(state_manager.get_item('source_paths')))
if is_image(state_manager.get_item('target_path')):
target_vision_frame = read_static_image(state_manager.get_item('target_path'))
reference_face_gallery_options['value'] = extract_gallery_frames(target_vision_frame)
reference_face_gallery_options['value'] = extract_gallery_frames(source_vision_frames, target_vision_frame)
if is_video(state_manager.get_item('target_path')):
target_vision_frame = read_video_frame(state_manager.get_item('target_path'), state_manager.get_item('reference_frame_number'))
reference_face_gallery_options['value'] = extract_gallery_frames(target_vision_frame)
reference_face_gallery_options['value'] = extract_gallery_frames(source_vision_frames, target_vision_frame)
FACE_SELECTOR_MODE_DROPDOWN = gradio.Dropdown(
label = translator.get('uis.face_selector_mode_dropdown'),
choices = facefusion.choices.face_selector_modes,
@@ -156,12 +157,12 @@ def update_face_selector_order(face_analyser_order : FaceSelectorOrder) -> gradi
return update_reference_position_gallery()
def update_face_selector_gender(face_selector_gender : Gender) -> gradio.Gallery:
def update_face_selector_gender(face_selector_gender : FaceSelectorGender) -> gradio.Gallery:
state_manager.set_item('face_selector_gender', convert_str_none(face_selector_gender))
return update_reference_position_gallery()
def update_face_selector_race(face_selector_race : Race) -> gradio.Gallery:
def update_face_selector_race(face_selector_race : FaceSelectorRace) -> gradio.Gallery:
state_manager.set_item('face_selector_race', convert_str_none(face_selector_race))
return update_reference_position_gallery()
@@ -194,30 +195,33 @@ def clear_reference_frame_number() -> None:
def clear_and_update_reference_position_gallery() -> gradio.Gallery:
clear_static_faces()
clear_faces()
return update_reference_position_gallery()
def update_reference_position_gallery(frame_number : int = 0) -> gradio.Gallery:
gallery_vision_frames = []
source_vision_frames = read_static_images(filter_image_paths(state_manager.get_item('source_paths')))
if is_image(state_manager.get_item('target_path')):
target_vision_frame = read_static_image(state_manager.get_item('target_path'))
gallery_vision_frames = extract_gallery_frames(target_vision_frame)
gallery_vision_frames = extract_gallery_frames(source_vision_frames, target_vision_frame)
if is_video(state_manager.get_item('target_path')):
target_vision_frame = read_video_frame(state_manager.get_item('target_path'), frame_number)
gallery_vision_frames = extract_gallery_frames(target_vision_frame)
gallery_vision_frames = extract_gallery_frames(source_vision_frames, target_vision_frame)
if gallery_vision_frames:
return gradio.Gallery(value = gallery_vision_frames)
return gradio.Gallery(value = None)
def extract_gallery_frames(target_vision_frame : VisionFrame) -> List[VisionFrame]:
def extract_gallery_frames(source_vision_frames : List[VisionFrame], target_vision_frame : VisionFrame) -> List[VisionFrame]:
gallery_vision_frames = []
faces = get_many_faces([ target_vision_frame ])
faces = sort_and_filter_faces(faces)
source_faces = get_many_faces(source_vision_frames)
target_faces = get_many_faces([ target_vision_frame ])
target_faces = sort_and_filter_faces(source_faces, target_faces)
for face in faces:
start_x, start_y, end_x, end_y = map(int, face.bounding_box)
for target_face in target_faces:
start_x, start_y, end_x, end_y = map(int, target_face.bounding_box)
padding_x = int((end_x - start_x) * 0.25)
padding_y = int((end_y - start_y) * 0.25)
start_x = max(0, start_x - padding_x)
@@ -228,4 +232,5 @@ def extract_gallery_frames(target_vision_frame : VisionFrame) -> List[VisionFram
crop_vision_frame = fit_cover_frame(crop_vision_frame, (128, 128))
crop_vision_frame = cv2.cvtColor(crop_vision_frame, cv2.COLOR_BGR2RGB)
gallery_vision_frames.append(crop_vision_frame)
return gallery_vision_frames
+32
View File
@@ -0,0 +1,32 @@
from typing import Optional
import gradio
import facefusion.choices
from facefusion import state_manager, translator
from facefusion.common_helper import calculate_float_step
from facefusion.types import Score
from facefusion.uis.core import register_ui_component
FACE_TRACKER_SCORE_SLIDER : Optional[gradio.Slider] = None
def render() -> None:
global FACE_TRACKER_SCORE_SLIDER
FACE_TRACKER_SCORE_SLIDER = gradio.Slider(
label = translator.get('uis.face_tracker_score_slider'),
value = state_manager.get_item('face_tracker_score'),
step = calculate_float_step(facefusion.choices.face_tracker_score_range),
minimum = facefusion.choices.face_tracker_score_range[0],
maximum = facefusion.choices.face_tracker_score_range[-1]
)
register_ui_component('face_tracker_score_slider', FACE_TRACKER_SCORE_SLIDER)
def listen() -> None:
FACE_TRACKER_SCORE_SLIDER.release(update_face_tracker_score, inputs = FACE_TRACKER_SCORE_SLIDER)
def update_face_tracker_score(face_tracker_score : Score) -> None:
state_manager.set_item('face_tracker_score', face_tracker_score)
-15
View File
@@ -4,39 +4,24 @@ import gradio
import facefusion.choices
from facefusion import state_manager, translator
from facefusion.common_helper import calculate_int_step
from facefusion.types import VideoMemoryStrategy
VIDEO_MEMORY_STRATEGY_DROPDOWN : Optional[gradio.Dropdown] = None
SYSTEM_MEMORY_LIMIT_SLIDER : Optional[gradio.Slider] = None
def render() -> None:
global VIDEO_MEMORY_STRATEGY_DROPDOWN
global SYSTEM_MEMORY_LIMIT_SLIDER
VIDEO_MEMORY_STRATEGY_DROPDOWN = gradio.Dropdown(
label = translator.get('uis.video_memory_strategy_dropdown'),
choices = facefusion.choices.video_memory_strategies,
value = state_manager.get_item('video_memory_strategy')
)
SYSTEM_MEMORY_LIMIT_SLIDER = gradio.Slider(
label = translator.get('uis.system_memory_limit_slider'),
step = calculate_int_step(facefusion.choices.system_memory_limit_range),
minimum = facefusion.choices.system_memory_limit_range[0],
maximum = facefusion.choices.system_memory_limit_range[-1],
value = state_manager.get_item('system_memory_limit')
)
def listen() -> None:
VIDEO_MEMORY_STRATEGY_DROPDOWN.change(update_video_memory_strategy, inputs = VIDEO_MEMORY_STRATEGY_DROPDOWN)
SYSTEM_MEMORY_LIMIT_SLIDER.release(update_system_memory_limit, inputs = SYSTEM_MEMORY_LIMIT_SLIDER)
def update_video_memory_strategy(video_memory_strategy : VideoMemoryStrategy) -> None:
state_manager.set_item('video_memory_strategy', video_memory_strategy)
def update_system_memory_limit(system_memory_limit : float) -> None:
state_manager.set_item('system_memory_limit', int(system_memory_limit))
+27 -27
View File
@@ -7,18 +7,18 @@ import numpy
from facefusion import logger, process_manager, state_manager, translator
from facefusion.audio import create_empty_audio_frame, get_voice_frame
from facefusion.common_helper import get_first
from facefusion.common_helper import get_first, get_middle
from facefusion.content_analyser import analyse_frame
from facefusion.face_analyser import get_one_face
from facefusion.face_creator import get_one_face
from facefusion.face_selector import select_faces
from facefusion.face_store import clear_static_faces
from facefusion.face_store import clear_faces
from facefusion.filesystem import filter_audio_paths, is_image, is_video
from facefusion.processors.core import get_processors_modules
from facefusion.types import AudioFrame, Face, Mask, VisionFrame
from facefusion.uis import choices as uis_choices
from facefusion.uis.core import get_ui_component, get_ui_components, register_ui_component
from facefusion.uis.types import ComponentOptions, PreviewMode
from facefusion.vision import detect_frame_orientation, extract_vision_mask, fit_cover_frame, merge_vision_mask, obscure_frame, read_static_image, read_static_images, read_video_frame, restrict_frame, unpack_resolution
from facefusion.vision import detect_frame_orientation, extract_vision_mask, fit_cover_frame, merge_vision_mask, obscure_frame, read_static_image, read_static_images, read_video_frame, restrict_frame, select_video_frames, unpack_resolution
PREVIEW_IMAGE : Optional[gradio.Image] = None
@@ -36,7 +36,7 @@ def render() -> None:
source_audio_frame = create_empty_audio_frame()
source_voice_frame = create_empty_audio_frame()
if source_audio_path and state_manager.get_item('output_video_fps') and state_manager.get_item('reference_frame_number'):
if source_audio_path and state_manager.get_item('output_video_fps'):
temp_voice_frame = get_voice_frame(source_audio_path, state_manager.get_item('output_video_fps'), state_manager.get_item('reference_frame_number'))
if numpy.any(temp_voice_frame):
source_voice_frame = temp_voice_frame
@@ -44,14 +44,14 @@ def render() -> None:
if is_image(state_manager.get_item('target_path')):
target_vision_frame = read_static_image(state_manager.get_item('target_path'))
reference_vision_frame = read_static_image(state_manager.get_item('target_path'))
preview_vision_frame = process_preview_frame(reference_vision_frame, source_vision_frames, source_audio_frame, source_voice_frame, target_vision_frame, uis_choices.preview_modes[0], uis_choices.preview_resolutions[-1])
preview_vision_frame = process_preview_frame(reference_vision_frame, source_vision_frames, source_audio_frame, source_voice_frame, [ target_vision_frame ], uis_choices.preview_modes[0], uis_choices.preview_resolutions[-1])
preview_image_options['value'] = cv2.cvtColor(preview_vision_frame, cv2.COLOR_BGR2RGB)
preview_image_options['elem_classes'] = [ 'image-preview', 'is-' + detect_frame_orientation(preview_vision_frame) ]
if is_video(state_manager.get_item('target_path')):
temp_vision_frame = read_video_frame(state_manager.get_item('target_path'), state_manager.get_item('reference_frame_number'))
reference_vision_frame = read_video_frame(state_manager.get_item('target_path'), state_manager.get_item('reference_frame_number'))
preview_vision_frame = process_preview_frame(reference_vision_frame, source_vision_frames, source_audio_frame, source_voice_frame, temp_vision_frame, uis_choices.preview_modes[0], uis_choices.preview_resolutions[-1])
target_vision_frames = select_video_frames(state_manager.get_item('target_path'), state_manager.get_item('reference_frame_number'), state_manager.get_item('target_frame_amount'))
preview_vision_frame = process_preview_frame(reference_vision_frame, source_vision_frames, source_audio_frame, source_voice_frame, target_vision_frames, uis_choices.preview_modes[0], uis_choices.preview_resolutions[-1])
preview_image_options['value'] = cv2.cvtColor(preview_vision_frame, cv2.COLOR_BGR2RGB)
preview_image_options['elem_classes'] = [ 'image-preview', 'is-' + detect_frame_orientation(preview_vision_frame) ]
preview_image_options['visible'] = True
@@ -134,6 +134,7 @@ def listen() -> None:
'lip_syncer_weight_slider',
'reference_face_distance_slider',
'face_selector_age_range_slider',
'face_tracker_score_slider',
'face_mask_blur_slider',
'face_mask_padding_top_slider',
'face_mask_padding_bottom_slider',
@@ -189,43 +190,42 @@ def update_preview_image(preview_mode : PreviewMode, preview_resolution : str, f
source_audio_frame = create_empty_audio_frame()
source_voice_frame = create_empty_audio_frame()
if source_audio_path and state_manager.get_item('output_video_fps') and state_manager.get_item('reference_frame_number'):
reference_audio_frame_number = state_manager.get_item('reference_frame_number')
if source_audio_path and state_manager.get_item('output_video_fps'):
audio_frame_number = frame_number
if state_manager.get_item('trim_frame_start'):
reference_audio_frame_number -= state_manager.get_item('trim_frame_start')
temp_voice_frame = get_voice_frame(source_audio_path, state_manager.get_item('output_video_fps'), reference_audio_frame_number)
audio_frame_number -= state_manager.get_item('trim_frame_start')
temp_voice_frame = get_voice_frame(source_audio_path, state_manager.get_item('output_video_fps'), audio_frame_number)
if numpy.any(temp_voice_frame):
source_voice_frame = temp_voice_frame
if is_image(state_manager.get_item('target_path')):
reference_vision_frame = read_static_image(state_manager.get_item('target_path'))
target_vision_frame = read_static_image(state_manager.get_item('target_path'), 'rgba')
target_vision_mask = extract_vision_mask(target_vision_frame)
target_vision_frame = merge_vision_mask(target_vision_frame, target_vision_mask)
preview_vision_frame = process_preview_frame(reference_vision_frame, source_vision_frames, source_audio_frame, source_voice_frame, target_vision_frame, preview_mode, preview_resolution)
preview_vision_frame = process_preview_frame(reference_vision_frame, source_vision_frames, source_audio_frame, source_voice_frame, [ target_vision_frame ], preview_mode, preview_resolution)
preview_vision_frame = cv2.cvtColor(preview_vision_frame, cv2.COLOR_BGRA2RGBA)
return gradio.Image(value = preview_vision_frame, elem_classes = [ 'image-preview', 'is-' + detect_frame_orientation(preview_vision_frame) ])
if is_video(state_manager.get_item('target_path')):
reference_vision_frame = read_video_frame(state_manager.get_item('target_path'), state_manager.get_item('reference_frame_number'))
temp_vision_frame = read_video_frame(state_manager.get_item('target_path'), frame_number)
temp_vision_mask = extract_vision_mask(temp_vision_frame)
temp_vision_frame = merge_vision_mask(temp_vision_frame, temp_vision_mask)
preview_vision_frame = process_preview_frame(reference_vision_frame, source_vision_frames, source_audio_frame, source_voice_frame, temp_vision_frame, preview_mode, preview_resolution)
target_vision_frames = select_video_frames(state_manager.get_item('target_path'), frame_number, state_manager.get_item('target_frame_amount'))
preview_vision_frame = process_preview_frame(reference_vision_frame, source_vision_frames, source_audio_frame, source_voice_frame, target_vision_frames, preview_mode, preview_resolution)
preview_vision_frame = cv2.cvtColor(preview_vision_frame, cv2.COLOR_BGRA2RGBA)
return gradio.Image(value = preview_vision_frame, elem_classes = [ 'image-preview', 'is-' + detect_frame_orientation(preview_vision_frame) ])
return gradio.Image(value = None, elem_classes = None)
def clear_and_update_preview_image(preview_mode : PreviewMode, preview_resolution : str, frame_number : int = 0) -> gradio.Image:
clear_static_faces()
clear_faces()
return update_preview_image(preview_mode, preview_resolution, frame_number)
def process_preview_frame(reference_vision_frame : VisionFrame, source_vision_frames : List[VisionFrame], source_audio_frame : AudioFrame, source_voice_frame : AudioFrame, target_vision_frame : VisionFrame, preview_mode : PreviewMode, preview_resolution : str) -> VisionFrame:
def process_preview_frame(reference_vision_frame : VisionFrame, source_vision_frames : List[VisionFrame], source_audio_frame : AudioFrame, source_voice_frame : AudioFrame, target_vision_frames : List[VisionFrame], preview_mode : PreviewMode, preview_resolution : str) -> VisionFrame:
target_vision_frame = get_middle(target_vision_frames)
target_vision_frame = restrict_frame(target_vision_frame, unpack_resolution(preview_resolution))
temp_vision_mask = extract_vision_mask(target_vision_frame)
target_vision_frame = merge_vision_mask(target_vision_frame, temp_vision_mask)
target_vision_frames = [ restrict_frame(vision_frame, unpack_resolution(preview_resolution))[:, :, :3] for vision_frame in target_vision_frames ]
temp_vision_frame = target_vision_frame.copy()
temp_vision_mask = extract_vision_mask(temp_vision_frame)
if analyse_frame(target_vision_frame[:, :, :3]):
if preview_mode == 'frame-by-frame':
@@ -233,7 +233,7 @@ def process_preview_frame(reference_vision_frame : VisionFrame, source_vision_fr
return numpy.hstack((temp_vision_frame, temp_vision_frame))
if preview_mode == 'face-by-face':
target_crop_vision_frame, output_crop_vision_frame = create_face_by_face(reference_vision_frame, target_vision_frame[:, :, :3], temp_vision_frame[:, :, :3])
target_crop_vision_frame, output_crop_vision_frame = create_face_by_face(reference_vision_frame, source_vision_frames, target_vision_frame[:, :, :3], temp_vision_frame[:, :, :3])
target_crop_vision_frame = obscure_frame(target_crop_vision_frame)
output_crop_vision_frame = obscure_frame(output_crop_vision_frame)
return numpy.hstack((target_crop_vision_frame, output_crop_vision_frame))
@@ -251,7 +251,7 @@ def process_preview_frame(reference_vision_frame : VisionFrame, source_vision_fr
'source_audio_frame': source_audio_frame,
'source_voice_frame': source_voice_frame,
'source_vision_frames': source_vision_frames,
'target_vision_frame': target_vision_frame[:, :, :3],
'target_vision_frames': target_vision_frames,
'temp_vision_frame': temp_vision_frame[:, :, :3],
'temp_vision_mask': temp_vision_mask
})
@@ -263,14 +263,14 @@ def process_preview_frame(reference_vision_frame : VisionFrame, source_vision_fr
return numpy.hstack((target_vision_frame, temp_vision_frame))
if preview_mode == 'face-by-face':
target_crop_vision_frame, output_crop_vision_frame = create_face_by_face(reference_vision_frame, target_vision_frame, temp_vision_frame)
target_crop_vision_frame, output_crop_vision_frame = create_face_by_face(reference_vision_frame, source_vision_frames, target_vision_frame, temp_vision_frame)
return numpy.hstack((target_crop_vision_frame, output_crop_vision_frame))
return temp_vision_frame
def create_face_by_face(reference_vision_frame : VisionFrame, target_vision_frame : VisionFrame, temp_vision_frame : VisionFrame) -> Tuple[VisionFrame, VisionFrame]:
target_faces = select_faces(reference_vision_frame[:, :, :3], target_vision_frame[:, :, :3])
def create_face_by_face(reference_vision_frame : VisionFrame, source_vision_frames : List[VisionFrame], target_vision_frame : VisionFrame, temp_vision_frame : VisionFrame) -> Tuple[VisionFrame, VisionFrame]:
target_faces = select_faces(reference_vision_frame[:, :, :3], source_vision_frames, [ target_vision_frame[:, :, :3] ])
target_face = get_one_face(target_faces)
if target_face:
+2 -2
View File
@@ -3,7 +3,7 @@ from typing import Optional, Tuple
import gradio
from facefusion import state_manager, translator
from facefusion.face_store import clear_static_faces
from facefusion.face_store import clear_faces
from facefusion.filesystem import is_image, is_video
from facefusion.uis.core import register_ui_component
from facefusion.uis.types import ComponentOptions, File
@@ -51,7 +51,7 @@ def listen() -> None:
def update(file : File) -> Tuple[gradio.Image, gradio.Video]:
clear_static_faces()
clear_faces()
if file and is_image(file.name):
state_manager.set_item('target_path', file.name)
+2 -2
View File
@@ -3,7 +3,7 @@ from typing import Optional, Tuple
from gradio_rangeslider import RangeSlider
from facefusion import state_manager, translator
from facefusion.face_store import clear_static_faces
from facefusion.face_store import clear_faces
from facefusion.filesystem import is_video
from facefusion.uis.core import get_ui_components
from facefusion.uis.types import ComponentOptions
@@ -53,7 +53,7 @@ def remote_update() -> RangeSlider:
def update_trim_frame(trim_frame : Tuple[float, float]) -> None:
clear_static_faces()
clear_faces()
trim_frame_start, trim_frame_end = trim_frame
video_frame_total = count_video_frame_total(state_manager.get_item('target_path'))
trim_frame_start = int(trim_frame_start) if trim_frame_start > 0 else None
+18 -12
View File
@@ -1,4 +1,4 @@
from typing import Optional
from typing import List, Optional
import gradio
@@ -6,7 +6,7 @@ import facefusion.choices
from facefusion import state_manager, translator, voice_extractor
from facefusion.filesystem import is_video
from facefusion.types import VoiceExtractorModel
from facefusion.uis.core import get_ui_components, register_ui_component
from facefusion.uis.core import get_ui_component, get_ui_components, register_ui_component
VOICE_EXTRACTOR_MODEL_DROPDOWN : Optional[gradio.Dropdown] = None
@@ -14,11 +14,12 @@ VOICE_EXTRACTOR_MODEL_DROPDOWN : Optional[gradio.Dropdown] = None
def render() -> None:
global VOICE_EXTRACTOR_MODEL_DROPDOWN
has_lip_syncer = 'lip_syncer' in state_manager.get_item('processors')
VOICE_EXTRACTOR_MODEL_DROPDOWN = gradio.Dropdown(
label = translator.get('uis.voice_extractor_model_dropdown'),
choices = facefusion.choices.voice_extractor_models,
value = state_manager.get_item('voice_extractor_model'),
visible = is_video(state_manager.get_item('target_path'))
visible = is_video(state_manager.get_item('target_path')) and has_lip_syncer
)
register_ui_component('voice_extractor_model_dropdown', VOICE_EXTRACTOR_MODEL_DROPDOWN)
@@ -26,17 +27,22 @@ def render() -> None:
def listen() -> None:
VOICE_EXTRACTOR_MODEL_DROPDOWN.change(update_voice_extractor_model, inputs = VOICE_EXTRACTOR_MODEL_DROPDOWN, outputs = VOICE_EXTRACTOR_MODEL_DROPDOWN)
for ui_component in get_ui_components(
[
'target_image',
'target_video'
]):
for method in [ 'change', 'clear' ]:
getattr(ui_component, method)(remote_update, outputs = VOICE_EXTRACTOR_MODEL_DROPDOWN)
processors_checkbox_group = get_ui_component('processors_checkbox_group')
if processors_checkbox_group:
processors_checkbox_group.change(remote_update, inputs = processors_checkbox_group, outputs = VOICE_EXTRACTOR_MODEL_DROPDOWN)
for ui_component in get_ui_components(
[
'target_image',
'target_video'
]):
for method in [ 'change', 'clear' ]:
getattr(ui_component, method)(remote_update, inputs = processors_checkbox_group, outputs = VOICE_EXTRACTOR_MODEL_DROPDOWN)
def remote_update() -> gradio.Dropdown:
if is_video(state_manager.get_item('target_path')):
def remote_update(processors : List[str]) -> gradio.Dropdown:
has_lip_syncer = 'lip_syncer' in processors
if is_video(state_manager.get_item('target_path')) and has_lip_syncer:
return gradio.Dropdown(visible = True)
return gradio.Dropdown(visible = False)
+4 -1
View File
@@ -1,7 +1,7 @@
import gradio
from facefusion import state_manager
from facefusion.uis.components import about, age_modifier_options, background_remover_options, common_options, deep_swapper_options, download, execution, execution_thread_count, expression_restorer_options, face_debugger_options, face_detector, face_editor_options, face_enhancer_options, face_landmarker, face_masker, face_selector, face_swapper_options, frame_colorizer_options, frame_enhancer_options, instant_runner, job_manager, job_runner, lip_syncer_options, memory, output, output_options, preview, preview_options, processors, source, target, temp_frame, terminal, trim_frame, ui_workflow, voice_extractor
from facefusion.uis.components import about, age_modifier_options, background_remover_options, common_options, deep_swapper_options, download, execution, execution_thread_count, expression_restorer_options, face_debugger_options, face_detector, face_editor_options, face_enhancer_options, face_landmarker, face_masker, face_selector, face_swapper_options, face_tracker, frame_colorizer_options, frame_enhancer_options, instant_runner, job_manager, job_runner, lip_syncer_options, memory, output, output_options, preview, preview_options, processors, source, target, temp_frame, terminal, trim_frame, ui_workflow, voice_extractor
def pre_check() -> bool:
@@ -73,6 +73,8 @@ def render() -> gradio.Blocks:
trim_frame.render()
with gradio.Blocks():
face_selector.render()
with gradio.Blocks():
face_tracker.render()
with gradio.Blocks():
face_masker.render()
with gradio.Blocks():
@@ -114,6 +116,7 @@ def listen() -> None:
preview_options.listen()
trim_frame.listen()
face_selector.listen()
face_tracker.listen()
face_masker.listen()
face_detector.listen()
face_landmarker.listen()
+1
View File
@@ -58,6 +58,7 @@ ComponentName = Literal\
'face_selector_mode_dropdown',
'face_selector_order_dropdown',
'face_selector_race_dropdown',
'face_tracker_score_slider',
'face_swapper_model_dropdown',
'face_swapper_pixel_boost_dropdown',
'face_swapper_weight_slider',
+53 -3
View File
@@ -1,6 +1,6 @@
import math
from functools import lru_cache
from typing import List, Optional, Tuple
from typing import Dict, List, Optional, Tuple
import cv2
import numpy
@@ -8,7 +8,7 @@ from cv2.typing import Size
from facefusion.common_helper import is_windows
from facefusion.filesystem import get_file_extension, is_image, is_video
from facefusion.thread_helper import thread_semaphore
from facefusion.thread_helper import thread_lock, thread_semaphore
from facefusion.types import ColorMode, Duration, Fps, Mask, Orientation, Resolution, Scale, VisionFrame
from facefusion.video_manager import get_video_capture
@@ -81,9 +81,10 @@ def read_video_frame(video_path : str, frame_number : int = 0) -> Optional[Visio
if video_capture and video_capture.isOpened():
frame_total = video_capture.get(cv2.CAP_PROP_FRAME_COUNT)
frame_position = min(frame_total, frame_number)
with thread_semaphore():
video_capture.set(cv2.CAP_PROP_POS_FRAMES, min(frame_total, frame_number - 1))
video_capture.set(cv2.CAP_PROP_POS_FRAMES, frame_position)
has_vision_frame, vision_frame = video_capture.read()
if has_vision_frame:
@@ -92,6 +93,51 @@ def read_video_frame(video_path : str, frame_number : int = 0) -> Optional[Visio
return None
@lru_cache(maxsize = 2)
def read_static_video_chunk(video_path : str, chunk_number : int, chunk_size : int) -> Dict[int, VisionFrame]:
return read_video_chunk(video_path, chunk_number, chunk_size)
def read_video_chunk(video_path : str, chunk_number : int, chunk_size : int) -> Dict[int, VisionFrame]:
video_frame_chunk = {}
if is_video(video_path) and chunk_number > -1:
video_capture = get_video_capture(video_path)
if video_capture and video_capture.isOpened():
frame_total = int(video_capture.get(cv2.CAP_PROP_FRAME_COUNT))
frame_position = chunk_number * chunk_size
with thread_semaphore():
video_capture.set(cv2.CAP_PROP_POS_FRAMES, frame_position)
for frame_number in range(frame_position, min(frame_position + chunk_size, frame_total)):
has_vision_frame, vision_frame = video_capture.read()
if has_vision_frame:
video_frame_chunk[frame_number] = vision_frame
return video_frame_chunk
def select_video_frames(video_path : str, frame_number : int = 0, frame_offset : int = 5) -> List[VisionFrame]:
vision_frames = []
chunk_size = (frame_offset * 2 + 1) * 4
if is_video(video_path):
with thread_lock():
for current_number in range(frame_number - frame_offset, frame_number + frame_offset + 1):
video_frame_chunk = read_static_video_chunk(video_path, current_number // chunk_size, chunk_size)
vision_frame = create_empty_vision_frame()
if current_number in video_frame_chunk:
vision_frame = video_frame_chunk.get(current_number)
vision_frames.append(vision_frame)
return vision_frames
def count_video_frame_total(video_path : str) -> int:
if is_video(video_path):
video_capture = get_video_capture(video_path)
@@ -307,6 +353,10 @@ def blend_vision_frames(source_vision_frame : VisionFrame, target_vision_frame :
return blend_vision_frame
def create_empty_vision_frame() -> VisionFrame:
return numpy.zeros((1, 1, 3)).astype(numpy.uint8)
def create_tile_frames(vision_frame : VisionFrame, size : Size) -> Tuple[List[VisionFrame], int, int]:
tile_width = size[0] - 2 * size[2]
pad_size_top = size[1] + size[2]
+8 -7
View File
@@ -38,10 +38,12 @@ def setup() -> ErrorCode:
if analyse_image(state_manager.get_item('target_path')):
return 3
logger.debug(translator.get('clearing_temp'), __name__)
clear_temp_directory(state_manager.get_item('target_path'))
logger.debug(translator.get('creating_temp'), __name__)
create_temp_directory(state_manager.get_item('target_path'))
if clear_temp_directory(state_manager.get_item('target_path')):
logger.debug(translator.get('clearing_temp'), __name__)
if create_temp_directory(state_manager.get_item('target_path')):
logger.debug(translator.get('creating_temp'), __name__)
return 0
@@ -65,8 +67,7 @@ def process_image() -> ErrorCode:
source_vision_frames = read_static_images(state_manager.get_item('source_paths'))
source_audio_frame = create_empty_audio_frame()
source_voice_frame = create_empty_audio_frame()
target_vision_frame = read_static_image(temp_image_path, 'rgba')
temp_vision_frame = target_vision_frame.copy()
temp_vision_frame = read_static_image(temp_image_path, 'rgba')
temp_vision_mask = extract_vision_mask(temp_vision_frame)
for processor_module in get_processors_modules(state_manager.get_item('processors')):
@@ -78,7 +79,7 @@ def process_image() -> ErrorCode:
'source_vision_frames': source_vision_frames,
'source_audio_frame': source_audio_frame,
'source_voice_frame': source_voice_frame,
'target_vision_frame': target_vision_frame[:, :, :3],
'target_vision_frames': [ temp_vision_frame[:, :, :3] ],
'temp_vision_frame': temp_vision_frame[:, :, :3],
'temp_vision_mask': temp_vision_mask
})
+18 -15
View File
@@ -11,10 +11,10 @@ from facefusion.common_helper import get_first
from facefusion.content_analyser import analyse_video
from facefusion.filesystem import filter_audio_paths, is_video
from facefusion.processors.core import get_processors_modules
from facefusion.temp_helper import clear_temp_directory, create_temp_directory, move_temp_file, resolve_temp_frame_paths
from facefusion.temp_helper import clear_temp_directory, create_temp_directory, move_temp_file, resolve_temp_frame_set
from facefusion.time_helper import calculate_end_time
from facefusion.types import ErrorCode
from facefusion.vision import conditional_merge_vision_mask, detect_video_resolution, extract_vision_mask, pack_resolution, read_static_image, read_static_images, read_static_video_frame, restrict_trim_frame, restrict_video_fps, restrict_video_resolution, scale_resolution, write_image
from facefusion.vision import conditional_merge_vision_mask, detect_video_resolution, extract_vision_mask, pack_resolution, read_static_image, read_static_images, read_static_video_frame, restrict_trim_frame, restrict_video_fps, restrict_video_resolution, scale_resolution, select_video_frames, write_image
from facefusion.workflows.core import is_process_stopping
@@ -47,10 +47,12 @@ def setup() -> ErrorCode:
if analyse_video(state_manager.get_item('target_path'), trim_frame_start, trim_frame_end):
return 3
logger.debug(translator.get('clearing_temp'), __name__)
clear_temp_directory(state_manager.get_item('target_path'))
logger.debug(translator.get('creating_temp'), __name__)
create_temp_directory(state_manager.get_item('target_path'))
if clear_temp_directory(state_manager.get_item('target_path')):
logger.debug(translator.get('clearing_temp'), __name__)
if create_temp_directory(state_manager.get_item('target_path')):
logger.debug(translator.get('creating_temp'), __name__)
return 0
@@ -72,16 +74,16 @@ def extract_frames() -> ErrorCode:
def process_video() -> ErrorCode:
temp_frame_paths = resolve_temp_frame_paths(state_manager.get_item('target_path'))
temp_frame_set = resolve_temp_frame_set(state_manager.get_item('target_path'))
if temp_frame_paths:
with tqdm(total = len(temp_frame_paths), desc = translator.get('processing'), unit = 'frame', ascii = ' =', disable = state_manager.get_item('log_level') in [ 'warn', 'error' ]) as progress:
if temp_frame_set:
with tqdm(total = len(temp_frame_set), desc = translator.get('processing'), unit = 'frame', ascii = ' =', disable = state_manager.get_item('log_level') in [ 'warn', 'error' ]) as progress:
progress.set_postfix(execution_providers = state_manager.get_item('execution_providers'))
with ThreadPoolExecutor(max_workers = state_manager.get_item('execution_thread_count')) as executor:
futures = []
for frame_number, temp_frame_path in enumerate(temp_frame_paths):
for frame_number, temp_frame_path in temp_frame_set.items():
future = executor.submit(process_temp_frame, temp_frame_path, frame_number)
futures.append(future)
@@ -153,16 +155,17 @@ def restore_audio() -> ErrorCode:
def process_temp_frame(temp_frame_path : str, frame_number : int) -> bool:
trim_frame_start, _ = restrict_trim_frame(state_manager.get_item('target_path'), state_manager.get_item('trim_frame_start'), state_manager.get_item('trim_frame_end'))
reference_vision_frame = read_static_video_frame(state_manager.get_item('target_path'), state_manager.get_item('reference_frame_number'))
source_vision_frames = read_static_images(state_manager.get_item('source_paths'))
source_audio_path = get_first(filter_audio_paths(state_manager.get_item('source_paths')))
target_vision_frames = select_video_frames(state_manager.get_item('target_path'), frame_number, state_manager.get_item('target_frame_amount'))
temp_video_fps = restrict_video_fps(state_manager.get_item('target_path'), state_manager.get_item('output_video_fps'))
target_vision_frame = read_static_image(temp_frame_path, 'rgba')
temp_vision_frame = target_vision_frame.copy()
temp_vision_frame = read_static_image(temp_frame_path, 'rgba')
temp_vision_mask = extract_vision_mask(temp_vision_frame)
source_audio_frame = get_audio_frame(source_audio_path, temp_video_fps, frame_number)
source_voice_frame = get_voice_frame(source_audio_path, temp_video_fps, frame_number)
source_audio_frame = get_audio_frame(source_audio_path, temp_video_fps, frame_number - trim_frame_start)
source_voice_frame = get_voice_frame(source_audio_path, temp_video_fps, frame_number - trim_frame_start)
if not numpy.any(source_audio_frame):
source_audio_frame = create_empty_audio_frame()
@@ -176,7 +179,7 @@ def process_temp_frame(temp_frame_path : str, frame_number : int) -> bool:
'source_vision_frames': source_vision_frames,
'source_audio_frame': source_audio_frame,
'source_voice_frame': source_voice_frame,
'target_vision_frame': target_vision_frame[:, :, :3],
'target_vision_frames': target_vision_frames,
'temp_vision_frame': temp_vision_frame[:, :, :3],
'temp_vision_mask': temp_vision_mask
})
+2 -2
View File
@@ -1,8 +1,8 @@
gradio-rangeslider==0.0.8
gradio==5.44.1
gradio==5.50.0
numpy==2.2.1
onnx==1.21.0
onnxruntime==1.24.4
onnxruntime==1.26.0
opencv-python==4.13.0.92
tqdm==4.67.3
scipy==1.17.1
+2 -2
View File
@@ -17,8 +17,8 @@ def before_all() -> None:
def test_get_audio_frame() -> None:
assert hasattr(get_audio_frame(get_test_example_file('source.mp3'), 25), '__array_interface__')
assert hasattr(get_audio_frame(get_test_example_file('source.wav'), 25), '__array_interface__')
assert get_audio_frame(get_test_example_file('source.mp3'), 25).shape == (80, 16)
assert get_audio_frame(get_test_example_file('source.wav'), 25).shape == (80, 16)
assert get_audio_frame('invalid', 25) is None
+7 -2
View File
@@ -1,4 +1,4 @@
from facefusion.common_helper import calculate_float_step, calculate_int_step, create_float_metavar, create_float_range, create_int_metavar, create_int_range
from facefusion.common_helper import calculate_float_step, calculate_int_step, create_float_metavar, create_float_range, create_int_metavar, create_int_range, get_middle
def test_create_int_metavar() -> None:
@@ -16,7 +16,7 @@ def test_create_int_range() -> None:
def test_create_float_range() -> None:
assert create_float_range(0.0, 1.0, 0.5) == [ 0.0, 0.5, 1.0 ]
assert create_float_range(0.0, 1.0, 0.05) == [ 0.0, 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95, 1.0 ]
assert create_float_range(0.0, 0.5, 0.05) == [ 0.0, 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, 0.50 ]
def test_calc_int_step() -> None:
@@ -25,3 +25,8 @@ def test_calc_int_step() -> None:
def test_calc_float_step() -> None:
assert calculate_float_step([ 0.1, 0.2 ]) == 0.1
def test_get_middle() -> None:
assert get_middle([ 1, 2, 3, 4, 5 ]) == 3
assert get_middle([ 1 ]) == 1
+4 -5
View File
@@ -1,14 +1,13 @@
from configparser import ConfigParser
import pytest
from facefusion import config
from facefusion import config, state_manager
@pytest.fixture(scope = 'module', autouse = True)
def before_all() -> None:
config.CONFIG_PARSER = ConfigParser()
config.CONFIG_PARSER.read_dict(
state_manager.init_item('config_path', 'facefusion.ini')
config_parser = config.get_static_config_parser()
config_parser.read_dict(
{
'str':
{
+105
View File
@@ -0,0 +1,105 @@
import subprocess
import numpy
import pytest
from facefusion import face_classifier, face_detector, face_landmarker, face_recognizer, state_manager
from facefusion.download import conditional_download
from facefusion.face_creator import average_face_geometry, get_many_faces, get_one_face, refill_faces
from facefusion.face_store import clear_faces
from facefusion.vision import read_static_image
from .helper import get_test_example_file, get_test_examples_directory
@pytest.fixture(scope = 'module', autouse = True)
def before_all() -> None:
conditional_download(get_test_examples_directory(),
[
'https://github.com/facefusion/facefusion-assets/releases/download/examples-3.0.0/source.jpg'
])
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('source.jpg'), '-vf', 'crop=iw*0.8:ih*0.8', get_test_example_file('source-80crop.jpg') ])
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('source.jpg'), '-vf', 'crop=iw*0.7:ih*0.7', get_test_example_file('source-70crop.jpg') ])
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('source.jpg'), '-vf', 'crop=iw*0.6:ih*0.6', get_test_example_file('source-60crop.jpg') ])
state_manager.init_item('execution_device_ids', [ 0 ])
state_manager.init_item('execution_providers', [ 'cpu' ])
state_manager.init_item('download_providers', [ 'github' ])
state_manager.init_item('face_detector_angles', [ 0 ])
state_manager.init_item('face_detector_model', 'many')
state_manager.init_item('face_detector_size', '640x640')
state_manager.init_item('face_detector_margin', (0, 0, 0, 0))
state_manager.init_item('face_detector_score', 0.5)
state_manager.init_item('face_landmarker_model', 'many')
state_manager.init_item('face_landmarker_score', 0.5)
face_classifier.pre_check()
face_detector.pre_check()
face_landmarker.pre_check()
face_recognizer.pre_check()
@pytest.fixture(autouse = True)
def before_each() -> None:
face_classifier.clear_inference_pool()
face_detector.clear_inference_pool()
face_landmarker.clear_inference_pool()
face_recognizer.clear_inference_pool()
clear_faces()
def test_get_one_face() -> None:
source_vision_frame = read_static_image(get_test_example_file('source.jpg'))
face = get_one_face(get_many_faces([ source_vision_frame ]))
assert face.bounding_box.size == 4
def test_get_many_faces() -> None:
source_path = get_test_example_file('source.jpg')
source_vision_frame = read_static_image(source_path)
many_faces = get_many_faces([ source_vision_frame, source_vision_frame, source_vision_frame ])
assert len(many_faces) == 3
def test_refill_faces() -> None:
source_vision_frame = read_static_image(get_test_example_file('source.jpg'))
face = get_one_face(get_many_faces([ source_vision_frame ]))
face_first = face._replace(bounding_box = numpy.array([ 0, 0, 10, 10 ]))
face_middle = face._replace(bounding_box = numpy.array([ 40, 40, 50, 50 ]))
face_last = face._replace(bounding_box = numpy.array([ 80, 80, 90, 90 ]))
fill_faces = refill_faces([ face_first, None, face_last ])
assert fill_faces[0].bounding_box.tolist() == [ 0.0, 0.0, 10.0, 10.0 ]
assert fill_faces[1].bounding_box.tolist() == [ 40.0, 40.0, 50.0, 50.0 ]
assert fill_faces[2].bounding_box.tolist() == [ 80.0, 80.0, 90.0, 90.0 ]
fill_faces = refill_faces([ face_first, None, None, None, face_last ])
assert fill_faces[0].bounding_box.tolist() == [ 0.0, 0.0, 10.0, 10.0 ]
assert fill_faces[1].bounding_box.tolist() == [ 20.0, 20.0, 30.0, 30.0 ]
assert fill_faces[2].bounding_box.tolist() == [ 40.0, 40.0, 50.0, 50.0 ]
assert fill_faces[3].bounding_box.tolist() == [ 60.0, 60.0, 70.0, 70.0 ]
assert fill_faces[4].bounding_box.tolist() == [ 80.0, 80.0, 90.0, 90.0 ]
fill_faces = refill_faces([ face_first, None, face_middle, None, face_last ])
assert fill_faces[0].bounding_box.tolist() == [ 0.0, 0.0, 10.0, 10.0 ]
assert fill_faces[1].bounding_box.tolist() == [ 20.0, 20.0, 30.0, 30.0 ]
assert fill_faces[2].bounding_box.tolist() == [ 40.0, 40.0, 50.0, 50.0 ]
assert fill_faces[3].bounding_box.tolist() == [ 60.0, 60.0, 70.0, 70.0 ]
assert fill_faces[4].bounding_box.tolist() == [ 80.0, 80.0, 90.0, 90.0 ]
def test_average_face_geometry() -> None:
source_vision_frame = read_static_image(get_test_example_file('source.jpg'))
face_previous = get_one_face(get_many_faces([ source_vision_frame ]))
face_next = get_one_face(get_many_faces([ source_vision_frame ]))
face_previous = face_previous._replace(bounding_box = numpy.array([ 0, 0, 10, 10 ]))
face_next = face_next._replace(bounding_box = numpy.array([ 80, 80, 90, 90 ]))
assert average_face_geometry([face_previous, face_next], 0.5).bounding_box.tolist() == [40.0, 40.0, 50.0, 50.0]
assert average_face_geometry([face_previous, face_next], 0.5).angle == face_next.angle
assert average_face_geometry([face_previous, face_next], 0.5).embedding is face_next.embedding
assert average_face_geometry([face_previous, face_next], 0.25).embedding is face_previous.embedding
@@ -2,10 +2,10 @@ import subprocess
import pytest
from facefusion import face_classifier, face_detector, face_landmarker, face_recognizer, state_manager
from facefusion import face_detector, state_manager
from facefusion.download import conditional_download
from facefusion.face_analyser import get_many_faces
from facefusion.face_store import clear_static_faces
from facefusion.face_detector import detect_with_retinaface, detect_with_scrfd, detect_with_yolo_face, detect_with_yunet
from facefusion.face_helper import apply_nms, get_nms_threshold
from facefusion.vision import read_static_image
from .helper import get_test_example_file, get_test_examples_directory
@@ -19,34 +19,23 @@ def before_all() -> None:
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('source.jpg'), '-vf', 'crop=iw*0.8:ih*0.8', get_test_example_file('source-80crop.jpg') ])
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('source.jpg'), '-vf', 'crop=iw*0.7:ih*0.7', get_test_example_file('source-70crop.jpg') ])
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('source.jpg'), '-vf', 'crop=iw*0.6:ih*0.6', get_test_example_file('source-60crop.jpg') ])
state_manager.init_item('execution_device_ids', [ 0 ])
state_manager.init_item('execution_providers', [ 'cpu' ])
state_manager.init_item('download_providers', [ 'github' ])
state_manager.init_item('face_detector_angles', [ 0 ])
state_manager.init_item('face_detector_model', 'many')
state_manager.init_item('face_detector_score', 0.5)
state_manager.init_item('face_landmarker_model', 'many')
state_manager.init_item('face_landmarker_score', 0.5)
face_classifier.pre_check()
face_landmarker.pre_check()
face_recognizer.pre_check()
face_detector.pre_check()
@pytest.fixture(autouse = True)
def before_each() -> None:
face_classifier.clear_inference_pool()
face_detector.clear_inference_pool()
face_landmarker.clear_inference_pool()
face_recognizer.clear_inference_pool()
clear_static_faces()
def test_get_one_face_with_retinaface() -> None:
state_manager.init_item('face_detector_model', 'retinaface')
state_manager.init_item('face_detector_size', '320x320')
state_manager.init_item('face_detector_margin', (0, 0, 0, 0))
face_detector.pre_check()
def test_detect_with_retinaface() -> None:
source_paths =\
[
get_test_example_file('source.jpg'),
@@ -57,17 +46,13 @@ def test_get_one_face_with_retinaface() -> None:
for source_path in source_paths:
source_frame = read_static_image(source_path)
many_faces = get_many_faces([ source_frame ])
bounding_boxes, face_scores, face_landmarks_5 = detect_with_retinaface(source_frame, '320x320')
keep_indices = apply_nms(bounding_boxes, face_scores, 0.5, get_nms_threshold('retinaface', [ 0 ]))
assert len(many_faces) == 1
assert len(keep_indices) == 1
def test_get_one_face_with_scrfd() -> None:
state_manager.init_item('face_detector_model', 'scrfd')
state_manager.init_item('face_detector_size', '320x320')
state_manager.init_item('face_detector_margin', (0, 0, 0, 0))
face_detector.pre_check()
def test_detect_with_scrfd() -> None:
source_paths =\
[
get_test_example_file('source.jpg'),
@@ -78,17 +63,13 @@ def test_get_one_face_with_scrfd() -> None:
for source_path in source_paths:
source_frame = read_static_image(source_path)
many_faces = get_many_faces([ source_frame ])
bounding_boxes, face_scores, face_landmarks_5 = detect_with_scrfd(source_frame, '320x320')
keep_indices = apply_nms(bounding_boxes, face_scores, 0.5, get_nms_threshold('scrfd', [ 0 ]))
assert len(many_faces) == 1
assert len(keep_indices) == 1
def test_get_one_face_with_yoloface() -> None:
state_manager.init_item('face_detector_model', 'yolo_face')
state_manager.init_item('face_detector_size', '640x640')
state_manager.init_item('face_detector_margin', (0, 0, 0, 0))
face_detector.pre_check()
def test_detect_with_yolo_face() -> None:
source_paths =\
[
get_test_example_file('source.jpg'),
@@ -99,17 +80,13 @@ def test_get_one_face_with_yoloface() -> None:
for source_path in source_paths:
source_frame = read_static_image(source_path)
many_faces = get_many_faces([ source_frame ])
bounding_boxes, face_scores, face_landmarks_5 = detect_with_yolo_face(source_frame, '640x640')
keep_indices = apply_nms(bounding_boxes, face_scores, 0.5, get_nms_threshold('yolo_face', [ 0 ]))
assert len(many_faces) == 1
assert len(keep_indices) == 1
def test_get_one_face_with_yunet() -> None:
state_manager.init_item('face_detector_model', 'yunet')
state_manager.init_item('face_detector_size', '640x640')
state_manager.init_item('face_detector_margin', (0, 0, 0, 0))
face_detector.pre_check()
def test_detect_with_yunet() -> None:
source_paths =\
[
get_test_example_file('source.jpg'),
@@ -120,14 +97,7 @@ def test_get_one_face_with_yunet() -> None:
for source_path in source_paths:
source_frame = read_static_image(source_path)
many_faces = get_many_faces([ source_frame ])
bounding_boxes, face_scores, face_landmarks_5 = detect_with_yunet(source_frame, '640x640')
keep_indices = apply_nms(bounding_boxes, face_scores, 0.5, get_nms_threshold('yunet', [ 0 ]))
assert len(many_faces) == 1
def test_get_many_faces() -> None:
source_path = get_test_example_file('source.jpg')
source_frame = read_static_image(source_path)
many_faces = get_many_faces([ source_frame, source_frame, source_frame ])
assert len(many_faces) == 3
assert len(keep_indices) == 1
+102
View File
@@ -0,0 +1,102 @@
import numpy
import pytest
from facefusion import face_classifier, face_detector, face_landmarker, face_recognizer, state_manager
from facefusion.common_helper import get_first, get_last
from facefusion.download import conditional_download
from facefusion.face_creator import get_many_faces, get_one_face
from facefusion.face_store import clear_faces
from facefusion.face_tracker import create_face_tracks, select_face_track, track_faces
from facefusion.vision import read_static_video_chunk, read_static_video_frame
from .helper import get_test_example_file, get_test_examples_directory
@pytest.fixture(scope = 'module', autouse = True)
def before_all() -> None:
conditional_download(get_test_examples_directory(),
[
'https://github.com/facefusion/facefusion-assets/releases/download/examples-3.0.0/target-240p.mp4'
])
state_manager.init_item('execution_device_ids', [ 0 ])
state_manager.init_item('execution_providers', [ 'cpu' ])
state_manager.init_item('download_providers', [ 'github' ])
state_manager.init_item('face_detector_angles', [ 0 ])
state_manager.init_item('face_detector_model', 'yolo_face')
state_manager.init_item('face_detector_size', '640x640')
state_manager.init_item('face_detector_margin', (0, 0, 0, 0))
state_manager.init_item('face_detector_score', 0.5)
state_manager.init_item('face_landmarker_model', 'many')
state_manager.init_item('face_landmarker_score', 0.5)
state_manager.init_item('face_tracker_score', 0.3)
face_classifier.pre_check()
face_detector.pre_check()
face_landmarker.pre_check()
face_recognizer.pre_check()
@pytest.fixture(autouse = True)
def before_each() -> None:
face_classifier.clear_inference_pool()
face_detector.clear_inference_pool()
face_landmarker.clear_inference_pool()
face_recognizer.clear_inference_pool()
clear_faces()
def test_track_faces() -> None:
target_path = get_test_example_file('target-240p.mp4')
video_frame_chunk = read_static_video_chunk(target_path, 0, 7)
target_vision_frames = [ video_frame_chunk.get(frame_number) for frame_number in sorted(video_frame_chunk) ]
empty_vision_frame = numpy.zeros_like(get_first(target_vision_frames))
target_vision_frames[2] = empty_vision_frame
target_vision_frames[3] = empty_vision_frame
target_vision_frames[4] = empty_vision_frame
target_vision_frames[5] = empty_vision_frame
assert len(track_faces(target_vision_frames, 0.3)) == 1
target_vision_frames = [ video_frame_chunk.get(frame_number) for frame_number in sorted(video_frame_chunk)[:5] ]
target_vision_frames[0] = empty_vision_frame
target_vision_frames[1] = empty_vision_frame
target_vision_frames[2] = empty_vision_frame
assert len(track_faces(target_vision_frames, 0.3)) == 0
def test_create_face_tracks() -> None:
target_vision_frame = read_static_video_frame(get_test_example_file('target-240p.mp4'), 0)
multi_face_vision_frame = numpy.hstack([ target_vision_frame, target_vision_frame ])
face_tracks = create_face_tracks([ target_vision_frame, target_vision_frame ], 0.3)
assert len(face_tracks) == 1
assert sorted(get_first(face_tracks)) == [ 0, 1 ]
face_tracks = create_face_tracks([ multi_face_vision_frame, multi_face_vision_frame ], 0.3)
assert len(face_tracks) == 2
assert sorted(get_first(face_tracks)) == [ 0, 1 ]
assert sorted(get_last(face_tracks)) == [ 0, 1 ]
assert len(create_face_tracks([ target_vision_frame, target_vision_frame ], 1.0)) == 2
def test_select_face_track() -> None:
target_vision_frame = read_static_video_frame(get_test_example_file('target-240p.mp4'), 0)
face = get_one_face(get_many_faces([ target_vision_frame ]))
face_overlap = face._replace(bounding_box = numpy.array([ 12, 12, 52, 52 ]))
face_distant = face._replace(bounding_box = numpy.array([ 200, 200, 240, 240 ]))
face_track_overlap =\
{
0 : face._replace(bounding_box = numpy.array([ 10, 10, 50, 50 ]))
}
face_track_distant =\
{
0 : face._replace(bounding_box = numpy.array([ 100, 100, 140, 140 ]))
}
assert select_face_track([ face_track_overlap, face_track_distant ], face_overlap, 0.3) is face_track_overlap
assert select_face_track([ face_track_overlap, face_track_distant ], face_distant, 0.3) == {}
+2 -2
View File
@@ -9,7 +9,7 @@ from facefusion import process_manager, state_manager
from facefusion.download import conditional_download
from facefusion.ffmpeg import concat_video, extract_frames, merge_video, read_audio_buffer, replace_audio, restore_audio
from facefusion.filesystem import copy_file
from facefusion.temp_helper import clear_temp_directory, create_temp_directory, get_temp_file_path, resolve_temp_frame_paths
from facefusion.temp_helper import clear_temp_directory, create_temp_directory, get_temp_file_path, resolve_temp_frame_set
from facefusion.types import EncoderSet
from .helper import get_test_example_file, get_test_examples_directory, get_test_output_file, prepare_test_output_directory
@@ -85,7 +85,7 @@ def test_extract_frames() -> None:
create_temp_directory(target_path)
assert extract_frames(target_path, (452, 240), 30.0, trim_frame_start, trim_frame_end) is True
assert len(resolve_temp_frame_paths(target_path)) == frame_total
assert len(resolve_temp_frame_set(target_path)) == frame_total
clear_temp_directory(target_path)
+19 -1
View File
@@ -1,7 +1,7 @@
from shutil import which
from facefusion import ffmpeg_builder
from facefusion.ffmpeg_builder import chain, concat, keep_video_alpha, run, select_frame_range, set_audio_quality, set_audio_sample_size, set_stream_mode, set_video_encoder, set_video_fps, set_video_quality
from facefusion.ffmpeg_builder import chain, concat, keep_video_alpha, run, select_frame_range, set_audio_quality, set_audio_sample_size, set_faststart, set_stream_mode, set_video_encoder, set_video_fps, set_video_quality, set_video_tag
def test_run() -> None:
@@ -71,6 +71,24 @@ def test_set_audio_quality() -> None:
assert set_audio_quality('flac', 100) == []
def test_set_faststart() -> None:
assert set_faststart('m4v') == [ '-movflags', '+faststart' ]
assert set_faststart('mov') == [ '-movflags', '+faststart' ]
assert set_faststart('mp4') == [ '-movflags', '+faststart' ]
assert set_faststart('mkv') == []
assert set_faststart('webm') == []
def test_set_video_tag() -> None:
assert set_video_tag('libx265', 'm4v') == [ '-tag:v', 'hvc1' ]
assert set_video_tag('hevc_nvenc', 'mov') == [ '-tag:v', 'hvc1' ]
assert set_video_tag('hevc_videotoolbox', 'mp4') == [ '-tag:v', 'hvc1' ]
assert set_video_tag('libx265', 'mkv') == []
assert set_video_tag('libx265', 'webm') == []
assert set_video_tag('libx264', 'mp4') == []
assert set_video_tag('h264_nvenc', 'mp4') == []
def test_set_video_quality() -> None:
assert set_video_quality('libx264', 0) == [ '-crf', '51' ]
assert set_video_quality('libx264', 50) == [ '-crf', '26' ]
-8
View File
@@ -1,8 +0,0 @@
from facefusion.common_helper import is_linux, is_macos
from facefusion.memory import limit_system_memory
def test_limit_system_memory() -> None:
assert limit_system_memory(4) is True
if is_linux() or is_macos():
assert limit_system_memory(1024) is False
+3 -3
View File
@@ -5,7 +5,7 @@ import pytest
from facefusion import state_manager
from facefusion.download import conditional_download
from facefusion.temp_helper import get_temp_directory_path, get_temp_file_path, get_temp_frames_pattern
from facefusion.temp_helper import get_temp_directory_path, get_temp_file_path, get_temp_frame_pattern
from .helper import get_test_example_file, get_test_examples_directory
@@ -29,6 +29,6 @@ def test_get_temp_directory_path() -> None:
assert get_temp_directory_path(get_test_example_file('target-240p.mp4')) == os.path.join(temp_directory, 'facefusion', 'target-240p')
def test_get_temp_frames_pattern() -> None:
def test_get_temp_frame_pattern() -> None:
temp_directory = tempfile.gettempdir()
assert get_temp_frames_pattern(get_test_example_file('target-240p.mp4'), '%04d') == os.path.join(temp_directory, 'facefusion', 'target-240p', '%04d.png')
assert get_temp_frame_pattern(get_test_example_file('target-240p.mp4'), '%04d') == os.path.join(temp_directory, 'facefusion', 'target-240p', '%04d.png')
+20 -2
View File
@@ -1,11 +1,12 @@
import os
import subprocess
import numpy
import pytest
from facefusion.common_helper import is_linux
from facefusion.download import conditional_download
from facefusion.vision import calculate_histogram_difference, count_trim_frame_total, count_video_frame_total, detect_image_resolution, detect_video_duration, detect_video_fps, detect_video_resolution, match_frame_color, normalize_resolution, pack_resolution, predict_video_frame_total, read_image, read_video_frame, restrict_image_resolution, restrict_trim_frame, restrict_video_fps, restrict_video_resolution, scale_resolution, unpack_resolution, write_image
from facefusion.vision import calculate_histogram_difference, count_trim_frame_total, count_video_frame_total, detect_image_resolution, detect_video_duration, detect_video_fps, detect_video_resolution, match_frame_color, normalize_resolution, pack_resolution, predict_video_frame_total, read_image, read_video_chunk, read_video_frame, restrict_image_resolution, restrict_trim_frame, restrict_video_fps, restrict_video_resolution, scale_resolution, select_video_frames, unpack_resolution, write_image
from .helper import get_test_example_file, get_test_examples_directory, get_test_output_file, prepare_test_output_directory
@@ -63,10 +64,27 @@ def test_restrict_image_resolution() -> None:
def test_read_video_frame() -> None:
assert hasattr(read_video_frame(get_test_example_file('target-240p-25fps.mp4')), '__array_interface__')
target_path = get_test_example_file('target-240p-25fps.mp4')
assert read_video_frame(target_path).shape == (226, 426, 3)
assert numpy.array_equal(read_video_frame(target_path, 49), select_video_frames(target_path, 49, 5)[5])
assert numpy.array_equal(read_video_frame(target_path, 50), select_video_frames(target_path, 50, 5)[5])
assert numpy.array_equal(read_video_frame(target_path, 51), select_video_frames(target_path, 51, 5)[5])
assert read_video_frame('invalid') is None
def test_read_video_chunk() -> None:
assert len(read_video_chunk(get_test_example_file('target-240p-25fps.mp4'), 1, 40)) == 40
assert read_video_chunk('invalid', 1, 40) == {}
def test_select_video_frames() -> None:
assert len(select_video_frames(get_test_example_file('target-240p-25fps.mp4'), 50, 5)) == 11
assert len(select_video_frames(get_test_example_file('target-240p-25fps.mp4'), 1, 5)) == 11
assert len(select_video_frames(get_test_example_file('target-240p-25fps.mp4'), 269, 5)) == 11
assert select_video_frames('invalid', 50, 5) == []
def test_count_video_frame_total() -> None:
assert count_video_frame_total(get_test_example_file('target-240p-25fps.mp4')) == 270
assert count_video_frame_total(get_test_example_file('target-240p-30fps.mp4')) == 324