* Rename calcXXX to calculateXXX

* Add migraphx support

* Add migraphx support

* Add migraphx support

* Add migraphx support

* Add migraphx support

* Add migraphx support

* Use True for the flags

* Add migraphx support

* add face-swapper-weight

* add face-swapper-weight to facefusion.ini

* changes

* change choice

* Fix typing for xxxWeight

* Feat/log inference session (#906)

* Log inference session, Introduce time helper

* Log inference session, Introduce time helper

* Log inference session, Introduce time helper

* Log inference session, Introduce time helper

* Mark as NEXT

* Follow industry standard x1, x2, y1 and y2

* Follow industry standard x1, x2, y1 and y2

* Follow industry standard in terms of naming (#908)

* Follow industry standard in terms of naming

* Improve xxx_embedding naming

* Fix norm vs. norms

* Reduce timeout to 5

* Sort out voice_extractor once again

* changes

* Introduce many to the occlusion mask (#910)

* Introduce many to the occlusion mask

* Then we use minimum

* Add support for wmv

* Run platform tests before has_execution_provider (#911)

* Add support for wmv

* Introduce benchmark mode (#912)

* Honestly makes no difference to me

* Honestly makes no difference to me

* Fix wording

* Bring back YuNet (#922)

* Reintroduce YuNet without cv2 dependency

* Fix variable naming

* Avoid RGB to YUV colorshift using libx264rgb

* Avoid RGB to YUV colorshift using libx264rgb

* Make libx264 the default again

* Make libx264 the default again

* Fix types in ffmpeg builder

* Fix quality stuff in ffmpeg builder

* Fix quality stuff in ffmpeg builder

* Add libx264rgb to test

* Revamp Processors (#923)

* Introduce new concept of pure target frames

* Radical refactoring of process flow

* Introduce new concept of pure target frames

* Fix webcam

* Minor improvements

* Minor improvements

* Use deque for video processing

* Use deque for video processing

* Extend the video manager

* Polish deque

* Polish deque

* Deque is not even used

* Improve speed with multiple futures

* Fix temp frame mutation and

* Fix RAM usage

* Remove old types and manage method

* Remove execution_queue_count

* Use init_state for benchmarker to avoid issues

* add voice extractor option

* Change the order of voice extractor in code

* Use official download urls

* Use official download urls

* add gui

* fix preview

* Add remote updates for voice extractor

* fix crash on headless-run

* update test_job_helper.py

* Fix it for good

* Remove pointless method

* Fix types and unused imports

* Revamp reference (#925)

* Initial revamp of face references

* Initial revamp of face references

* Initial revamp of face references

* Terminate find_similar_faces

* Improve find mutant faces

* Improve find mutant faces

* Move sort where it belongs

* Forward reference vision frame

* Forward reference vision frame also in preview

* Fix reference selection

* Use static video frame

* Fix CI

* Remove reference type from frame processors

* Improve some naming

* Fix types and unused imports

* Fix find mutant faces

* Fix find mutant faces

* Fix imports

* Correct naming

* Correct naming

* simplify pad

* Improve webcam performance on highres

* Camera manager (#932)

* Introduce webcam manager

* Fix order

* Rename to camera manager, improve video manager

* Fix CI

* Remove optional

* Fix naming in webcam options

* Avoid using temp faces (#933)

* output video scale

* Fix imports

* output image scale

* upscale fix (not limiter)

* add unit test scale_resolution & remove unused methods

* fix and add test

* fix

* change pack_resolution

* fix tests

* Simplify output scale testing

* Fix benchmark UI

* Fix benchmark UI

* Update dependencies

* Introduce REAL multi gpu support using multi dimensional inference pool (#935)

* Introduce REAL multi gpu support using multi dimensional inference pool

* Remove the MULTI:GPU flag

* Restore "processing stop"

* Restore "processing stop"

* Remove old templates

* Go fill in with caching

* add expression restorer areas

* re-arrange

* rename method

* Fix stop for extract frames and merge video

* Replace arcface_converter models with latest crossface models

* Replace arcface_converter models with latest crossface models

* Move module logs to debug mode

* Refactor/streamer (#938)

* Introduce webcam manager

* Fix order

* Rename to camera manager, improve video manager

* Fix CI

* Fix naming in webcam options

* Move logic over to streamer

* Fix streamer, improve webcam experience

* Improve webcam experience

* Revert method

* Revert method

* Improve webcam again

* Use release on capture instead

* Only forward valid frames

* Fix resolution logging

* Add AVIF support

* Add AVIF support

* Limit avif to unix systems

* Drop avif

* Drop avif

* Drop avif

* Default to Documents in the UI if output path is not set

* Update wording.py (#939)

"succeed" is grammatically incorrect in the given context. To succeed is the infinitive form of the verb. Correct would be either "succeeded" or alternatively a form involving the noun "success".

* Fix more grammar issue

* Fix more grammar issue

* Sort out caching

* Move webcam choices back to UI

* Move preview options to own file (#940)

* Fix Migraphx execution provider

* Fix benchmark

* Reuse blend frame method

* Fix CI

* Fix CI

* Fix CI

* Hotfix missing check in face debugger, Enable logger for preview

* Fix reference selection (#942)

* Fix reference selection

* Fix reference selection

* Fix reference selection

* Fix reference selection

* Side by side preview (#941)

* Initial side by side preview

* More work on preview, remove UI only stuff from vision.py

* Improve more

* Use fit frame

* Add different fit methods for vision

* Improve preview part2

* Improve preview part3

* Improve preview part4

* Remove none as choice

* Remove useless methods

* Fix CI

* Fix naming

* use 1024 as preview resolution default

* Fix fit_cover_frame

* Uniform fit_xxx_frame methods

* Add back disabled logger

* Use ui choices alias

* Extract select face logic from processors (#943)

* Extract select face logic from processors to use it for face by face in preview

* Fix order

* Remove old code

* Merge methods

* Refactor face debugger (#944)

* Refactor huge method of face debugger

* Remove text metrics from face debugger

* Remove useless copy of temp frame

* Resort methods

* Fix spacing

* Remove old method

* Fix hard exit to work without signals

* Prevent upscaling for face-by-face

* Switch to version

* Improve exiting

---------

Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com>
Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>
Co-authored-by: Rafael Tappe Maestro <rafael@tappemaestro.com>
This commit is contained in:
Henry Ruhs
2025-09-08 10:43:58 +02:00
committed by GitHub
parent 7b8bea4e0a
commit da0da3a4b4
97 changed files with 2113 additions and 1934 deletions
+106 -65
View File
@@ -3,19 +3,19 @@ import itertools
import shutil
import signal
import sys
from concurrent.futures import ThreadPoolExecutor, as_completed
from time import time
import numpy
from tqdm import tqdm
from facefusion import benchmarker, cli_helper, content_analyser, face_classifier, face_detector, face_landmarker, face_masker, face_recognizer, hash_helper, logger, process_manager, state_manager, video_manager, voice_extractor, wording
from facefusion.args import apply_args, collect_job_args, reduce_job_args, reduce_step_args
from facefusion.audio import create_empty_audio_frame, get_audio_frame, get_voice_frame
from facefusion.common_helper import get_first
from facefusion.content_analyser import analyse_image, analyse_video
from facefusion.download import conditional_download_hashes, conditional_download_sources
from facefusion.exit_helper import hard_exit, signal_exit
from facefusion.face_analyser import get_average_face, get_many_faces, get_one_face
from facefusion.face_selector import sort_and_filter_faces
from facefusion.face_store import append_reference_face, clear_reference_faces, get_reference_faces
from facefusion.ffmpeg import copy_image, extract_frames, finalize_image, merge_video, replace_audio, restore_audio
from facefusion.filesystem import filter_audio_paths, get_file_name, is_image, is_video, resolve_file_paths, resolve_file_pattern
from facefusion.jobs import job_helper, job_manager, job_runner
@@ -25,8 +25,9 @@ from facefusion.processors.core import get_processors_modules
from facefusion.program import create_program
from facefusion.program_helper import validate_args
from facefusion.temp_helper import clear_temp_directory, create_temp_directory, get_temp_file_path, move_temp_file, resolve_temp_frame_paths
from facefusion.time_helper import calculate_end_time
from facefusion.types import Args, ErrorCode
from facefusion.vision import pack_resolution, read_image, read_static_images, read_video_frame, restrict_image_resolution, restrict_trim_frame, restrict_video_fps, restrict_video_resolution, unpack_resolution
from facefusion.vision import detect_image_resolution, detect_video_resolution, pack_resolution, read_static_image, read_static_images, read_static_video_frame, restrict_image_resolution, restrict_trim_frame, restrict_video_fps, restrict_video_resolution, scale_resolution, write_image
def cli() -> None:
@@ -57,11 +58,11 @@ def route(args : Args) -> None:
if state_manager.get_item('command') == 'force-download':
error_code = force_download()
return hard_exit(error_code)
hard_exit(error_code)
if state_manager.get_item('command') == 'benchmark':
if not common_pre_check() or not processors_pre_check() or not benchmarker.pre_check():
return hard_exit(2)
hard_exit(2)
benchmarker.render()
if state_manager.get_item('command') in [ 'job-list', 'job-create', 'job-submit', 'job-submit-all', 'job-delete', 'job-delete-all', 'job-add-step', 'job-remix-step', 'job-insert-step', 'job-remove-step' ]:
@@ -74,10 +75,10 @@ def route(args : Args) -> None:
import facefusion.uis.core as ui
if not common_pre_check() or not processors_pre_check():
return hard_exit(2)
hard_exit(2)
for ui_layout in ui.get_ui_layouts_modules(state_manager.get_item('ui_layouts')):
if not ui_layout.pre_check():
return hard_exit(2)
hard_exit(2)
ui.init()
ui.launch()
@@ -128,9 +129,9 @@ def common_pre_check() -> bool:
]
content_analyser_content = inspect.getsource(content_analyser).encode()
is_valid = hash_helper.create_hash(content_analyser_content) == 'b159fd9d'
content_analyser_hash = hash_helper.create_hash(content_analyser_content)
return all(module.pre_check() for module in common_modules) and is_valid
return all(module.pre_check() for module in common_modules) and content_analyser_hash == '803b5ec7'
def processors_pre_check() -> bool:
@@ -251,7 +252,7 @@ def route_job_runner() -> ErrorCode:
if state_manager.get_item('command') == 'job-run':
logger.info(wording.get('running_job').format(job_id = state_manager.get_item('job_id')), __name__)
if job_runner.run_job(state_manager.get_item('job_id'), process_step):
logger.info(wording.get('processing_job_succeed').format(job_id = state_manager.get_item('job_id')), __name__)
logger.info(wording.get('processing_job_succeeded').format(job_id = state_manager.get_item('job_id')), __name__)
return 0
logger.info(wording.get('processing_job_failed').format(job_id = state_manager.get_item('job_id')), __name__)
return 1
@@ -259,7 +260,7 @@ def route_job_runner() -> ErrorCode:
if state_manager.get_item('command') == 'job-run-all':
logger.info(wording.get('running_jobs'), __name__)
if job_runner.run_jobs(process_step, state_manager.get_item('halt_on_error')):
logger.info(wording.get('processing_jobs_succeed'), __name__)
logger.info(wording.get('processing_jobs_succeeded'), __name__)
return 0
logger.info(wording.get('processing_jobs_failed'), __name__)
return 1
@@ -267,7 +268,7 @@ def route_job_runner() -> ErrorCode:
if state_manager.get_item('command') == 'job-retry':
logger.info(wording.get('retrying_job').format(job_id = state_manager.get_item('job_id')), __name__)
if job_runner.retry_job(state_manager.get_item('job_id'), process_step):
logger.info(wording.get('processing_job_succeed').format(job_id = state_manager.get_item('job_id')), __name__)
logger.info(wording.get('processing_job_succeeded').format(job_id = state_manager.get_item('job_id')), __name__)
return 0
logger.info(wording.get('processing_job_failed').format(job_id = state_manager.get_item('job_id')), __name__)
return 1
@@ -275,7 +276,7 @@ def route_job_runner() -> ErrorCode:
if state_manager.get_item('command') == 'job-retry-all':
logger.info(wording.get('retrying_jobs'), __name__)
if job_runner.retry_jobs(process_step, state_manager.get_item('halt_on_error')):
logger.info(wording.get('processing_jobs_succeed'), __name__)
logger.info(wording.get('processing_jobs_succeeded'), __name__)
return 0
logger.info(wording.get('processing_jobs_failed'), __name__)
return 1
@@ -321,7 +322,6 @@ def process_batch(args : Args) -> ErrorCode:
def process_step(job_id : str, step_index : int, step_args : Args) -> bool:
clear_reference_faces()
step_total = job_manager.count_step_total(job_id)
step_args.update(collect_job_args())
apply_args(step_args, state_manager.set_item)
@@ -340,8 +340,6 @@ def conditional_process() -> ErrorCode:
if not processor_module.pre_process('output'):
return 2
conditional_append_reference_faces()
if is_image(state_manager.get_item('target_path')):
return process_image(start_time)
if is_video(state_manager.get_item('target_path')):
@@ -350,28 +348,6 @@ def conditional_process() -> ErrorCode:
return 0
def conditional_append_reference_faces() -> None:
if 'reference' in state_manager.get_item('face_selector_mode') and not get_reference_faces():
source_frames = read_static_images(state_manager.get_item('source_paths'))
source_faces = get_many_faces(source_frames)
source_face = get_average_face(source_faces)
if is_video(state_manager.get_item('target_path')):
reference_frame = read_video_frame(state_manager.get_item('target_path'), state_manager.get_item('reference_frame_number'))
else:
reference_frame = read_image(state_manager.get_item('target_path'))
reference_faces = sort_and_filter_faces(get_many_faces([ reference_frame ]))
reference_face = get_one_face(reference_faces, state_manager.get_item('reference_face_position'))
append_reference_face('origin', reference_face)
if source_face and reference_face:
for processor_module in get_processors_modules(state_manager.get_item('processors')):
abstract_reference_frame = processor_module.get_reference_frame(source_face, reference_face, reference_frame)
if numpy.any(abstract_reference_frame):
abstract_reference_faces = sort_and_filter_faces(get_many_faces([ abstract_reference_frame ]))
abstract_reference_face = get_one_face(abstract_reference_faces, state_manager.get_item('reference_face_position'))
append_reference_face(processor_module.__name__, abstract_reference_face)
def process_image(start_time : float) -> ErrorCode:
if analyse_image(state_manager.get_item('target_path')):
return 3
@@ -382,27 +358,47 @@ def process_image(start_time : float) -> ErrorCode:
create_temp_directory(state_manager.get_item('target_path'))
process_manager.start()
temp_image_resolution = pack_resolution(restrict_image_resolution(state_manager.get_item('target_path'), unpack_resolution(state_manager.get_item('output_image_resolution'))))
logger.info(wording.get('copying_image').format(resolution = temp_image_resolution), __name__)
output_image_resolution = scale_resolution(detect_image_resolution(state_manager.get_item('target_path')), state_manager.get_item('output_image_scale'))
temp_image_resolution = restrict_image_resolution(state_manager.get_item('target_path'), output_image_resolution)
logger.info(wording.get('copying_image').format(resolution = pack_resolution(temp_image_resolution)), __name__)
if copy_image(state_manager.get_item('target_path'), temp_image_resolution):
logger.debug(wording.get('copying_image_succeed'), __name__)
logger.debug(wording.get('copying_image_succeeded'), __name__)
else:
logger.error(wording.get('copying_image_failed'), __name__)
process_manager.end()
return 1
temp_image_path = get_temp_file_path(state_manager.get_item('target_path'))
reference_vision_frame = read_static_image(temp_image_path)
source_vision_frames = read_static_images(state_manager.get_item('source_paths'))
source_audio_frame = create_empty_audio_frame()
source_voice_frame = create_empty_audio_frame()
target_vision_frame = read_static_image(temp_image_path)
temp_vision_frame = target_vision_frame.copy()
for processor_module in get_processors_modules(state_manager.get_item('processors')):
logger.info(wording.get('processing'), processor_module.__name__)
processor_module.process_image(state_manager.get_item('source_paths'), temp_image_path, temp_image_path)
temp_vision_frame = processor_module.process_frame(
{
'reference_vision_frame': reference_vision_frame,
'source_vision_frames': source_vision_frames,
'source_audio_frame': source_audio_frame,
'source_voice_frame': source_voice_frame,
'target_vision_frame': target_vision_frame,
'temp_vision_frame': temp_vision_frame
})
processor_module.post_process()
write_image(temp_image_path, temp_vision_frame)
if is_process_stopping():
process_manager.end()
return 4
logger.info(wording.get('finalizing_image').format(resolution = state_manager.get_item('output_image_resolution')), __name__)
if finalize_image(state_manager.get_item('target_path'), state_manager.get_item('output_path'), state_manager.get_item('output_image_resolution')):
logger.debug(wording.get('finalizing_image_succeed'), __name__)
logger.info(wording.get('finalizing_image').format(resolution = pack_resolution(output_image_resolution)), __name__)
if finalize_image(state_manager.get_item('target_path'), state_manager.get_item('output_path'), output_image_resolution):
logger.debug(wording.get('finalizing_image_succeeded'), __name__)
else:
logger.warn(wording.get('finalizing_image_skipped'), __name__)
@@ -410,8 +406,7 @@ def process_image(start_time : float) -> ErrorCode:
clear_temp_directory(state_manager.get_item('target_path'))
if is_image(state_manager.get_item('output_path')):
seconds = '{:.2f}'.format((time() - start_time) % 60)
logger.info(wording.get('processing_image_succeed').format(seconds = seconds), __name__)
logger.info(wording.get('processing_image_succeeded').format(seconds = calculate_end_time(start_time)), __name__)
else:
logger.error(wording.get('processing_image_failed'), __name__)
process_manager.end()
@@ -431,25 +426,45 @@ def process_video(start_time : float) -> ErrorCode:
create_temp_directory(state_manager.get_item('target_path'))
process_manager.start()
temp_video_resolution = pack_resolution(restrict_video_resolution(state_manager.get_item('target_path'), unpack_resolution(state_manager.get_item('output_video_resolution'))))
output_video_resolution = scale_resolution(detect_video_resolution(state_manager.get_item('target_path')), state_manager.get_item('output_video_scale'))
temp_video_resolution = restrict_video_resolution(state_manager.get_item('target_path'), output_video_resolution)
temp_video_fps = restrict_video_fps(state_manager.get_item('target_path'), state_manager.get_item('output_video_fps'))
logger.info(wording.get('extracting_frames').format(resolution = temp_video_resolution, fps = temp_video_fps), __name__)
logger.info(wording.get('extracting_frames').format(resolution = pack_resolution(temp_video_resolution), fps = temp_video_fps), __name__)
if extract_frames(state_manager.get_item('target_path'), temp_video_resolution, temp_video_fps, trim_frame_start, trim_frame_end):
logger.debug(wording.get('extracting_frames_succeed'), __name__)
logger.debug(wording.get('extracting_frames_succeeded'), __name__)
else:
if is_process_stopping():
process_manager.end()
return 4
logger.error(wording.get('extracting_frames_failed'), __name__)
process_manager.end()
return 1
temp_frame_paths = resolve_temp_frame_paths(state_manager.get_item('target_path'))
if temp_frame_paths:
with tqdm(total = len(temp_frame_paths), desc = wording.get('processing'), unit = 'frame', ascii = ' =', disable = state_manager.get_item('log_level') in [ 'warn', 'error' ]) as progress:
progress.set_postfix(execution_providers = state_manager.get_item('execution_providers'))
with ThreadPoolExecutor(max_workers = state_manager.get_item('execution_thread_count')) as executor:
futures = []
for frame_number, temp_frame_path in enumerate(temp_frame_paths):
future = executor.submit(process_temp_frame, temp_frame_path, frame_number)
futures.append(future)
for future in as_completed(futures):
if is_process_stopping():
for __future__ in futures:
__future__.cancel()
if not future.cancelled():
future.result()
progress.update()
for processor_module in get_processors_modules(state_manager.get_item('processors')):
logger.info(wording.get('processing'), processor_module.__name__)
processor_module.process_video(state_manager.get_item('source_paths'), temp_frame_paths)
processor_module.post_process()
if is_process_stopping():
return 4
else:
@@ -457,12 +472,11 @@ def process_video(start_time : float) -> ErrorCode:
process_manager.end()
return 1
logger.info(wording.get('merging_video').format(resolution = state_manager.get_item('output_video_resolution'), fps = state_manager.get_item('output_video_fps')), __name__)
if merge_video(state_manager.get_item('target_path'), temp_video_fps, state_manager.get_item('output_video_resolution'), state_manager.get_item('output_video_fps'), trim_frame_start, trim_frame_end):
logger.debug(wording.get('merging_video_succeed'), __name__)
logger.info(wording.get('merging_video').format(resolution = pack_resolution(output_video_resolution), fps = state_manager.get_item('output_video_fps')), __name__)
if merge_video(state_manager.get_item('target_path'), temp_video_fps, output_video_resolution, state_manager.get_item('output_video_fps'), trim_frame_start, trim_frame_end):
logger.debug(wording.get('merging_video_succeeded'), __name__)
else:
if is_process_stopping():
process_manager.end()
return 4
logger.error(wording.get('merging_video_failed'), __name__)
process_manager.end()
@@ -476,22 +490,20 @@ def process_video(start_time : float) -> ErrorCode:
if source_audio_path:
if replace_audio(state_manager.get_item('target_path'), source_audio_path, state_manager.get_item('output_path')):
video_manager.clear_video_pool()
logger.debug(wording.get('replacing_audio_succeed'), __name__)
logger.debug(wording.get('replacing_audio_succeeded'), __name__)
else:
video_manager.clear_video_pool()
if is_process_stopping():
process_manager.end()
return 4
logger.warn(wording.get('replacing_audio_skipped'), __name__)
move_temp_file(state_manager.get_item('target_path'), state_manager.get_item('output_path'))
else:
if restore_audio(state_manager.get_item('target_path'), state_manager.get_item('output_path'), trim_frame_start, trim_frame_end):
video_manager.clear_video_pool()
logger.debug(wording.get('restoring_audio_succeed'), __name__)
logger.debug(wording.get('restoring_audio_succeeded'), __name__)
else:
video_manager.clear_video_pool()
if is_process_stopping():
process_manager.end()
return 4
logger.warn(wording.get('restoring_audio_skipped'), __name__)
move_temp_file(state_manager.get_item('target_path'), state_manager.get_item('output_path'))
@@ -500,8 +512,7 @@ def process_video(start_time : float) -> ErrorCode:
clear_temp_directory(state_manager.get_item('target_path'))
if is_video(state_manager.get_item('output_path')):
seconds = '{:.2f}'.format((time() - start_time))
logger.info(wording.get('processing_video_succeed').format(seconds = seconds), __name__)
logger.info(wording.get('processing_video_succeeded').format(seconds = calculate_end_time(start_time)), __name__)
else:
logger.error(wording.get('processing_video_failed'), __name__)
process_manager.end()
@@ -510,6 +521,36 @@ def process_video(start_time : float) -> ErrorCode:
return 0
def process_temp_frame(temp_frame_path : str, frame_number : int) -> bool:
reference_vision_frame = read_static_video_frame(state_manager.get_item('target_path'), state_manager.get_item('reference_frame_number'))
source_vision_frames = read_static_images(state_manager.get_item('source_paths'))
source_audio_path = get_first(filter_audio_paths(state_manager.get_item('source_paths')))
temp_video_fps = restrict_video_fps(state_manager.get_item('target_path'), state_manager.get_item('output_video_fps'))
target_vision_frame = read_static_image(temp_frame_path)
temp_vision_frame = target_vision_frame.copy()
source_audio_frame = get_audio_frame(source_audio_path, temp_video_fps, frame_number)
source_voice_frame = get_voice_frame(source_audio_path, temp_video_fps, frame_number)
if not numpy.any(source_audio_frame):
source_audio_frame = create_empty_audio_frame()
if not numpy.any(source_voice_frame):
source_voice_frame = create_empty_audio_frame()
for processor_module in get_processors_modules(state_manager.get_item('processors')):
temp_vision_frame = processor_module.process_frame(
{
'reference_vision_frame': reference_vision_frame,
'source_vision_frames': source_vision_frames,
'source_audio_frame': source_audio_frame,
'source_voice_frame': source_voice_frame,
'target_vision_frame': target_vision_frame,
'temp_vision_frame': temp_vision_frame
})
return write_image(temp_frame_path, temp_vision_frame)
def is_process_stopping() -> bool:
if process_manager.is_stopping():
process_manager.end()