mirror of
https://github.com/facefusion/facefusion.git
synced 2026-07-31 14:27:24 +02:00
3.8.0 (#1212)
* mark as next * unify the dependency checks in pre_check and add ffprobe (#1181) * drop keep_temp and the common options component (#1180) Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * introduce ffprobe and ffprobe_builder (#1182) * introduce ffprobe and ffprobe_builder Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * introduce ffprobe and ffprobe_builder Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * probe video metadata via ffprobe in vision (#1184) * probe video metadata via ffprobe in vision Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * probe video metadata via ffprobe in vision Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * adopt the workflow task vocabulary from next major (#1185) Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * introduce workflow-mode and workflow-strategy like next major (#1187) Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * restrict hdr color transfer and tag the merge output as bt709 (#1188) * restrict hdr color transfer and tag the merge output as bt709 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * restrict hdr color transfer and tag the merge output as bt709 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * full video migration * compose the hdr fixture via the builder chain Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * compose the test fixtures via the builder and run_ffmpeg Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * compose every test fixture via the builder and run_ffmpeg (#1189) * compose every test fixture via the builder and run_ffmpeg Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * use loops in tests for ffmpeg stuff --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * New Video Manager (#1191) * tiny adjustment for tests * address the review on the video manager Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * introduce the stream strategy for the video workflow (#1192) * introduce the stream strategy for the video workflow Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * address the review on the stream strategy Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * annotate the changes for review Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * annotate the new tests for review Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * match the temp pixel format help to the locale style Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * question the set_input_seek naming Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * question the reader and writer keys Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * drop the review annotations from the encoder mapping tests Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * drop the review annotations from the thread count tests Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * drop the review annotations from the ui files Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * capture the open review questions as annotations Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a * drop the settled annotations from the types Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tbcd6VWCiU4BQP1gywPr2a --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * switch to ffmpeg.style for audio.py * remove todos that were never needed * fix for ffmpeg7 * Add frame_store module (#1194) * add frame_store module * rename and change tests * rename and update tests * route window read through frame_store (#1196) * route window read through frame_store * update proper id * restore todos * restore todos * go v4 style for workflow (#1197) * go v4 style for workflow * remove some todos * route chunk read through frame_store (#1198) * vision integration * Deleted read_video_chunk + read_static_video_chunk * margin decouple (#1199) * fix windows CI fail (#1200) * Cleanup Part1 (#1201) * remove some todos, improve video manager, simplify ffmpeg commands and more * do more * remove thread count for filters * Cleanup Part 2 (#1202) * tons of renaming * tons of renaming * multi reader approach * bring tests to an okay-ish state * bring drain back * improve read_video_frame speed * rename method * move variables * seek video reader only when trim frame start is larger 0 * make stream the default * Cleanup/part 3 (#1203) * remove todo * sort out workflow, to match upcoming v4 * remove look ahead * remove core namespace again * Revamp execution provider overrides/adjustments (#1206) * Split provider hooks into override/adjust with cached CoreML base Replace the single resolve_inference_providers processor hook with two: override_inference_providers (full replacement) and adjust_inference_providers (merge options onto the base providers built by create_inference_providers). This lets CoreML processors inherit ModelCacheDirectory + SpecializationStrategy from the base while layering ModelFormat/MLComputeUnits on top. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HTQCZiYjJyUX11bDpbRSiB * fix caching for execution provider by having override and adjust ways * fix caching for execution provider by having override and adjust ways * fix lint * use proper pytest fixtures --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix update preview bug (#1205) * fix update preview bug * fix update preview bug * remove guard * add is_vision_frame * Restrict the preview frame slider and the reader seek to the last frame index (#1207) * fix index bug * fix rounding bug * avoid tobytes copy (#1208) * beautify tests * hide ffmpeg warnings * simplify process_stream_frame * Use is vision frame everywhere (#1210) * use is_vision_frame everywhere * fix hash * fix lint * fix hash creation in face store * that model does not exist * update workflow ffmpeg * guard workflow (#1211) * bump version and dependencies * Update preview * switch workflow strategy to disk|memory * update preview * update preview * fix wording * last minute change workflow position * adjust wording --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com> Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com>
This commit is contained in:
Henry Ruhs
co-authored by
Claude Opus 4.8
Harisreedhar
harisreedhar
Claude Opus 4.8
Harisreedhar
harisreedhar
parent
3f81a8a784
commit
b60ea40d26
+106
-31
@@ -1,16 +1,17 @@
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
import pytest
|
||||
|
||||
import facefusion.ffmpeg
|
||||
from facefusion import process_manager, state_manager
|
||||
from facefusion import ffmpeg, ffmpeg_builder, process_manager, state_manager
|
||||
from facefusion.download import conditional_download
|
||||
from facefusion.ffmpeg import concat_video, extract_frames, merge_video, read_audio_buffer, replace_audio, restore_audio
|
||||
from facefusion.ffmpeg import concat_video, extract_frames, fix_audio_encoder, fix_video_encoder, merge_video, read_audio_buffer, replace_audio, restore_audio
|
||||
from facefusion.ffprobe import extract_video_metadata
|
||||
from facefusion.filesystem import copy_file
|
||||
from facefusion.temp_helper import clear_temp_directory, create_temp_directory, get_temp_file_path, resolve_temp_frame_set
|
||||
from facefusion.types import EncoderSet
|
||||
from facefusion.vision import read_image
|
||||
from .helper import get_test_example_file, get_test_examples_directory, get_test_output_file, prepare_test_output_directory
|
||||
|
||||
|
||||
@@ -23,15 +24,53 @@ def before_all() -> None:
|
||||
'https://github.com/facefusion/facefusion-assets/releases/download/examples-3.0.0/source.mp3',
|
||||
'https://github.com/facefusion/facefusion-assets/releases/download/examples-3.0.0/target-240p.mp4'
|
||||
])
|
||||
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('source.mp3'), get_test_example_file('source.wav') ])
|
||||
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('target-240p.mp4'), '-vf', 'fps=25', get_test_example_file('target-240p-25fps.mp4') ])
|
||||
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('target-240p.mp4'), '-vf', 'fps=30', get_test_example_file('target-240p-30fps.mp4') ])
|
||||
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('target-240p.mp4'), '-vf', 'fps=60', get_test_example_file('target-240p-60fps.mp4') ])
|
||||
|
||||
ffmpeg.run_ffmpeg(
|
||||
ffmpeg_builder.chain(
|
||||
ffmpeg_builder.set_input(get_test_example_file('source.mp3')),
|
||||
ffmpeg_builder.set_output(get_test_example_file('source.wav'))
|
||||
)
|
||||
)
|
||||
|
||||
for video_fps in [ 25, 30, 60 ]:
|
||||
ffmpeg.run_ffmpeg(
|
||||
ffmpeg_builder.chain(
|
||||
ffmpeg_builder.set_input(get_test_example_file('target-240p.mp4')),
|
||||
ffmpeg_builder.set_video_fps(video_fps),
|
||||
ffmpeg_builder.set_output(get_test_example_file('target-240p-' + str(video_fps) + 'fps.mp4'))
|
||||
)
|
||||
)
|
||||
|
||||
ffmpeg.run_ffmpeg(
|
||||
ffmpeg_builder.chain(
|
||||
ffmpeg_builder.set_input(get_test_example_file('target-240p.mp4')),
|
||||
[
|
||||
'-vf',
|
||||
'scale=out_transfer=smpte2084'
|
||||
],
|
||||
ffmpeg_builder.set_output(get_test_example_file('target-240p-smpte2084.mp4'))
|
||||
)
|
||||
)
|
||||
|
||||
for output_video_format in [ 'avi', 'm4v', 'mkv', 'mov', 'mp4', 'webm', 'wmv' ]:
|
||||
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('source.mp3'), '-i', get_test_example_file('target-240p.mp4'), '-ar', '16000', get_test_example_file('target-240p-16khz.' + output_video_format) ])
|
||||
ffmpeg.run_ffmpeg(
|
||||
ffmpeg_builder.chain(
|
||||
ffmpeg_builder.set_input(get_test_example_file('source.mp3')),
|
||||
ffmpeg_builder.set_input(get_test_example_file('target-240p.mp4')),
|
||||
ffmpeg_builder.set_audio_sample_rate(16000),
|
||||
ffmpeg_builder.set_output(get_test_example_file('target-240p-16khz.' + output_video_format))
|
||||
)
|
||||
)
|
||||
|
||||
ffmpeg.run_ffmpeg(
|
||||
ffmpeg_builder.chain(
|
||||
ffmpeg_builder.set_input(get_test_example_file('source.mp3')),
|
||||
ffmpeg_builder.set_input(get_test_example_file('target-240p.mp4')),
|
||||
ffmpeg_builder.set_audio_sample_rate(48000),
|
||||
ffmpeg_builder.set_output(get_test_example_file('target-240p-48khz.mp4'))
|
||||
)
|
||||
)
|
||||
|
||||
subprocess.run([ 'ffmpeg', '-i', get_test_example_file('source.mp3'), '-i', get_test_example_file('target-240p.mp4'), '-ar', '48000', get_test_example_file('target-240p-48khz.mp4') ])
|
||||
state_manager.init_item('temp_path', tempfile.gettempdir())
|
||||
state_manager.init_item('temp_frame_format', 'png')
|
||||
state_manager.init_item('output_audio_encoder', 'aac')
|
||||
@@ -67,43 +106,49 @@ def test_get_available_encoder_set() -> None:
|
||||
def test_extract_frames() -> None:
|
||||
test_set =\
|
||||
[
|
||||
(get_test_example_file('target-240p-25fps.mp4'), 0, 270, 324),
|
||||
(get_test_example_file('target-240p-25fps.mp4'), 224, 270, 55),
|
||||
(get_test_example_file('target-240p-25fps.mp4'), 124, 224, 120),
|
||||
(get_test_example_file('target-240p-25fps.mp4'), 0, 100, 120),
|
||||
(get_test_example_file('target-240p-30fps.mp4'), 0, 324, 324),
|
||||
(get_test_example_file('target-240p-30fps.mp4'), 224, 324, 100),
|
||||
(get_test_example_file('target-240p-30fps.mp4'), 124, 224, 100),
|
||||
(get_test_example_file('target-240p-30fps.mp4'), 0, 100, 100),
|
||||
(get_test_example_file('target-240p-60fps.mp4'), 0, 648, 324),
|
||||
(get_test_example_file('target-240p-60fps.mp4'), 224, 648, 212),
|
||||
(get_test_example_file('target-240p-60fps.mp4'), 124, 224, 50),
|
||||
(get_test_example_file('target-240p-60fps.mp4'), 0, 100, 50)
|
||||
(get_test_example_file('target-240p-25fps.mp4'), 0, 270, 324, 55, 250),
|
||||
(get_test_example_file('target-240p-25fps.mp4'), 224, 270, 55, 55, 250),
|
||||
(get_test_example_file('target-240p-25fps.mp4'), 124, 224, 120, 55, 250),
|
||||
(get_test_example_file('target-240p-25fps.mp4'), 0, 100, 120, 55, 250),
|
||||
(get_test_example_file('target-240p-30fps.mp4'), 0, 324, 324, 55, 250),
|
||||
(get_test_example_file('target-240p-30fps.mp4'), 224, 324, 100, 55, 250),
|
||||
(get_test_example_file('target-240p-30fps.mp4'), 124, 224, 100, 55, 250),
|
||||
(get_test_example_file('target-240p-30fps.mp4'), 0, 100, 100, 55, 250),
|
||||
(get_test_example_file('target-240p-60fps.mp4'), 0, 648, 324, 55, 250),
|
||||
(get_test_example_file('target-240p-60fps.mp4'), 224, 648, 212, 55, 250),
|
||||
(get_test_example_file('target-240p-60fps.mp4'), 124, 224, 50, 55, 250),
|
||||
(get_test_example_file('target-240p-60fps.mp4'), 0, 100, 50, 55, 250),
|
||||
(get_test_example_file('target-240p-smpte2084.mp4'), 0, 1, 1, 32, 190)
|
||||
]
|
||||
|
||||
for target_path, trim_frame_start, trim_frame_end, frame_total in test_set:
|
||||
for target_path, trim_frame_start, trim_frame_end, frame_total, frame_std, frame_max in test_set:
|
||||
create_temp_directory(target_path)
|
||||
|
||||
assert extract_frames(target_path, (452, 240), 30.0, trim_frame_start, trim_frame_end) is True
|
||||
assert len(resolve_temp_frame_set(target_path)) == frame_total
|
||||
|
||||
temp_vision_frame = read_image(resolve_temp_frame_set(target_path).get(trim_frame_start))
|
||||
|
||||
assert temp_vision_frame.std() > frame_std
|
||||
assert temp_vision_frame.max() > frame_max
|
||||
|
||||
clear_temp_directory(target_path)
|
||||
|
||||
|
||||
def test_merge_video() -> None:
|
||||
target_paths =\
|
||||
test_set =\
|
||||
[
|
||||
get_test_example_file('target-240p-16khz.avi'),
|
||||
get_test_example_file('target-240p-16khz.m4v'),
|
||||
get_test_example_file('target-240p-16khz.mkv'),
|
||||
get_test_example_file('target-240p-16khz.mp4'),
|
||||
get_test_example_file('target-240p-16khz.mov'),
|
||||
get_test_example_file('target-240p-16khz.webm'),
|
||||
get_test_example_file('target-240p-16khz.wmv')
|
||||
(get_test_example_file('target-240p-16khz.avi'), [ 'bt709', 'unknown' ]),
|
||||
(get_test_example_file('target-240p-16khz.m4v'), [ 'bt709' ]),
|
||||
(get_test_example_file('target-240p-16khz.mkv'), [ 'bt709' ]),
|
||||
(get_test_example_file('target-240p-16khz.mp4'), [ 'bt709' ]),
|
||||
(get_test_example_file('target-240p-16khz.mov'), [ 'bt709' ]),
|
||||
(get_test_example_file('target-240p-16khz.webm'), [ 'bt709' ]),
|
||||
(get_test_example_file('target-240p-16khz.wmv'), [ 'bt709' ])
|
||||
]
|
||||
output_video_encoders = get_available_encoder_set().get('video')
|
||||
|
||||
for target_path in target_paths:
|
||||
for target_path, color_transfers in test_set:
|
||||
for output_video_encoder in output_video_encoders:
|
||||
state_manager.init_item('output_video_encoder', output_video_encoder)
|
||||
create_temp_directory(target_path)
|
||||
@@ -111,6 +156,10 @@ def test_merge_video() -> None:
|
||||
|
||||
assert merge_video(target_path, 25.0, (452, 240), 25.0, 0, 1) is True
|
||||
|
||||
video_metadata = extract_video_metadata(get_temp_file_path(target_path))
|
||||
|
||||
assert video_metadata.get('color_transfer') in color_transfers
|
||||
|
||||
clear_temp_directory(target_path)
|
||||
|
||||
state_manager.init_item('output_video_encoder', 'libx264')
|
||||
@@ -187,3 +236,29 @@ def test_replace_audio() -> None:
|
||||
clear_temp_directory(target_path)
|
||||
|
||||
state_manager.init_item('output_audio_encoder', 'aac')
|
||||
|
||||
|
||||
def test_fix_audio_encoder() -> None:
|
||||
assert fix_audio_encoder('avi', 'libopus') == 'aac'
|
||||
assert fix_audio_encoder('m4v', 'libopus') == 'aac'
|
||||
assert fix_audio_encoder('mpeg', 'libopus') == 'aac'
|
||||
assert fix_audio_encoder('wmv', 'libopus') == 'aac'
|
||||
assert fix_audio_encoder('mov', 'flac') == 'aac'
|
||||
assert fix_audio_encoder('mov', 'libopus') == 'aac'
|
||||
assert fix_audio_encoder('mxf', 'libopus') == 'pcm_s16le'
|
||||
assert fix_audio_encoder('webm', 'aac') == 'libopus'
|
||||
assert fix_audio_encoder('mp4', 'aac') == 'aac'
|
||||
assert fix_audio_encoder('avi', 'aac') == 'aac'
|
||||
|
||||
|
||||
def test_fix_video_encoder() -> None:
|
||||
assert fix_video_encoder('m4v', 'libx265') == 'libx264'
|
||||
assert fix_video_encoder('mpeg', 'libx265') == 'libx264'
|
||||
assert fix_video_encoder('mxf', 'libx265') == 'libx264'
|
||||
assert fix_video_encoder('wmv', 'libx265') == 'libx264'
|
||||
assert fix_video_encoder('mkv', 'rawvideo') == 'libx264'
|
||||
assert fix_video_encoder('mp4', 'rawvideo') == 'libx264'
|
||||
assert fix_video_encoder('mov', 'libvpx-vp9') == 'libx264'
|
||||
assert fix_video_encoder('webm', 'libx264') == 'libvpx-vp9'
|
||||
assert fix_video_encoder('mp4', 'libx265') == 'libx265'
|
||||
assert fix_video_encoder('avi', 'rawvideo') == 'rawvideo'
|
||||
|
||||
Reference in New Issue
Block a user