Commit Graph

86 Commits

Author SHA1 Message Date
henryruhs cc915171ea Add lip-syncer support to source component 2024-02-01 12:59:23 +01:00
henryruhs 3c8ad63d25 Add lip-syncer support to source component 2024-02-01 12:51:59 +01:00
Henry Ruhs d44b38103d Cuda 12 installer (#362)
* Make cuda nightly (12) the default

* Better keep legacy cuda just in case

* Use CUDA and ROCM versions

* Remove MacOS options from installer (CoreML include in default package)
2024-02-01 10:14:31 +01:00
henryruhs 0d00c74bc1 Minor cleanup 2024-01-31 19:10:13 +01:00
henryruhs aa9dd20cc1 Fix testing 2024-01-31 18:38:44 +01:00
henryruhs 06cb05bd3f Fix testing 2024-01-31 18:33:11 +01:00
henryruhs 872094a1fd Rename to lip syncer 2024-01-31 18:25:58 +01:00
henryruhs 7f9348dcaa Simplify testing 2024-01-31 18:12:46 +01:00
Harisreedhar 808460ae09 Lip Sync (#356)
* Cli implementation of wav2lip

* - create get_first_item()
- remove non gan wav2lip model
- implement video memory strategy
- implement get_reference_frame()
- implement process_image()
- rearrange crop_mask_list
- implement test_cli
2024-01-31 17:37:04 +01:00
henryruhs b50f021743 Fix the output framerate vs. time 2024-01-31 09:28:40 +01:00
Henry Ruhs d19e5918dd Add age and gender to face debugger items (#353)
* Add age and gender to face debugger items

* Rename like suggested in the code review
2024-01-31 09:16:34 +01:00
henryruhs 0b345a7e33 Minor cleanups 2024-01-30 21:26:24 +01:00
henryruhs ce579846b4 Enforce output resolution on merge video 2024-01-30 13:23:37 +01:00
Harisreedhar bc408a9102 include audio file functions 2024-01-30 15:11:36 +05:30
henryruhs 4b56086fd0 Reduce terminal output 2024-01-29 20:44:05 +01:00
henryruhs 5a4cd4bbdf Clear temp before processing 2024-01-29 20:35:08 +01:00
henryruhs 0870422731 Force Gradio to shut up 2024-01-29 14:50:24 +01:00
henryruhs 7703eeda95 Force Gradio to shut up 2024-01-29 14:46:30 +01:00
henryruhs 9effc3c877 Revert venv on the fly 2024-01-28 20:50:14 +01:00
henryruhs 32d2941f32 Revert venv on the fly 2024-01-28 20:40:53 +01:00
henryruhs 6fb23fe747 Enable venv on the fly 2024-01-28 20:35:36 +01:00
henryruhs f1c7b3d6c9 Enable venv on the fly 2024-01-28 20:14:23 +01:00
henryruhs f84ff766bd Fix indent 2024-01-28 19:45:18 +01:00
henryruhs 87bcc3e334 Add more todo for testing 2024-01-28 15:18:45 +01:00
henryruhs dcd4576e03 Add more todo for testing 2024-01-28 15:17:21 +01:00
henryruhs 066665aae7 Introduce audio testing 2024-01-28 12:54:42 +01:00
henryruhs 895aac5549 Minor tidy up 2024-01-28 12:42:17 +01:00
Henry Ruhs b7b81303e9 Rename Frame to VisionFrame (#346) 2024-01-28 12:31:22 +01:00
Harisreedhar 3a2127eb63 Audio functions (#345)
* Update ffmpeg.py

* Create audio.py

* Update ffmpeg.py

* Update audio.py

* Update audio.py

* Update typing.py

* Update ffmpeg.py

* Update audio.py
2024-01-28 12:25:49 +01:00
Henry Ruhs 32cbf0ca5b Update help message for arguments, Notation based wording approach (#347)
* Update help message for arguments, Notation based wording approach

* Fix installer
2024-01-28 10:49:44 +01:00
henryruhs 0f65313d05 Use correct typing in choices 2024-01-27 20:33:02 +01:00
henryruhs 9a0a9e6f7a Fix naming 2024-01-26 18:06:56 +01:00
henryruhs e2e3cd7dbd Change order 2024-01-25 11:46:30 +01:00
henryruhs 38ba53e5a3 Add testing for audio handlers 2024-01-23 10:52:12 +01:00
Harisreedhar 42c591f758 include audio file functions (#336) 2024-01-23 08:31:49 +01:00
Henry Ruhs 6aea186ee8 feat/yoloface (#334)
* added yolov8 to face_detector (#323)

* added yolov8 to face_detector

* added yolov8 to face_detector

* Initial cleanup and renaming

* Update README

* refactored detect_with_yoloface (#329)

* refactored detect_with_yoloface

* apply review

* Change order again

* Restore working code

* modified code (#330)

* refactored detect_with_yoloface

* apply review

* use temp_frame in detect_with_yoloface

* reorder

* modified

* reorder models

* Tiny cleanup

---------

Co-authored-by: tamoharu <133945583+tamoharu@users.noreply.github.com>
2024-01-22 13:56:45 +01:00
henryruhs 122da0545b Improve naming 2024-01-21 14:47:28 +01:00
Henry Ruhs 906c78126a Allow partial facefusion.ini file (#325) 2024-01-21 14:41:59 +01:00
Henry Ruhs 7e990ac5e5 Fix/image to image (#322)
* Fix image to image when using tolerate

* Bump version
2024-01-21 14:17:53 +01:00
Henry Ruhs 7c3900685c Replace restoreformer with restoreformer++ (#324)
* Replace restoreformer with restoreformer++

* Replace restoreformer with restoreformer++
2024-01-21 14:17:35 +01:00
Henry Ruhs f6e56a3d8c Next (#318)
* renaming and restructuring (#282)

* Renaming and restructuring

* Renaming and restructuring

* Renaming and restructuring

* Fix gender detection

* Implement distance to face debugger

* Implement distance to face debugger part2

* Implement distance to face debugger part3

* Mark as next

* Fix reference when face_debugger comes first

* Use official onnxruntime nightly

* CUDA on steroids

* CUDA on steroids

* Add some testing

* Set inswapper_128_fp16 as default

* Feat/block until post check (#292)

* Block until download is done

* Introduce post_check()

* Fix webcam

* Update dependencies

* Add --force-reinstall to installer

* Introduce config ini (#298)

* Introduce config ini

* Fix output video encoder

* Revert help listings back to commas, Move SSL hack to download.py

* Introduce output-video-preset which defaults to veryfast

* Mapping for nvenc encoders

* Rework on events and non-blocking UI

* Add fast bmp to temp_frame_formats

* Add fast bmp to temp_frame_formats

* Show total processing time on success

* Show total processing time on success

* Show total processing time on success

* Move are_images, is_image and is_video back to filesystem

* Fix some spacings

* Pissing everyone of by renaming stuff

* Fix seconds output

* feat/video output fps (#312)

* added output fps slider, removed 'keep fps' option (#311)

* added output fps slider, removed 'keep fps' option

* now uses passed fps instead of global fps for ffmpeg

* fps values are now floats instead of ints

* fix previous commit

* removed default value from fps slider

this is so we can implement a dynamic default value later

* Fix seconds output

* Some cleanup

---------

Co-authored-by: Ran Shaashua <47498956+ranshaa05@users.noreply.github.com>

* Allow 0.01 steps for fps

* Make fps unregulated

* Make fps unregulated

* Remove distance from face debugger again (does not work)

* Fix gender age

* Fix gender age

* Hotfix benchmark suite

* Warp face normalize (#313)

* use normalized kp templates

* Update face_helper.py

* My 50 cents to warp_face()

---------

Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>

* face-swapper-weight (#315)

* Move prepare_crop_frame and normalize_crop_frame out of apply_swap

* Fix UI bug with different range

* feat/output video resolution (#316)

* Introduce detect_video_resolution, Rename detect_fps to detect_video_fps

* Add calc_video_resolution_range

* Make output resolution work, does not auto-select yet

* Make output resolution work, does not auto-select yet

* Try to keep the origin resolution

* Split code into more fragments

* Add pack/unpack resolution

* Move video_template_sizes to choices

* Improve create_video_resolutions

* Reword benchmark suite

* Optimal speed for benchmark

* Introduce different video memory strategies, rename max_memory to max… (#317)

* Introduce different video memory strategies, rename max_memory to max_system_memory

* Update readme

* Fix limit_system_memory call

* Apply video_memory_strategy to face debugger

* Limit face swapper weight to 3.0

* Remove face swapper weight due bad render outputs

* Show/dide logic for output video preset

* fix uint8 conversion

* Fix whitespace

* Finalize layout and update preview

* Fix multi renders on face debugger

* Restore less restrictive rendering of preview and stream

* Fix block mode for model downloads

* Add testing

* Cosmetic changes

* Enforce valid fps and resolution via CLI

* Empty config

* Cosmetics on args processing

* Memory workover (#319)

* Cosmetics on args processing

* Fix for MacOS

* Rename all max_ to _limit

* More fixes

* Update preview

* Fix whitespace

---------

Co-authored-by: Ran Shaashua <47498956+ranshaa05@users.noreply.github.com>
Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>
2024-01-19 19:57:48 +01:00
henryruhs 3e93f99eeb Bump version 2023-12-26 19:41:49 +01:00
henryruhs 98d0bd2d4a Hotfix onnxruntime-directml for Windows 2023-12-26 19:38:36 +01:00
Henry Ruhs b267629161 2.1.2 (#271)
* Fix round issues on CLI arguments with range

* Refresh face references on frame processor change

* Add cuda 12 support to installer

* Add cuda 12 support to installer

* Add cuda 12 support to installer

* Add cuda 12 support to installer

* Simplify check

* Simplify check
2023-12-24 15:55:21 +01:00
henryruhs e18e84e4f8 Bump to 2.1.1 2023-12-20 22:46:59 +01:00
henryruhs d67a84642c Remove falsy clear reference faces in preview 2023-12-20 22:35:32 +01:00
Henry Ruhs 3a5fe2a602 2.1.0 (#253)
* Operating system specific installer options

* Update dependencies

* Sorting before NMS according to the standard

* Minor typing fix

* Change the wording

* Update preview.py (#222)

Added a release listener to the preview frame slider, this will update the frame preview with the latest frame

* Combine preview slider listener

* Remove change listener

* Introduce multi source (#223)

* Implement multi source

* Adjust face enhancer and face debugger to multi source

* Implement multi source to UI

* Implement multi source to UI part2

* Implement multi source to UI part3

* Implement multi source to UI part4

* Some cleanup

* Add face occluder (#225) (#226)

* Add face occluder (#225)

* add face-occluder (commandline only)

* review 1

* Update face_masker.py

* Update face_masker.py

* Add gui & fix typing

* Minor naming cleanup

* Minor naming cleanup part2

---------

Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>

* Update usage information

* Fix averaged normed_embedding

* Remove blur from face occluder, enable accelerators

* Switch to RANSAC with 100 threshold

* Update face_enhancer.py (#229)

* Update face_debugger.py (#230)

* Split utilities (#232)

* Split utilities

* Split utilities part2

* Split utilities part3

* Split utilities part4

* Some cleanup

* Implement log level support (#233)

* Implement log level support

* Fix testing

* Implement debug logger

* Implement debug logger

* Fix alignment offset (#235)

* Update face_helper.py

* fix 2

* Enforce virtual environment via installer

* Enforce virtual environment via installer

* Enforce virtual environment via installer

* Enforce virtual environment via installer

* Feat/multi process reference faces (#239)

* Multi processing aware reference faces

* First clean up and joining of files

* Finalize the face store

* Reduce similar face detection to one set, use __name__ for scopes in logger

* Rename to face_occluder

* Introduce ModelSet type

* Improve webcam error handling

* Prevent null pointer on is_image() and is_video()

* Prevent null pointer on is_image() and is_video()

* Fix find similar faces

* Fix find similar faces

* Fix process_images for face enhancer

* Bunch of minor improvements

* onnxruntime for ROCM under linux

* Improve mask related naming

* Fix falsy import

* Fix typo

* Feat/face parser refactoring (#247)

* Face parser update (#244)

* face-parser

* Update face_masker.py

* update debugger

* Update globals.py

* Update face_masker.py

* Refactor code to split occlusion from region

* fix (#246)

* fix

* fix debugger resolution

* flip input to horizontal

* Clean up UI

* Reduce the regions to inside face only

* Reduce the regions to inside face only

---------

Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>

* Fix enhancer, remove useless dest in add_argument()

* Prevent unselect of the face_mask_regions via UI

* Prepare next release

* Shorten arguments that have choices and nargs

* Add missing clear to face debugger

---------

Co-authored-by: Mathias <github@feroc.de>
Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>
2023-12-20 00:00:32 +01:00
henryruhs e70430703b Fix typo in README 2023-12-15 23:07:31 +01:00
henryruhs 2e5cce112c Rename blendface to blendswap 2023-12-10 12:47:06 +01:00
Henry Ruhs 6587d2def1 Next (#216)
* Simplify bbox access

* Code cleanup

* Simplify bbox access

* Move code to face helper

* Swap and paste back without insightface

* Swap and paste back without insightface

* Remove semaphore where possible

* Improve paste back performance

* Cosmetic changes

* Move the predictor to ONNX to avoid tensorflow, Use video ranges for prediction

* Make CI happy

* Move template and size to the options

* Fix different color on box

* Uniform model handling for predictor

* Uniform frame handling for predictor

* Pass kps direct to warp_face

* Fix urllib

* Analyse based on matches

* Analyse based on rate

* Fix CI

* ROCM and OpenVINO mapping for torch backends

* Fix the paste back speed

* Fix import

* Replace retinaface with yunet (#168)

* Remove insightface dependency

* Fix urllib

* Some fixes

* Analyse based on matches

* Analyse based on rate

* Fix CI

* Migrate to Yunet

* Something is off here

* We indeed need semaphore for yunet

* Normalize the normed_embedding

* Fix download of models

* Fix download of models

* Fix download of models

* Add score and improve affine_matrix

* Temp fix for bbox out of frame

* Temp fix for bbox out of frame

* ROCM and OpenVINO mapping for torch backends

* Normalize bbox

* Implement gender age

* Cosmetics on cli args

* Prevent face jumping

* Fix the paste back speed

* FIx import

* Introduce detection size

* Cosmetics on face analyser ARGS and globals

* Temp fix for shaking face

* Accurate event handling

* Accurate event handling

* Accurate event handling

* Set the reference_frame_number in face_selector component

* Simswap model (#171)

* Add simswap models

* Add ghost models

* Introduce normed template

* Conditional prepare and normalize for ghost

* Conditional prepare and normalize for ghost

* Get simswap working

* Get simswap working

* Fix refresh of swapper model

* Refine face selection and detection (#174)

* Refine face selection and detection

* Update README.md

* Fix some face analyser UI

* Fix some face analyser UI

* Introduce range handling for CLI arguments

* Introduce range handling for CLI arguments

* Fix some spacings

* Disable onnxruntime warnings

* Use cv2.blur over cv2.GaussianBlur for better performance

* Revert "Use cv2.blur over cv2.GaussianBlur for better performance"

This reverts commit bab666d6f9.

* Prepare universal face detection

* Prepare universal face detection part2

* Reimplement retinaface

* Introduce cached anchors creation

* Restore filtering to enhance performance

* Minor changes

* Minor changes

* More code but easier to understand

* Minor changes

* Rename predictor to content analyser

* Change detection/recognition to detector/recognizer

* Fix crop frame borders

* Fix spacing

* Allow normalize output without a source

* Improve conditional set face reference

* Update dependencies

* Add timeout for get_download_size

* Fix performance due disorder

* Move models to assets repository, Adjust namings

* Refactor face analyser

* Rename models once again

* Fix spacing

* Highres simswap (#192)

* Introduce highres simswap

* Fix simswap 256 color issue (#191)

* Fix simswap 256 color issue

* Update face_swapper.py

* Normalize models and host in our repo

* Normalize models and host in our repo

---------

Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>

* Rename face analyser direction to face analyser order

* Improve the UI for face selector

* Add best-worst, worst-best detector ordering

* Clear as needed and fix zero score bug

* Fix linter

* Improve startup time by multi thread remote download size

* Just some cosmetics

* Normalize swagger source input, Add blendface_256 (unfinished)

* New paste back (#195)

* add new paste_back (#194)

* add new paste_back

* Update face_helper.py

* Update face_helper.py

* add commandline arguments and gui

* fix conflict

* Update face_mask.py

* type fix

* Clean some wording and typing

---------

Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>

* Clean more names, use blur range approach

* Add blur padding range

* Change the padding order

* Fix yunet filename

* Introduce face debugger

* Use percent for mask padding

* Ignore this

* Ignore this

* Simplify debugger output

* implement blendface (#198)

* Clean up after the genius

* Add gpen_bfr_256

* Cosmetics

* Ignore face_mask_padding on face enhancer

* Update face_debugger.py (#202)

* Shrink debug_face() to a minimum

* Mark as 2.0.0 release

* remove unused (#204)

* Apply NMS (#205)

* Apply NMS

* Apply NMS part2

* Fix restoreformer url

* Add debugger cli and gui components (#206)

* Add debugger cli and gui components

* update

* Polishing the types

* Fix usage in README.md

* Update onnxruntime

* Support for webp

* Rename paste-back to face-mask

* Add license to README

* Add license to README

* Extend face selector mode by one

* Update utilities.py (#212)

* Stop inline camera on stream

* Minor webcam updates

* Gracefully start and stop webcam

* Rename capture to video_capture

* Make get webcam capture pure

* Check webcam to not be None

* Remove some is not None

* Use index 0 for webcam

* Remove memory lookup within progress bar

* Less progress bar updates

* Uniform progress bar

* Use classic progress bar

* Fix image and video validation

* Use different hash for cache

* Use best-worse order for webcam

* Normalize padding like CSS

* Update preview

* Fix max memory

* Move disclaimer and license to the docs

* Update wording in README

* Add LICENSE.md

* Fix argument in README

---------

Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>
Co-authored-by: alex00ds <31631959+alex00ds@users.noreply.github.com>
2023-11-28 17:29:24 +01:00