diff --git a/README.md b/README.md index 3471013..ecfb737 100644 --- a/README.md +++ b/README.md @@ -2,32 +2,58 @@ ## Proceedings of the 28th ACM International Conference on Multimedia **The official repository with Pytorch** -Currently, only the test code is available, and training scripts are coming soon +*Our method can realize **arbitrary face swapping** on images and videos with **one single trained model**.* + +Currently, only the test code is available. Training scripts are coming soon + [![simswaplogo](/doc/img/logo1.png)](https://github.com/neuralchen/SimSwap) +Our paper can be downloaded from [[Arxiv]](https://arxiv.org/pdf/2106.06340v1.pdf) + -[[Arxiv paper]](https://arxiv.org/pdf/2106.06340v1.pdf) +## Top News -[[ACM DOI paper]](https://dl.acm.org/doi/10.1145/3394171.3413630) +**`2021-06-20`**: We release the scripts for arbitrary video and image processing. -[[Google Drive Paper link]](https://drive.google.com/file/d/1fcfWOGt1mkBo7F0gXVKitf8GJMAXQxZD/view?usp=sharing) +## Dependencies +- python3.6+ +- pytorch1.5+ +- torchvision +- opencv +- pillow +- numpy +- moviepy +- insightface + +## Usage +[Preparation](./doc/guidance/preparation.md) + +[Inference for image or video face swapping](./doc/guidance/usage.md) + +Training: **coming soon** -[[Baidu Drive Paper link]](https://pan.baidu.com/s/1-TKFuycRNUKut8hn4IimvA) Password: ```ummt``` - -![video2](/doc/img/anni.webp) -![video3](/doc/img/chenglong.webp) -![video4](/doc/img/zhoujielun.webp) -![video5](/doc/img/zhuyin.webp) +## Video + +
+ + + + +
## Results ![Results1](/doc/img/results1.PNG) ![Results2](/doc/img/total.PNG) -## Video - + **High-quality videos can be found in the link below:** @@ -45,34 +71,6 @@ Currently, only the test code is available, and training scripts are coming soon [[Online Video]](https://www.bilibili.com/video/BV12v411p7j5/) -## Dependencies -- python3.6+ -- pytorch1.5+ -- torchvision -- opencv -- pillow -- numpy - - -## Usage -### To test the pretrained model -``` -python test_one_image.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path crop_224/6.jpg --pic_b_path crop_224/ds.jpg --output_path output/ -``` - ---name refers to the SimSwap training logs name. - -## Pretrained model - -### Usage -There are two archive files in the drive: **checkpoints.zip** and **arcface_checkpoint.tar** - -- **Copy the arcface_checkpoint.tar into ./arcface_model** -- **Unzip checkpoints.zip, place it in the root dir ./** - -[[Google Drive]](https://drive.google.com/drive/folders/1jV6_0FIMPC53FZ2HzZNJZGMe55bbu17R?usp=sharing) - -[[Baidu Drive]](https://pan.baidu.com/s/1wFV11RVZMHqd-ky4YpLdcA) Password: ```jd2v``` ## To cite our paper diff --git a/doc/guidance/preparation.md b/doc/guidance/preparation.md new file mode 100644 index 0000000..d2704a4 --- /dev/null +++ b/doc/guidance/preparation.md @@ -0,0 +1,28 @@ + +# Preparation + +### Installation +**We highly recommand that you use Anaconda for Installation** +``` +conda create -n simswap python=3.6 +conda activate simswap +conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=10.2 -c pytorch +(option): pip install --ignore-installed imageio +pip install insightface==0.2.1 onnxruntime moviepy +``` +- We use the face detection and alignment methods from **[insightface](https://github.com/deepinsight/insightface)** for image preprocessing. Please download the relative files and unzip them to ./insightface_func/models from [this link](https://onedrive.live.com/?authkey=%21ADJ0aAOSsc90neY&cid=4A83B6B633B029CC&id=4A83B6B633B029CC%215837&parId=4A83B6B633B029CC%215834&action=locate). +- The pytorch and cuda versions above are most recommanded. They may vary. +- Using insightface with different versions is not recommanded. Please use this specific version. +- These settings are tested valid on both Windows and Ununtu. + +### Pretrained model +There are two archive files in the drive: **checkpoints.zip** and **arcface_checkpoint.tar** + +- **Copy the arcface_checkpoint.tar into ./arcface_model** +- **Unzip checkpoints.zip, place it in the root dir ./** + +[[Google Drive]](https://drive.google.com/drive/folders/1jV6_0FIMPC53FZ2HzZNJZGMe55bbu17R?usp=sharing) +[[Baidu Drive]](https://pan.baidu.com/s/1wFV11RVZMHqd-ky4YpLdcA) Password: ```jd2v``` + +### Note +We expect users to have GPU with at least 8G memory. For those who do not, we will provide Colab Notebook implementation in the future. diff --git a/doc/guidance/usage.md b/doc/guidance/usage.md new file mode 100644 index 0000000..adafd99 --- /dev/null +++ b/doc/guidance/usage.md @@ -0,0 +1,47 @@ + +# Usage + +### Simple face swapping for already face-aligned images +``` +python test_one_image.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path crop_224/6.jpg --pic_b_path crop_224/ds.jpg --output_path output/ +``` + +### Face swapping for video +- Swap only one face within the video(the one with highest confidence by face detection). +``` +python test_video_swapsingle.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path ./demo_file/Iron_man.jpg --video_path ./demo_file/mutil_people_1080p.mp4 --output_path ./output/mutil_test_swapsingle.mp4 --temp_path ./temp_results +``` + +- Swap all faces within the video. +``` +python test_video_swapmutil.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path ./demo_file/Iron_man.jpg --video_path ./demo_file/mutil_people_1080p.mp4 --output_path ./output/mutil_test_swapmutil.mp4 --temp_path ./temp_results +``` + + + + +### Face swapping for Arbitrary images +- Swap only one face within one image(the one with highest confidence by face detection). The result would be saved to ./output/result_whole_swapsingle.jpg +``` +python test_wholeimage_swapsingle.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path ./demo_file/Iron_man.jpg --pic_b_path ./demo_file/mutil_people.jpg --output_path ./output/ +``` + +- Swap all faces within one image. The result would be saved to ./output/result_whole_swapmutil.jpg +``` +python test_wholeimage_swapmutil.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path ./demo_file/Iron_man.jpg --pic_b_path ./demo_file/mutil_people.jpg --output_path ./output/ +``` +Difference between single face swapping and all face swapping are shown below. + + +### Parameters +| Parameters | Function | +| :---- | :---- | +| --name | The SimSwap training logs name | +| --pic_a_path | Path of image with the target face | +| --pic_b_path | Path of image with the source face to swap | +| --video_path | Path of video with the source face to swap | +| --temp_path | Path to store intermediate files | +| --output_path | Path of directory to store the face swapping result | + +### Note +We expect users to have GPU with at least 8G memory. For those who do not, we will provide Colab Notebook implementation in the future. diff --git a/doc/img/multi_face_comparison.png b/doc/img/multi_face_comparison.png new file mode 100644 index 0000000..b319572 Binary files /dev/null and b/doc/img/multi_face_comparison.png differ