CalvinBackup/neuralchen-SimSwap
1
0
Fork 0
Code Issues 288 Pull Requests 16 Actions Packages Projects Releases 2 Wiki Activity

Update guidance #13

Merged
AceSix merged 7 commits from Six into main 2021-06-20 07:25:03 +02:00
Conversation 0 Commits 7 Files Changed 4 +113 -40
4 changed files with 113 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files

78
README.md
View File

@@ -2,32 +2,58 @@
## Proceedings of the 28th ACM International Conference on Multimedia
**The official repository with Pytorch**
Currently, only the test code is available, and training scripts are coming soon
*Our method can realize **arbitrary face swapping** on images and videos with **one single trained model**.*
Currently, only the test code is available. Training scripts are coming soon
[![simswaplogo](/doc/img/logo1.png)](https://github.com/neuralchen/SimSwap)
Our paper can be downloaded from [[Arxiv]](https://arxiv.org/pdf/2106.06340v1.pdf)
<!-- [[ACM DOI]](https://dl.acm.org/doi/10.1145/3394171.3413630)
[[Google Drive]](https://drive.google.com/file/d/1fcfWOGt1mkBo7F0gXVKitf8GJMAXQxZD/view?usp=sharing)
[[Baidu Drive ]](https://pan.baidu.com/s/1-TKFuycRNUKut8hn4IimvA) Password: ```ummt``` -->
[[Arxiv paper]](https://arxiv.org/pdf/2106.06340v1.pdf)
## Top News
[[ACM DOI paper]](https://dl.acm.org/doi/10.1145/3394171.3413630)
**`2021-06-20`**: We release the scripts for arbitrary video and image processing.
[[Google Drive Paper link]](https://drive.google.com/file/d/1fcfWOGt1mkBo7F0gXVKitf8GJMAXQxZD/view?usp=sharing)
## Dependencies
- python3.6+
- pytorch1.5+
- torchvision
- opencv
- pillow
- numpy
- moviepy
- insightface
## Usage
[Preparation](./doc/guidance/preparation.md)
[Inference for image or video face swapping](./doc/guidance/usage.md)
Training: **coming soon**
[[Baidu Drive Paper link]](https://pan.baidu.com/s/1-TKFuycRNUKut8hn4IimvA) Password: ```ummt```
![video2](/doc/img/anni.webp)
![video3](/doc/img/chenglong.webp)
![video4](/doc/img/zhoujielun.webp)
![video5](/doc/img/zhuyin.webp)
## Video
<img src="./doc/img/video.webp"/>
<div>
<img width=24% src="./doc/img/anni.webp"/>
<img width=24% src="./doc/img/chenglong.webp"/>
<img width=24% src="./doc/img/zhoujielun.webp"/>
<img width=24% src="./doc/img/zhuyin.webp"/>
</div>
## Results
![Results1](/doc/img/results1.PNG)
![Results2](/doc/img/total.PNG)
## Video
<img src="./doc/img/video.webp"/>
<!-- ![video2](/doc/img/anni.webp)
![video3](/doc/img/chenglong.webp)
![video4](/doc/img/zhoujielun.webp)
![video5](/doc/img/zhuyin.webp) -->
**High-quality videos can be found in the link below:**
@@ -45,34 +71,6 @@ Currently, only the test code is available, and training scripts are coming soon
[[Online Video]](https://www.bilibili.com/video/BV12v411p7j5/)
## Dependencies
- python3.6+
- pytorch1.5+
- torchvision
- opencv
- pillow
- numpy
## Usage
### To test the pretrained model
```
python test_one_image.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path crop_224/6.jpg --pic_b_path crop_224/ds.jpg --output_path output/
```
--name refers to the SimSwap training logs name.
## Pretrained model
### Usage
There are two archive files in the drive: **checkpoints.zip** and **arcface_checkpoint.tar**
- **Copy the arcface_checkpoint.tar into ./arcface_model**
- **Unzip checkpoints.zip, place it in the root dir ./**
[[Google Drive]](https://drive.google.com/drive/folders/1jV6_0FIMPC53FZ2HzZNJZGMe55bbu17R?usp=sharing)
[[Baidu Drive]](https://pan.baidu.com/s/1wFV11RVZMHqd-ky4YpLdcA) Password: ```jd2v```
## To cite our paper

28
doc/guidance/preparation.md Normal file
View File

@@ -0,0 +1,28 @@
# Preparation
### Installation
**We highly recommand that you use Anaconda for Installation**
```
conda create -n simswap python=3.6
conda activate simswap
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=10.2 -c pytorch
(option): pip install --ignore-installed imageio
pip install insightface==0.2.1 onnxruntime moviepy
```
- We use the face detection and alignment methods from **[insightface](https://github.com/deepinsight/insightface)** for image preprocessing. Please download the relative files and unzip them to ./insightface_func/models from [this link](https://onedrive.live.com/?authkey=%21ADJ0aAOSsc90neY&cid=4A83B6B633B029CC&id=4A83B6B633B029CC%215837&parId=4A83B6B633B029CC%215834&action=locate).
- The pytorch and cuda versions above are most recommanded. They may vary.
- Using insightface with different versions is not recommanded. Please use this specific version.
- These settings are tested valid on both Windows and Ununtu.
### Pretrained model
There are two archive files in the drive: **checkpoints.zip** and **arcface_checkpoint.tar**
- **Copy the arcface_checkpoint.tar into ./arcface_model**
- **Unzip checkpoints.zip, place it in the root dir ./**
[[Google Drive]](https://drive.google.com/drive/folders/1jV6_0FIMPC53FZ2HzZNJZGMe55bbu17R?usp=sharing)
[[Baidu Drive]](https://pan.baidu.com/s/1wFV11RVZMHqd-ky4YpLdcA) Password: ```jd2v```
### Note
We expect users to have GPU with at least 8G memory. For those who do not, we will provide Colab Notebook implementation in the future.

47
doc/guidance/usage.md Normal file
View File

@@ -0,0 +1,47 @@
# Usage
### Simple face swapping for already face-aligned images
```
python test_one_image.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path crop_224/6.jpg --pic_b_path crop_224/ds.jpg --output_path output/
```
### Face swapping for video
- Swap only one face within the video(the one with highest confidence by face detection).
```
python test_video_swapsingle.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path ./demo_file/Iron_man.jpg --video_path ./demo_file/mutil_people_1080p.mp4 --output_path ./output/mutil_test_swapsingle.mp4 --temp_path ./temp_results
```
- Swap all faces within the video.
```
python test_video_swapmutil.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path ./demo_file/Iron_man.jpg --video_path ./demo_file/mutil_people_1080p.mp4 --output_path ./output/mutil_test_swapmutil.mp4 --temp_path ./temp_results
```
### Face swapping for Arbitrary images
- Swap only one face within one image(the one with highest confidence by face detection). The result would be saved to ./output/result_whole_swapsingle.jpg
```
python test_wholeimage_swapsingle.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path ./demo_file/Iron_man.jpg --pic_b_path ./demo_file/mutil_people.jpg --output_path ./output/
```
- Swap all faces within one image. The result would be saved to ./output/result_whole_swapmutil.jpg
```
python test_wholeimage_swapmutil.py --isTrain false --name people --Arc_path arcface_model/arcface_checkpoint.tar --pic_a_path ./demo_file/Iron_man.jpg --pic_b_path ./demo_file/mutil_people.jpg --output_path ./output/
```
Difference between single face swapping and all face swapping are shown below.
<img src="../img/multi_face_comparison.png"/>
### Parameters
| Parameters | Function |
| :---- | :---- |
| --name | The SimSwap training logs name |
| --pic_a_path | Path of image with the target face |
| --pic_b_path | Path of image with the source face to swap |
| --video_path | Path of video with the source face to swap |
| --temp_path | Path to store intermediate files |
| --output_path | Path of directory to store the face swapping result |
### Note
We expect users to have GPU with at least 8G memory. For those who do not, we will provide Colab Notebook implementation in the future.

BIN
doc/img/multi_face_comparison.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.7 MiB