29 KiB
29 KiB
In [1]:
## make sure you are using a runtime with GPU
## you can check at Runtime/Change runtime type in the top bar.
!nvidia-smiSun Jul 4 06:39:39 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.27 Driver Version: 460.32.03 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |
| N/A 45C P8 9W / 70W | 0MiB / 15109MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
In [2]:
!git clone https://github.com/neuralchen/SimSwap
!cd SimSwap && git pullCloning into 'SimSwap'... remote: Enumerating objects: 667, done.[K remote: Counting objects: 100% (48/48), done.[K remote: Compressing objects: 100% (35/35), done.[K remote: Total 667 (delta 19), reused 28 (delta 13), pack-reused 619[K Receiving objects: 100% (667/667), 132.14 MiB | 44.44 MiB/s, done. Resolving deltas: 100% (292/292), done. Already up to date.
In [3]:
!pip install insightface==0.2.1 onnxruntime moviepy
!pip install googledrivedownloader
!pip install imageio==2.4.1Collecting insightface==0.2.1 Downloading https://files.pythonhosted.org/packages/ee/1e/6395bbe0db665f187c8e49266cda54fcf661f182192370d409423e4943e4/insightface-0.2.1-py2.py3-none-any.whl Collecting onnxruntime [?25l Downloading https://files.pythonhosted.org/packages/f9/76/3d0f8bb2776961c7335693df06eccf8d099e48fa6fb552c7546867192603/onnxruntime-1.8.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.5MB) [K |████████████████████████████████| 4.5MB 37.5MB/s [?25hRequirement already satisfied: moviepy in /usr/local/lib/python3.7/dist-packages (0.2.3.5) Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (0.22.2.post1) Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (2.23.0) Collecting onnx [?25l Downloading https://files.pythonhosted.org/packages/3f/9b/54c950d3256e27f970a83cd0504efb183a24312702deed0179453316dbd0/onnx-1.9.0-cp37-cp37m-manylinux2010_x86_64.whl (12.2MB) [K |████████████████████████████████| 12.2MB 32.2MB/s [?25hRequirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (3.2.2) Requirement already satisfied: Pillow in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (7.1.2) Requirement already satisfied: scikit-image in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (0.16.2) Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (4.1.2.30) Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (4.41.1) Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (1.4.1) Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (1.19.5) Requirement already satisfied: easydict in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (1.9) Requirement already satisfied: flatbuffers in /usr/local/lib/python3.7/dist-packages (from onnxruntime) (1.12) Requirement already satisfied: protobuf in /usr/local/lib/python3.7/dist-packages (from onnxruntime) (3.12.4) Requirement already satisfied: imageio<3.0,>=2.1.2 in /usr/local/lib/python3.7/dist-packages (from moviepy) (2.4.1) Requirement already satisfied: decorator<5.0,>=4.0.2 in /usr/local/lib/python3.7/dist-packages (from moviepy) (4.4.2) Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->insightface==0.2.1) (1.0.1) Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (3.0.4) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (1.24.3) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (2021.5.30) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (2.10) Requirement already satisfied: typing-extensions>=3.6.2.1 in /usr/local/lib/python3.7/dist-packages (from onnx->insightface==0.2.1) (3.7.4.3) Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from onnx->insightface==0.2.1) (1.15.0) Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (2.8.1) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (0.10.0) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (2.4.7) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (1.3.1) Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->insightface==0.2.1) (2.5.1) Requirement already satisfied: PyWavelets>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->insightface==0.2.1) (1.1.1) Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from protobuf->onnxruntime) (57.0.0) Installing collected packages: onnx, insightface, onnxruntime Successfully installed insightface-0.2.1 onnx-1.9.0 onnxruntime-1.8.0 Requirement already satisfied: googledrivedownloader in /usr/local/lib/python3.7/dist-packages (0.4) Requirement already satisfied: imageio==2.4.1 in /usr/local/lib/python3.7/dist-packages (2.4.1) Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from imageio==2.4.1) (1.19.5) Requirement already satisfied: pillow in /usr/local/lib/python3.7/dist-packages (from imageio==2.4.1) (7.1.2)
In [4]:
import os
os.chdir("SimSwap")
!lscrop_224 simswaplogo data test_one_image.py demo_file test_video_swapmulti.py docs test_video_swap_multispecific.py insightface_func test_video_swapsingle.py LICENSE test_video_swapspecific.py models test_wholeimage_swapmulti.py options test_wholeimage_swap_multispecific.py output test_wholeimage_swapsingle.py README.md test_wholeimage_swapspecific.py 'SimSwap colab.ipynb' util
In [5]:
## You can upload filed manually
# from google.colab import drive
# drive.mount('/content/gdrive')In [6]:
from google_drive_downloader import GoogleDriveDownloader
### it seems that google drive link may not be permenant, you can find this ID from our open url.
# GoogleDriveDownloader.download_file_from_google_drive(file_id='1TLNdIufzwesDbyr_nVTR7Zrx9oRHLM_N',
# dest_path='./arcface_model/arcface_checkpoint.tar')
# GoogleDriveDownloader.download_file_from_google_drive(file_id='1PXkRiBUYbu1xWpQyDEJvGKeqqUFthJcI',
# dest_path='./checkpoints.zip')
!wget -P ./arcface_model https://github.com/neuralchen/SimSwap/releases/download/1.0/arcface_checkpoint.tar
!wget https://github.com/neuralchen/SimSwap/releases/download/1.0/checkpoints.zip
!unzip ./checkpoints.zip -d ./checkpoints
!wget -P ./parsing_model/checkpoint https://github.com/neuralchen/SimSwap/releases/download/1.0/79999_iter.pth--2021-07-04 06:39:56-- https://github.com/neuralchen/SimSwap/releases/download/1.0/arcface_checkpoint.tar Resolving github.com (github.com)... 140.82.114.3 Connecting to github.com (github.com)|140.82.114.3|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://github-releases.githubusercontent.com/374891081/e17b9d00-dcb8-11eb-8c4f-1412bcea78a6?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210704%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210704T063956Z&X-Amz-Expires=300&X-Amz-Signature=b6d431c65405e894ddc994061c5fe8fe87db4e71e702513aec01f398a1004825&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=374891081&response-content-disposition=attachment%3B%20filename%3Darcface_checkpoint.tar&response-content-type=application%2Foctet-stream [following] --2021-07-04 06:39:56-- https://github-releases.githubusercontent.com/374891081/e17b9d00-dcb8-11eb-8c4f-1412bcea78a6?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210704%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210704T063956Z&X-Amz-Expires=300&X-Amz-Signature=b6d431c65405e894ddc994061c5fe8fe87db4e71e702513aec01f398a1004825&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=374891081&response-content-disposition=attachment%3B%20filename%3Darcface_checkpoint.tar&response-content-type=application%2Foctet-stream Resolving github-releases.githubusercontent.com (github-releases.githubusercontent.com)... 185.199.108.154, 185.199.109.154, 185.199.110.154, ... Connecting to github-releases.githubusercontent.com (github-releases.githubusercontent.com)|185.199.108.154|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 766871429 (731M) [application/octet-stream] Saving to: ‘./arcface_model/arcface_checkpoint.tar’ arcface_checkpoint. 100%[===================>] 731.34M 64.4MB/s in 11s 2021-07-04 06:40:07 (68.4 MB/s) - ‘./arcface_model/arcface_checkpoint.tar’ saved [766871429/766871429] --2021-07-04 06:40:07-- https://github.com/neuralchen/SimSwap/releases/download/1.0/checkpoints.zip Resolving github.com (github.com)... 140.82.113.3 Connecting to github.com (github.com)|140.82.113.3|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://github-releases.githubusercontent.com/374891081/a8dac400-dcb6-11eb-933f-977cd7f5f554?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210704%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210704T063831Z&X-Amz-Expires=300&X-Amz-Signature=3fd2850d03abb9301bf5ba5969d82eb73cb0b940b85e45de2e1e34f1ba2eaf09&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=374891081&response-content-disposition=attachment%3B%20filename%3Dcheckpoints.zip&response-content-type=application%2Foctet-stream [following] --2021-07-04 06:40:07-- https://github-releases.githubusercontent.com/374891081/a8dac400-dcb6-11eb-933f-977cd7f5f554?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210704%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210704T063831Z&X-Amz-Expires=300&X-Amz-Signature=3fd2850d03abb9301bf5ba5969d82eb73cb0b940b85e45de2e1e34f1ba2eaf09&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=374891081&response-content-disposition=attachment%3B%20filename%3Dcheckpoints.zip&response-content-type=application%2Foctet-stream Resolving github-releases.githubusercontent.com (github-releases.githubusercontent.com)... 185.199.108.154, 185.199.109.154, 185.199.110.154, ... Connecting to github-releases.githubusercontent.com (github-releases.githubusercontent.com)|185.199.108.154|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 256461775 (245M) [application/octet-stream] Saving to: ‘checkpoints.zip’ checkpoints.zip 100%[===================>] 244.58M 219MB/s in 1.1s 2021-07-04 06:40:08 (219 MB/s) - ‘checkpoints.zip’ saved [256461775/256461775] Archive: ./checkpoints.zip creating: ./checkpoints/people/ inflating: ./checkpoints/people/iter.txt inflating: ./checkpoints/people/latest_net_D1.pth inflating: ./checkpoints/people/latest_net_D2.pth inflating: ./checkpoints/people/latest_net_G.pth inflating: ./checkpoints/people/loss_log.txt inflating: ./checkpoints/people/opt.txt creating: ./checkpoints/people/web/ creating: ./checkpoints/people/web/images/
In [7]:
!wget --no-check-certificate "https://sh23tw.dm.files.1drv.com/y4mmGiIkNVigkSwOKDcV3nwMJulRGhbtHdkheehR5TArc52UjudUYNXAEvKCii2O5LAmzGCGK6IfleocxuDeoKxDZkNzDRSt4ZUlEt8GlSOpCXAFEkBwaZimtWGDRbpIGpb_pz9Nq5jATBQpezBS6G_UtspWTkgrXHHxhviV2nWy8APPx134zOZrUIbkSF6xnsqzs3uZ_SEX_m9Rey0ykpx9w" -O antelope.zip
!unzip ./antelope.zip -d ./insightface_func/models/--2021-07-04 06:40:11-- https://sh23tw.dm.files.1drv.com/y4mmGiIkNVigkSwOKDcV3nwMJulRGhbtHdkheehR5TArc52UjudUYNXAEvKCii2O5LAmzGCGK6IfleocxuDeoKxDZkNzDRSt4ZUlEt8GlSOpCXAFEkBwaZimtWGDRbpIGpb_pz9Nq5jATBQpezBS6G_UtspWTkgrXHHxhviV2nWy8APPx134zOZrUIbkSF6xnsqzs3uZ_SEX_m9Rey0ykpx9w Resolving sh23tw.dm.files.1drv.com (sh23tw.dm.files.1drv.com)... 13.107.42.12 Connecting to sh23tw.dm.files.1drv.com (sh23tw.dm.files.1drv.com)|13.107.42.12|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 248024513 (237M) [application/zip] Saving to: ‘antelope.zip’ antelope.zip 100%[===================>] 236.53M 52.4MB/s in 4.7s 2021-07-04 06:40:16 (49.9 MB/s) - ‘antelope.zip’ saved [248024513/248024513] Archive: ./antelope.zip creating: ./insightface_func/models/antelope/ inflating: ./insightface_func/models/antelope/glintr100.onnx inflating: ./insightface_func/models/antelope/scrfd_10g_bnkps.onnx
In [8]:
import cv2
import torch
import fractions
import numpy as np
from PIL import Image
import torch.nn.functional as F
from torchvision import transforms
from models.models import create_model
from options.test_options import TestOptions
from insightface_func.face_detect_crop_multi import Face_detect_crop
from util.videoswap_multispecific import video_swap
import os
import globImageio: 'ffmpeg-linux64-v3.3.1' was not found on your computer; downloading it now. Try 1. Download from https://github.com/imageio/imageio-binaries/raw/master/ffmpeg/ffmpeg-linux64-v3.3.1 (43.8 MB) Downloading: 8192/45929032 bytes (0.0%)3432448/45929032 bytes (7.5%)7036928/45929032 bytes (15.3%)10641408/45929032 bytes (23.2%)14278656/45929032 bytes (31.1%)18104320/45929032 bytes (39.4%)21954560/45929032 bytes (47.8%)25780224/45929032 bytes (56.1%)29736960/45929032 bytes (64.7%)33488896/45929032 bytes (72.9%)37093376/45929032 bytes (80.8%)40689664/45929032 bytes (88.6%)44392448/45929032 bytes (96.7%)45929032/45929032 bytes (100.0%) Done File saved as /root/.imageio/ffmpeg/ffmpeg-linux64-v3.3.1.
In [9]:
def lcm(a, b): return abs(a * b) / fractions.gcd(a, b) if a and b else 0
transformer = transforms.Compose([
transforms.ToTensor(),
#transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
transformer_Arcface = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
In [10]:
!ls ./checkpointspeople
In [11]:
opt = TestOptions()
opt.initialize()
opt.parser.add_argument('-f') ## dummy arg to avoid bug
opt = opt.parse()
opt.multisepcific_dir = './demo_file/multispecific' ## or replace it with folder from your own google drive
## and remember to follow the dir structure in usage.md
opt.video_path = './demo_file/multi_people_1080p.mp4' ## or replace it with video from your own google drive
opt.output_path = './output/multi_test_multispecific.mp4'
opt.temp_path = './tmp'
opt.Arc_path = './arcface_model/arcface_checkpoint.tar'
opt.name = 'people'
opt.isTrain = False
opt.use_mask = True ## new feature up-to-date
crop_size = opt.crop_size
------------ Options ------------- Arc_path: models/BEST_checkpoint.tar aspect_ratio: 1.0 batchSize: 8 checkpoints_dir: ./checkpoints cluster_path: features_clustered_010.npy data_type: 32 dataroot: ./datasets/cityscapes/ display_winsize: 512 engine: None export_onnx: None f: /root/.local/share/jupyter/runtime/kernel-19937219-895d-4d02-9a72-5cfa0e889adf.json feat_num: 3 fineSize: 512 fp16: False gpu_ids: [0] how_many: 50 id_thres: 0.03 image_size: 224 input_nc: 3 instance_feat: False isTrain: False label_feat: False label_nc: 0 latent_size: 512 loadSize: 1024 load_features: False local_rank: 0 max_dataset_size: inf model: pix2pixHD multisepcific_dir: ./demo_file/multispecific nThreads: 2 n_blocks_global: 6 n_blocks_local: 3 n_clusters: 10 n_downsample_E: 4 n_downsample_global: 3 n_local_enhancers: 1 name: people nef: 16 netG: global ngf: 64 niter_fix_global: 0 no_flip: False no_instance: False no_simswaplogo: False norm: batch norm_G: spectralspadesyncbatch3x3 ntest: inf onnx: None output_nc: 3 output_path: ./output/ phase: test pic_a_path: ./crop_224/gdg.jpg pic_b_path: ./crop_224/zrf.jpg pic_specific_path: ./crop_224/zrf.jpg resize_or_crop: scale_width results_dir: ./results/ semantic_nc: 3 serial_batches: False temp_path: ./temp_results tf_log: False use_dropout: False use_encoded_image: False verbose: False video_path: ./demo_file/multi_people_1080p.mp4 which_epoch: latest -------------- End ----------------
In [12]:
pic_specific = opt.pic_specific_path
crop_size = opt.crop_size
multisepcific_dir = opt.multisepcific_dir
torch.nn.Module.dump_patches = True
model = create_model(opt)
model.eval()
app = Face_detect_crop(name='antelope', root='./insightface_func/models')
app.prepare(ctx_id= 0, det_thresh=0.6, det_size=(640,640))
# The specific person to be swapped(source)
source_specific_id_nonorm_list = []
source_path = os.path.join(multisepcific_dir,'SRC_*')
source_specific_images_path = sorted(glob.glob(source_path))
with torch.no_grad():
for source_specific_image_path in source_specific_images_path:
specific_person_whole = cv2.imread(source_specific_image_path)
specific_person_align_crop, _ = app.get(specific_person_whole,crop_size)
specific_person_align_crop_pil = Image.fromarray(cv2.cvtColor(specific_person_align_crop[0],cv2.COLOR_BGR2RGB))
specific_person = transformer_Arcface(specific_person_align_crop_pil)
specific_person = specific_person.view(-1, specific_person.shape[0], specific_person.shape[1], specific_person.shape [2])
# convert numpy to tensor
specific_person = specific_person.cuda()
#create latent id
specific_person_downsample = F.interpolate(specific_person, size=(112,112))
specific_person_id_nonorm = model.netArc(specific_person_downsample)
source_specific_id_nonorm_list.append(specific_person_id_nonorm.clone())
# The person who provides id information (list)
target_id_norm_list = []
target_path = os.path.join(multisepcific_dir,'DST_*')
target_images_path = sorted(glob.glob(target_path))
for target_image_path in target_images_path:
img_a_whole = cv2.imread(target_image_path)
img_a_align_crop, _ = app.get(img_a_whole,crop_size)
img_a_align_crop_pil = Image.fromarray(cv2.cvtColor(img_a_align_crop[0],cv2.COLOR_BGR2RGB))
img_a = transformer_Arcface(img_a_align_crop_pil)
img_id = img_a.view(-1, img_a.shape[0], img_a.shape[1], img_a.shape[2])
# convert numpy to tensor
img_id = img_id.cuda()
#create latent id
img_id_downsample = F.interpolate(img_id, size=(112,112))
latend_id = model.netArc(img_id_downsample)
latend_id = F.normalize(latend_id, p=2, dim=1)
target_id_norm_list.append(latend_id.clone())
assert len(target_id_norm_list) == len(source_specific_id_nonorm_list), "The number of images in source and target directory must be same !!!"
video_swap(opt.video_path, target_id_norm_list,source_specific_id_nonorm_list, opt.id_thres, \
model, app, opt.output_path,temp_results_dir=opt.temp_path,no_simswaplogo=opt.no_simswaplogo,use_mask=opt.use_mask)input mean and std: 127.5 127.5 find model: ./insightface_func/models/antelope/glintr100.onnx recognition find model: ./insightface_func/models/antelope/scrfd_10g_bnkps.onnx detection set det-size: (640, 640)
0%| | 0/594 [00:00<?, ?it/s]
(142, 366, 4)
100%|██████████| 594/594 [08:28<00:00, 1.17it/s]
[MoviePy] >>>> Building video ./output/multi_test_multispecific.mp4 [MoviePy] Writing audio in multi_test_multispecificTEMP_MPY_wvf_snd.mp3
100%|██████████| 438/438 [00:00<00:00, 832.16it/s]
[MoviePy] Done. [MoviePy] Writing video ./output/multi_test_multispecific.mp4
100%|██████████| 595/595 [00:53<00:00, 11.11it/s]
[MoviePy] Done. [MoviePy] >>>> Video ready: ./output/multi_test_multispecific.mp4
In [ ]: