27 KiB
27 KiB
In [1]:
## make sure you are using a runtime with GPU
## you can check at Runtime/Change runtime type in the top bar.
!nvidia-smiMon Jun 21 02:13:20 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.27 Driver Version: 460.32.03 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |
| N/A 45C P8 10W / 70W | 0MiB / 15109MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
In [2]:
!git clone https://github.com/neuralchen/SimSwap
!cd SimSwap && git pullCloning into 'SimSwap'... remote: Enumerating objects: 362, done.[K remote: Counting objects: 100% (362/362), done.[K remote: Compressing objects: 100% (281/281), done.[K remote: Total 362 (delta 149), reused 272 (delta 67), pack-reused 0[K Receiving objects: 100% (362/362), 101.31 MiB | 32.47 MiB/s, done. Resolving deltas: 100% (149/149), done. Already up to date.
In [3]:
!pip install insightface==0.2.1 onnxruntime moviepy
!pip install googledrivedownloader
!pip install imageio==2.4.1Collecting insightface==0.2.1 Downloading https://files.pythonhosted.org/packages/ee/1e/6395bbe0db665f187c8e49266cda54fcf661f182192370d409423e4943e4/insightface-0.2.1-py2.py3-none-any.whl Collecting onnxruntime [?25l Downloading https://files.pythonhosted.org/packages/f9/76/3d0f8bb2776961c7335693df06eccf8d099e48fa6fb552c7546867192603/onnxruntime-1.8.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.5MB) [K |████████████████████████████████| 4.5MB 10.2MB/s [?25hRequirement already satisfied: moviepy in /usr/local/lib/python3.7/dist-packages (0.2.3.5) Collecting onnx [?25l Downloading https://files.pythonhosted.org/packages/3f/9b/54c950d3256e27f970a83cd0504efb183a24312702deed0179453316dbd0/onnx-1.9.0-cp37-cp37m-manylinux2010_x86_64.whl (12.2MB) [K |████████████████████████████████| 12.2MB 51.4MB/s [?25hRequirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (3.2.2) Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (4.41.1) Requirement already satisfied: Pillow in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (7.1.2) Requirement already satisfied: scikit-image in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (0.16.2) Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (2.23.0) Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (0.22.2.post1) Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (4.1.2.30) Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (1.19.5) Requirement already satisfied: easydict in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (1.9) Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (1.4.1) Requirement already satisfied: flatbuffers in /usr/local/lib/python3.7/dist-packages (from onnxruntime) (1.12) Requirement already satisfied: protobuf in /usr/local/lib/python3.7/dist-packages (from onnxruntime) (3.12.4) Requirement already satisfied: decorator<5.0,>=4.0.2 in /usr/local/lib/python3.7/dist-packages (from moviepy) (4.4.2) Requirement already satisfied: imageio<3.0,>=2.1.2 in /usr/local/lib/python3.7/dist-packages (from moviepy) (2.4.1) Requirement already satisfied: typing-extensions>=3.6.2.1 in /usr/local/lib/python3.7/dist-packages (from onnx->insightface==0.2.1) (3.7.4.3) Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from onnx->insightface==0.2.1) (1.15.0) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (1.3.1) Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (2.8.1) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (0.10.0) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (2.4.7) Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->insightface==0.2.1) (2.5.1) Requirement already satisfied: PyWavelets>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->insightface==0.2.1) (1.1.1) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (2.10) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (2021.5.30) Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (3.0.4) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (1.24.3) Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->insightface==0.2.1) (1.0.1) Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from protobuf->onnxruntime) (57.0.0) Installing collected packages: onnx, insightface, onnxruntime Successfully installed insightface-0.2.1 onnx-1.9.0 onnxruntime-1.8.0 Requirement already satisfied: googledrivedownloader in /usr/local/lib/python3.7/dist-packages (0.4) Requirement already satisfied: imageio==2.4.1 in /usr/local/lib/python3.7/dist-packages (2.4.1) Requirement already satisfied: pillow in /usr/local/lib/python3.7/dist-packages (from imageio==2.4.1) (7.1.2) Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from imageio==2.4.1) (1.19.5)
In [4]:
import os
os.chdir("SimSwap")
!lscrop_224 models test_one_image.py data options test_video_swapmulti.py demo_file output test_video_swapsingle.py doc README.md test_wholeimage_swapmulti.py insightface_func 'SimSwap colab.ipynb' test_wholeimage_swapsingle.py LICENSE simswaplogo util
In [5]:
from google_drive_downloader import GoogleDriveDownloader
### it seems that google drive link may not be permenant, you can find this ID from our open url.
# GoogleDriveDownloader.download_file_from_google_drive(file_id='1TLNdIufzwesDbyr_nVTR7Zrx9oRHLM_N',
# dest_path='./arcface_model/arcface_checkpoint.tar')
# GoogleDriveDownloader.download_file_from_google_drive(file_id='1PXkRiBUYbu1xWpQyDEJvGKeqqUFthJcI',
# dest_path='./checkpoints.zip')
!wget -P ./arcface_model https://github.com/neuralchen/SimSwap/releases/download/1.0/arcface_checkpoint.tar
!wget https://github.com/neuralchen/SimSwap/releases/download/1.0/checkpoints.zip
!unzip ./checkpoints.zip -d ./checkpoints
!wget -P ./parsing_model/checkpoint https://github.com/neuralchen/SimSwap/releases/download/1.0/79999_iter.pthDownloading 1TLNdIufzwesDbyr_nVTR7Zrx9oRHLM_N into ./arcface_model/arcface_checkpoint.tar... Done. Downloading 1PXkRiBUYbu1xWpQyDEJvGKeqqUFthJcI into ./checkpoints.zip... Done. Archive: ./checkpoints.zip creating: ./checkpoints/people/ inflating: ./checkpoints/people/iter.txt inflating: ./checkpoints/people/latest_net_D1.pth inflating: ./checkpoints/people/latest_net_D2.pth inflating: ./checkpoints/people/latest_net_G.pth inflating: ./checkpoints/people/loss_log.txt inflating: ./checkpoints/people/opt.txt creating: ./checkpoints/people/web/ creating: ./checkpoints/people/web/images/
In [6]:
## You can upload filed manually
# from google.colab import drive
# drive.mount('/content/gdrive')
### Now onedrive file can be downloaded in Colab directly!
### If the link blow is not permanent, you can just download it from the
### open url(can be found at [our repo]/doc/guidance/preparation.md) and copy the assigned download link here.
### many thanks to woctezuma for this very useful help
!wget --no-check-certificate "https://sh23tw.dm.files.1drv.com/y4mmGiIkNVigkSwOKDcV3nwMJulRGhbtHdkheehR5TArc52UjudUYNXAEvKCii2O5LAmzGCGK6IfleocxuDeoKxDZkNzDRSt4ZUlEt8GlSOpCXAFEkBwaZimtWGDRbpIGpb_pz9Nq5jATBQpezBS6G_UtspWTkgrXHHxhviV2nWy8APPx134zOZrUIbkSF6xnsqzs3uZ_SEX_m9Rey0ykpx9w" -O antelope.zip
!unzip ./antelope.zip -d ./insightface_func/models/
--2021-06-21 02:14:17-- https://sh23tw.dm.files.1drv.com/y4mmGiIkNVigkSwOKDcV3nwMJulRGhbtHdkheehR5TArc52UjudUYNXAEvKCii2O5LAmzGCGK6IfleocxuDeoKxDZkNzDRSt4ZUlEt8GlSOpCXAFEkBwaZimtWGDRbpIGpb_pz9Nq5jATBQpezBS6G_UtspWTkgrXHHxhviV2nWy8APPx134zOZrUIbkSF6xnsqzs3uZ_SEX_m9Rey0ykpx9w Resolving sh23tw.dm.files.1drv.com (sh23tw.dm.files.1drv.com)... 13.107.42.12 Connecting to sh23tw.dm.files.1drv.com (sh23tw.dm.files.1drv.com)|13.107.42.12|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 248024513 (237M) [application/zip] Saving to: ‘antelope.zip’ antelope.zip 100%[===================>] 236.53M 6.16MB/s in 31s 2021-06-21 02:14:48 (7.66 MB/s) - ‘antelope.zip’ saved [248024513/248024513] Archive: ./antelope.zip creating: ./insightface_func/models/antelope/ inflating: ./insightface_func/models/antelope/glintr100.onnx inflating: ./insightface_func/models/antelope/scrfd_10g_bnkps.onnx
In [7]:
import cv2
import torch
import fractions
import numpy as np
from PIL import Image
import torch.nn.functional as F
from torchvision import transforms
from models.models import create_model
from options.test_options import TestOptions
from insightface_func.face_detect_crop_multi import Face_detect_crop
from util.videoswap import video_swap
from util.add_watermark import watermark_imageImageio: 'ffmpeg-linux64-v3.3.1' was not found on your computer; downloading it now. Try 1. Download from https://github.com/imageio/imageio-binaries/raw/master/ffmpeg/ffmpeg-linux64-v3.3.1 (43.8 MB) Downloading: 8192/45929032 bytes (0.0%)1286144/45929032 bytes (2.8%)3653632/45929032 bytes (8.0%)7479296/45929032 bytes (16.3%)11526144/45929032 bytes (25.1%)15171584/45929032 bytes (33.0%)18997248/45929032 bytes (41.4%)22724608/45929032 bytes (49.5%)26673152/45929032 bytes (58.1%)30728192/45929032 bytes (66.9%)34725888/45929032 bytes (75.6%)38879232/45929032 bytes (84.7%)42680320/45929032 bytes (92.9%)45929032/45929032 bytes (100.0%) Done File saved as /root/.imageio/ffmpeg/ffmpeg-linux64-v3.3.1.
In [8]:
transformer = transforms.Compose([
transforms.ToTensor(),
#transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
transformer_Arcface = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
detransformer = transforms.Compose([
transforms.Normalize([0, 0, 0], [1/0.229, 1/0.224, 1/0.225]),
transforms.Normalize([-0.485, -0.456, -0.406], [1, 1, 1])
])In [9]:
opt = TestOptions()
opt.initialize()
opt.parser.add_argument('-f') ## dummy arg to avoid bug
opt = opt.parse()
opt.pic_a_path = './demo_file/Iron_man.jpg' ## or replace it with image from your own google drive
opt.video_path = './demo_file/multi_people_1080p.mp4' ## or replace it with video from your own google drive
opt.output_path = './output/demo.mp4'
opt.temp_path = './tmp'
opt.Arc_path = './arcface_model/arcface_checkpoint.tar'
opt.isTrain = False
opt.use_mask = True ## new feature up-to-date
crop_size = 224
torch.nn.Module.dump_patches = True
model = create_model(opt)
model.eval()
app = Face_detect_crop(name='antelope', root='./insightface_func/models')
app.prepare(ctx_id= 0, det_thresh=0.6, det_size=(640,640))
with torch.no_grad():
pic_a = opt.pic_a_path
# img_a = Image.open(pic_a).convert('RGB')
img_a_whole = cv2.imread(pic_a)
img_a_align_crop, _ = app.get(img_a_whole,crop_size)
img_a_align_crop_pil = Image.fromarray(cv2.cvtColor(img_a_align_crop[0],cv2.COLOR_BGR2RGB))
img_a = transformer_Arcface(img_a_align_crop_pil)
img_id = img_a.view(-1, img_a.shape[0], img_a.shape[1], img_a.shape[2])
# convert numpy to tensor
img_id = img_id.cuda()
#create latent id
img_id_downsample = F.interpolate(img_id, scale_factor=0.5)
latend_id = model.netArc(img_id_downsample)
latend_id = latend_id.detach().to('cpu')
latend_id = latend_id/np.linalg.norm(latend_id,axis=1,keepdims=True)
latend_id = latend_id.to('cuda')
video_swap(opt.video_path, latend_id, model, app, opt.output_path, temp_results_dir=opt.temp_path, use_mask=opt.use_mask)------------ Options ------------- Arc_path: models/BEST_checkpoint.tar aspect_ratio: 1.0 batchSize: 8 checkpoints_dir: ./checkpoints cluster_path: features_clustered_010.npy data_type: 32 dataroot: ./datasets/cityscapes/ display_winsize: 512 engine: None export_onnx: None f: /root/.local/share/jupyter/runtime/kernel-6d955151-4911-464a-824d-f0806d8071f6.json feat_num: 3 fineSize: 512 fp16: False gpu_ids: [0] how_many: 50 image_size: 224 input_nc: 3 instance_feat: False isTrain: False label_feat: False label_nc: 0 latent_size: 512 loadSize: 1024 load_features: False local_rank: 0 max_dataset_size: inf model: pix2pixHD nThreads: 2 n_blocks_global: 6 n_blocks_local: 3 n_clusters: 10 n_downsample_E: 4 n_downsample_global: 3 n_local_enhancers: 1 name: people nef: 16 netG: global ngf: 64 niter_fix_global: 0 no_flip: False no_instance: False norm: batch norm_G: spectralspadesyncbatch3x3 ntest: inf onnx: None output_nc: 3 output_path: ./output/ phase: test pic_a_path: ./crop_224/gdg.jpg pic_b_path: ./crop_224/zrf.jpg resize_or_crop: scale_width results_dir: ./results/ semantic_nc: 3 serial_batches: False temp_path: ./temp_results tf_log: False use_dropout: False use_encoded_image: False verbose: False video_path: ./demo_file/multi_people_1080p.mp4 which_epoch: latest -------------- End ---------------- input mean and std: 127.5 127.5 find model: ./insightface_func/models/antelope/glintr100.onnx recognition find model: ./insightface_func/models/antelope/scrfd_10g_bnkps.onnx detection set det-size: (640, 640)
0%| | 0/594 [00:00<?, ?it/s]
(142, 366, 4)
100%|██████████| 594/594 [08:45<00:00, 1.13it/s]
[MoviePy] >>>> Building video ./output/demo.mp4 [MoviePy] Writing audio in demoTEMP_MPY_wvf_snd.mp3
100%|██████████| 438/438 [00:00<00:00, 877.18it/s]
[MoviePy] Done. [MoviePy] Writing video ./output/demo.mp4
100%|██████████| 595/595 [00:53<00:00, 11.15it/s]
[MoviePy] Done. [MoviePy] >>>> Video ready: ./output/demo.mp4
In [ ]: