CalvinBackup/neuralchen-SimSwap
1
0 Fork 0
Code Issues 288 Pull Requests 16 Actions Packages Projects Releases 2 Wiki Activity

Merge branch 'main' of https://github.com/neuralchen/SimSwap into main

Browse Source
This commit is contained in:
NNNNAI committed 2021-06-21 11:04:46 +08:00
2 parents 9d186aec16 0b2cde8a52
commit 19cb97aa5c
2 files changed
+577 -1
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+4
View File
@@ -73,6 +73,10 @@ Training: **coming soon**
[[Online Video]](https://www.bilibili.com/video/BV12v411p7j5/)
## User case
If you have some interesting results after using our project and are willing to share, you can contact us by email or share directly on the issue. Later, we may make a separate section to show these results, which should be cool.
At the same time, if you have suggestions for our project, please feel free to ask questions in the issue, or contact us directly via email: [email1](chenxuanhongzju@outlook.com), [email2](nicklau26@foxmail.com), [email3](ziangliu824@gmail.com). (All three can be contacted, just choose any one)
SimSwap colab.ipynb
+573 -1
View File
@@ -1 +1,573 @@
{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"SimSwap colab.ipynb","provenance":[],"collapsed_sections":[],"authorship_tag":"ABX9TyNOd2gHqzvEERlg8scYqsHg"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"7_gtFoV8BuRx"},"source":["This is a simple example of SimSwap on processing video with multiple faces. You can change the codes for inference based on our other scripts for image or single face swapping.\n","\n","Code path: https://github.com/neuralchen/SimSwap\n","\n","Paper path: https://arxiv.org/pdf/2106.06340v1.pdf or https://dl.acm.org/doi/10.1145/3394171.3413630"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"0Y1RfpzsCAl9","executionInfo":{"status":"ok","timestamp":1624195942908,"user_tz":-480,"elapsed":1168,"user":{"displayName":"José Lampreia","photoUrl":"","userId":"16015278604201270582"}},"outputId":"2d4d3c09-2baf-45f2-af4a-51347a0c5943"},"source":["## make sure you are using a runtime with GPU\n","## you can check at Runtime/Change runtime type in the top bar.\n","!nvidia-smi"],"execution_count":1,"outputs":[{"output_type":"stream","text":["Sun Jun 20 13:32:21 2021 \n","+-----------------------------------------------------------------------------+\n","| NVIDIA-SMI 465.27 Driver Version: 460.32.03 CUDA Version: 11.2 |\n","|-------------------------------+----------------------+----------------------+\n","| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n","| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n","| | | MIG M. |\n","|===============================+======================+======================|\n","| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |\n","| N/A 63C P8 11W / 70W | 0MiB / 15109MiB | 0% Default |\n","| | | N/A |\n","+-------------------------------+----------------------+----------------------+\n"," \n","+-----------------------------------------------------------------------------+\n","| Processes: |\n","| GPU GI CI PID Type Process name GPU Memory |\n","| ID ID Usage |\n","|=============================================================================|\n","| No running processes found |\n","+-----------------------------------------------------------------------------+\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"0Qzzx2UpDkqw"},"source":["## Installation\n","\n","All file changes made by this notebook are temporary. \n","You can try to mount your own google drive to store files if you want.\n"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"VA_4CeWZCHLP","executionInfo":{"status":"ok","timestamp":1624195953150,"user_tz":-480,"elapsed":9654,"user":{"displayName":"José Lampreia","photoUrl":"","userId":"16015278604201270582"}},"outputId":"d8994b2f-859e-4be2-ae3b-1bfcd8470054"},"source":["!git clone https://github.com/neuralchen/SimSwap\n","!cd SimSwap && git pull"],"execution_count":2,"outputs":[{"output_type":"stream","text":["Cloning into 'SimSwap'...\n","remote: Enumerating objects: 329, done.\u001b[K\n","remote: Counting objects: 100% (329/329), done.\u001b[K\n","remote: Compressing objects: 100% (251/251), done.\u001b[K\n","remote: Total 329 (delta 130), reused 261 (delta 64), pack-reused 0\u001b[K\n","Receiving objects: 100% (329/329), 101.29 MiB | 48.72 MiB/s, done.\n","Resolving deltas: 100% (130/130), done.\n","Already up to date.\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"Y5K4au_UCkKn","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1624195964743,"user_tz":-480,"elapsed":11600,"user":{"displayName":"José Lampreia","photoUrl":"","userId":"16015278604201270582"}},"outputId":"5a711b0f-c5f0-48c1-bafd-41d3b35cd9c6"},"source":["!pip install insightface==0.2.1 onnxruntime moviepy\n","!pip install googledrivedownloader\n","!pip install imageio==2.4.1"],"execution_count":3,"outputs":[{"output_type":"stream","text":["Collecting insightface==0.2.1\n"," Downloading https://files.pythonhosted.org/packages/ee/1e/6395bbe0db665f187c8e49266cda54fcf661f182192370d409423e4943e4/insightface-0.2.1-py2.py3-none-any.whl\n","Collecting onnxruntime\n","\u001b[?25l Downloading https://files.pythonhosted.org/packages/f9/76/3d0f8bb2776961c7335693df06eccf8d099e48fa6fb552c7546867192603/onnxruntime-1.8.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whlLine truncated
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "SimSwap colab.ipynb",
"provenance": [],
"collapsed_sections": []
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "7_gtFoV8BuRx"
},
"source": [
"This is a simple example of SimSwap on processing video with multiple faces. You can change the codes for inference based on our other scripts for image or single face swapping.\n",
"\n",
"Code path: https://github.com/neuralchen/SimSwap\n",
"\n",
"Paper path: https://arxiv.org/pdf/2106.06340v1.pdf or https://dl.acm.org/doi/10.1145/3394171.3413630"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "0Y1RfpzsCAl9",
"outputId": "a39470a0-9689-409d-a0a4-e2afd5d3b5dd"
},
"source": [
"## make sure you are using a runtime with GPU\n",
"## you can check at Runtime/Change runtime type in the top bar.\n",
"!nvidia-smi"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"text": [
"Mon Jun 21 02:13:20 2021 \n",
"+-----------------------------------------------------------------------------+\n",
"| NVIDIA-SMI 465.27 Driver Version: 460.32.03 CUDA Version: 11.2 |\n",
"|-------------------------------+----------------------+----------------------+\n",
"| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
"| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n",
"| | | MIG M. |\n",
"|===============================+======================+======================|\n",
"| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |\n",
"| N/A 45C P8 10W / 70W | 0MiB / 15109MiB | 0% Default |\n",
"| | | N/A |\n",
"+-------------------------------+----------------------+----------------------+\n",
" \n",
"+-----------------------------------------------------------------------------+\n",
"| Processes: |\n",
"| GPU GI CI PID Type Process name GPU Memory |\n",
"| ID ID Usage |\n",
"|=============================================================================|\n",
"| No running processes found |\n",
"+-----------------------------------------------------------------------------+\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0Qzzx2UpDkqw"
},
"source": [
"## Installation\n",
"\n",
"All file changes made by this notebook are temporary. \n",
"You can try to mount your own google drive to store files if you want.\n"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "VA_4CeWZCHLP",
"outputId": "4b0f176f-87e7-4772-8b47-c2098d8f3bf6"
},
"source": [
"!git clone https://github.com/neuralchen/SimSwap\n",
"!cd SimSwap && git pull"
],
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"text": [
"Cloning into 'SimSwap'...\n",
"remote: Enumerating objects: 362, done.\u001b[K\n",
"remote: Counting objects: 100% (362/362), done.\u001b[K\n",
"remote: Compressing objects: 100% (281/281), done.\u001b[K\n",
"remote: Total 362 (delta 149), reused 272 (delta 67), pack-reused 0\u001b[K\n",
"Receiving objects: 100% (362/362), 101.31 MiB | 32.47 MiB/s, done.\n",
"Resolving deltas: 100% (149/149), done.\n",
"Already up to date.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "Y5K4au_UCkKn",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "9691a7a4-192e-4ec2-c3c1-1f2c933d7b6a"
},
"source": [
"!pip install insightface==0.2.1 onnxruntime moviepy\n",
"!pip install googledrivedownloader\n",
"!pip install imageio==2.4.1"
],
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"text": [
"Collecting insightface==0.2.1\n",
" Downloading https://files.pythonhosted.org/packages/ee/1e/6395bbe0db665f187c8e49266cda54fcf661f182192370d409423e4943e4/insightface-0.2.1-py2.py3-none-any.whl\n",
"Collecting onnxruntime\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/f9/76/3d0f8bb2776961c7335693df06eccf8d099e48fa6fb552c7546867192603/onnxruntime-1.8.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.5MB)\n",
"\u001b[K |████████████████████████████████| 4.5MB 10.2MB/s \n",
"\u001b[?25hRequirement already satisfied: moviepy in /usr/local/lib/python3.7/dist-packages (0.2.3.5)\n",
"Collecting onnx\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/3f/9b/54c950d3256e27f970a83cd0504efb183a24312702deed0179453316dbd0/onnx-1.9.0-cp37-cp37m-manylinux2010_x86_64.whl (12.2MB)\n",
"\u001b[K |████████████████████████████████| 12.2MB 51.4MB/s \n",
"\u001b[?25hRequirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (3.2.2)\n",
"Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (4.41.1)\n",
"Requirement already satisfied: Pillow in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (7.1.2)\n",
"Requirement already satisfied: scikit-image in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (0.16.2)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (2.23.0)\n",
"Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (0.22.2.post1)\n",
"Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (4.1.2.30)\n",
"Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (1.19.5)\n",
"Requirement already satisfied: easydict in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (1.9)\n",
"Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from insightface==0.2.1) (1.4.1)\n",
"Requirement already satisfied: flatbuffers in /usr/local/lib/python3.7/dist-packages (from onnxruntime) (1.12)\n",
"Requirement already satisfied: protobuf in /usr/local/lib/python3.7/dist-packages (from onnxruntime) (3.12.4)\n",
"Requirement already satisfied: decorator<5.0,>=4.0.2 in /usr/local/lib/python3.7/dist-packages (from moviepy) (4.4.2)\n",
"Requirement already satisfied: imageio<3.0,>=2.1.2 in /usr/local/lib/python3.7/dist-packages (from moviepy) (2.4.1)\n",
"Requirement already satisfied: typing-extensions>=3.6.2.1 in /usr/local/lib/python3.7/dist-packages (from onnx->insightface==0.2.1) (3.7.4.3)\n",
"Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from onnx->insightface==0.2.1) (1.15.0)\n",
"Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (1.3.1)\n",
"Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (2.8.1)\n",
"Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (0.10.0)\n",
"Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->insightface==0.2.1) (2.4.7)\n",
"Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->insightface==0.2.1) (2.5.1)\n",
"Requirement already satisfied: PyWavelets>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->insightface==0.2.1) (1.1.1)\n",
"Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (2.10)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (2021.5.30)\n",
"Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (3.0.4)\n",
"Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->insightface==0.2.1) (1.24.3)\n",
"Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->insightface==0.2.1) (1.0.1)\n",
"Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from protobuf->onnxruntime) (57.0.0)\n",
"Installing collected packages: onnx, insightface, onnxruntime\n",
"Successfully installed insightface-0.2.1 onnx-1.9.0 onnxruntime-1.8.0\n",
"Requirement already satisfied: googledrivedownloader in /usr/local/lib/python3.7/dist-packages (0.4)\n",
"Requirement already satisfied: imageio==2.4.1 in /usr/local/lib/python3.7/dist-packages (2.4.1)\n",
"Requirement already satisfied: pillow in /usr/local/lib/python3.7/dist-packages (from imageio==2.4.1) (7.1.2)\n",
"Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from imageio==2.4.1) (1.19.5)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "gQ7ZoIbLFCye",
"outputId": "bb35e7e2-14b7-4f36-d62a-499ba041cf64"
},
"source": [
"import os\n",
"os.chdir(\"SimSwap\")\n",
"!ls"
],
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"text": [
" crop_224\t models\t\t test_one_image.py\n",
" data\t\t options\t\t test_video_swapmutil.py\n",
" demo_file\t output\t\t test_video_swapsingle.py\n",
" doc\t\t README.md\t\t test_wholeimage_swapmutil.py\n",
" insightface_func 'SimSwap colab.ipynb' test_wholeimage_swapsingle.py\n",
" LICENSE\t simswaplogo\t\t util\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "gLti1J0pEFjJ",
"outputId": "e93c3f98-01df-458e-b791-c32f7343e705"
},
"source": [
"from google_drive_downloader import GoogleDriveDownloader\n",
"\n",
"GoogleDriveDownloader.download_file_from_google_drive(file_id='1TLNdIufzwesDbyr_nVTR7Zrx9oRHLM_N',\n",
" dest_path='./arcface_model/arcface_checkpoint.tar')\n",
"GoogleDriveDownloader.download_file_from_google_drive(file_id='1PXkRiBUYbu1xWpQyDEJvGKeqqUFthJcI',\n",
" dest_path='./checkpoints.zip')\n",
"!unzip ./checkpoints.zip -d ./checkpoints"
],
"execution_count": 5,
"outputs": [
{
"output_type": "stream",
"text": [
"Downloading 1TLNdIufzwesDbyr_nVTR7Zrx9oRHLM_N into ./arcface_model/arcface_checkpoint.tar... Done.\n",
"Downloading 1PXkRiBUYbu1xWpQyDEJvGKeqqUFthJcI into ./checkpoints.zip... Done.\n",
"Archive: ./checkpoints.zip\n",
" creating: ./checkpoints/people/\n",
" inflating: ./checkpoints/people/iter.txt \n",
" inflating: ./checkpoints/people/latest_net_D1.pth \n",
" inflating: ./checkpoints/people/latest_net_D2.pth \n",
" inflating: ./checkpoints/people/latest_net_G.pth \n",
" inflating: ./checkpoints/people/loss_log.txt \n",
" inflating: ./checkpoints/people/opt.txt \n",
" creating: ./checkpoints/people/web/\n",
" creating: ./checkpoints/people/web/images/\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "aSRnK5V4HI-k",
"outputId": "e688746c-c33a-485c-808c-54a7370f0c53"
},
"source": [
"## You can upload filed manually\n",
"# from google.colab import drive\n",
"# drive.mount('/content/gdrive')\n",
"\n",
"### Now onedrive file can be downloaded in Colab directly!\n",
"### If the link blow is not permanent, you can just download it from the \n",
"### open url(can be found at [our repo]/doc/guidance/preparation.md) and copy the assigned download link here.\n",
"### many thanks to woctezuma for this very useful help\n",
"!wget --no-check-certificate \"https://sh23tw.dm.files.1drv.com/y4mmGiIkNVigkSwOKDcV3nwMJulRGhbtHdkheehR5TArc52UjudUYNXAEvKCii2O5LAmzGCGK6IfleocxuDeoKxDZkNzDRSt4ZUlEt8GlSOpCXAFEkBwaZimtWGDRbpIGpb_pz9Nq5jATBQpezBS6G_UtspWTkgrXHHxhviV2nWy8APPx134zOZrUIbkSF6xnsqzs3uZ_SEX_m9Rey0ykpx9w\" -O antelope.zip\n",
"!unzip ./antelope.zip -d ./insightface_func/models/\n"
],
"execution_count": 6,
"outputs": [
{
"output_type": "stream",
"text": [
"--2021-06-21 02:14:17-- https://sh23tw.dm.files.1drv.com/y4mmGiIkNVigkSwOKDcV3nwMJulRGhbtHdkheehR5TArc52UjudUYNXAEvKCii2O5LAmzGCGK6IfleocxuDeoKxDZkNzDRSt4ZUlEt8GlSOpCXAFEkBwaZimtWGDRbpIGpb_pz9Nq5jATBQpezBS6G_UtspWTkgrXHHxhviV2nWy8APPx134zOZrUIbkSF6xnsqzs3uZ_SEX_m9Rey0ykpx9w\n",
"Resolving sh23tw.dm.files.1drv.com (sh23tw.dm.files.1drv.com)... 13.107.42.12\n",
"Connecting to sh23tw.dm.files.1drv.com (sh23tw.dm.files.1drv.com)|13.107.42.12|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 248024513 (237M) [application/zip]\n",
"Saving to: ‘antelope.zip’\n",
"\n",
"antelope.zip 100%[===================>] 236.53M 6.16MB/s in 31s \n",
"\n",
"2021-06-21 02:14:48 (7.66 MB/s) - ‘antelope.zip’ saved [248024513/248024513]\n",
"\n",
"Archive: ./antelope.zip\n",
" creating: ./insightface_func/models/antelope/\n",
" inflating: ./insightface_func/models/antelope/glintr100.onnx \n",
" inflating: ./insightface_func/models/antelope/scrfd_10g_bnkps.onnx \n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BsGmIMxLVxyO"
},
"source": [
"## Inference"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "PfSsND36EMvn",
"outputId": "f28c98fd-4c6d-40fa-e3c7-99b606c7492a"
},
"source": [
"import cv2\n",
"import torch\n",
"import fractions\n",
"import numpy as np\n",
"from PIL import Image\n",
"import torch.nn.functional as F\n",
"from torchvision import transforms\n",
"from models.models import create_model\n",
"from options.test_options import TestOptions\n",
"from insightface_func.face_detect_crop_mutil import Face_detect_crop\n",
"from util.videoswap import video_swap\n",
"from util.add_watermark import watermark_image"
],
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"text": [
"Imageio: 'ffmpeg-linux64-v3.3.1' was not found on your computer; downloading it now.\n",
"Try 1. Download from https://github.com/imageio/imageio-binaries/raw/master/ffmpeg/ffmpeg-linux64-v3.3.1 (43.8 MB)\n",
"Downloading: 8192/45929032 bytes (0.0%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b1286144/45929032 bytes (2.8%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b3653632/45929032 bytes (8.0%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b7479296/45929032 bytes (16.3%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b11526144/45929032 bytes (25.1%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b15171584/45929032 bytes (33.0%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b18997248/45929032 bytes (41.4%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b22724608/45929032 bytes (49.5%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b26673152/45929032 bytes (58.1%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b30728192/45929032 bytes (66.9%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b34725888/45929032 bytes (75.6%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b38879232/45929032 bytes (84.7%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b42680320/45929032 bytes (92.9%)\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b45929032/45929032 bytes (100.0%)\n",
" Done\n",
"File saved as /root/.imageio/ffmpeg/ffmpeg-linux64-v3.3.1.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "rxSbZ2EDNDlf"
},
"source": [
"transformer = transforms.Compose([\n",
" transforms.ToTensor(),\n",
" #transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n",
" ])\n",
"\n",
"transformer_Arcface = transforms.Compose([\n",
" transforms.ToTensor(),\n",
" transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n",
" ])\n",
"\n",
"detransformer = transforms.Compose([\n",
" transforms.Normalize([0, 0, 0], [1/0.229, 1/0.224, 1/0.225]),\n",
" transforms.Normalize([-0.485, -0.456, -0.406], [1, 1, 1])\n",
" ])"
],
"execution_count": 8,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "wwJOwR9LNKRz",
"outputId": "bdc82f7b-21c4-403f-94d1-b92911698b4a"
},
"source": [
"opt = TestOptions()\n",
"opt.initialize()\n",
"opt.parser.add_argument('-f') ## dummy arg to avoid bug\n",
"opt = opt.parse()\n",
"opt.pic_a_path = './demo_file/Iron_man.jpg' ## or replace it with image from your own google drive\n",
"opt.video_path = './demo_file/mutil_people_1080p.mp4' ## or replace it with video from your own google drive\n",
"opt.output_path = './output/demo.mp4'\n",
"opt.temp_path = './tmp'\n",
"opt.Arc_path = './arcface_model/arcface_checkpoint.tar'\n",
"opt.isTrain = False\n",
"\n",
"crop_size = 224\n",
"\n",
"torch.nn.Module.dump_patches = True\n",
"model = create_model(opt)\n",
"model.eval()\n",
"\n",
"\n",
"app = Face_detect_crop(name='antelope', root='./insightface_func/models')\n",
"app.prepare(ctx_id= 0, det_thresh=0.6, det_size=(640,640))\n",
"\n",
"pic_a = opt.pic_a_path\n",
"# img_a = Image.open(pic_a).convert('RGB')\n",
"img_a_whole = cv2.imread(pic_a)\n",
"img_a_align_crop, _ = app.get(img_a_whole,crop_size)\n",
"img_a_align_crop_pil = Image.fromarray(cv2.cvtColor(img_a_align_crop[0],cv2.COLOR_BGR2RGB)) \n",
"img_a = transformer_Arcface(img_a_align_crop_pil)\n",
"img_id = img_a.view(-1, img_a.shape[0], img_a.shape[1], img_a.shape[2])\n",
"\n",
"# convert numpy to tensor\n",
"img_id = img_id.cuda()\n",
"\n",
"#create latent id\n",
"img_id_downsample = F.interpolate(img_id, scale_factor=0.5)\n",
"latend_id = model.netArc(img_id_downsample)\n",
"latend_id = latend_id.detach().to('cpu')\n",
"latend_id = latend_id/np.linalg.norm(latend_id,axis=1,keepdims=True)\n",
"latend_id = latend_id.to('cuda')\n",
"\n",
"video_swap(opt.video_path, latend_id, model, app, opt.output_path,temp_results_dir=opt.temp_path)"
],
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"text": [
"------------ Options -------------\n",
"Arc_path: models/BEST_checkpoint.tar\n",
"aspect_ratio: 1.0\n",
"batchSize: 8\n",
"checkpoints_dir: ./checkpoints\n",
"cluster_path: features_clustered_010.npy\n",
"data_type: 32\n",
"dataroot: ./datasets/cityscapes/\n",
"display_winsize: 512\n",
"engine: None\n",
"export_onnx: None\n",
"f: /root/.local/share/jupyter/runtime/kernel-6d955151-4911-464a-824d-f0806d8071f6.json\n",
"feat_num: 3\n",
"fineSize: 512\n",
"fp16: False\n",
"gpu_ids: [0]\n",
"how_many: 50\n",
"image_size: 224\n",
"input_nc: 3\n",
"instance_feat: False\n",
"isTrain: False\n",
"label_feat: False\n",
"label_nc: 0\n",
"latent_size: 512\n",
"loadSize: 1024\n",
"load_features: False\n",
"local_rank: 0\n",
"max_dataset_size: inf\n",
"model: pix2pixHD\n",
"nThreads: 2\n",
"n_blocks_global: 6\n",
"n_blocks_local: 3\n",
"n_clusters: 10\n",
"n_downsample_E: 4\n",
"n_downsample_global: 3\n",
"n_local_enhancers: 1\n",
"name: people\n",
"nef: 16\n",
"netG: global\n",
"ngf: 64\n",
"niter_fix_global: 0\n",
"no_flip: False\n",
"no_instance: False\n",
"norm: batch\n",
"norm_G: spectralspadesyncbatch3x3\n",
"ntest: inf\n",
"onnx: None\n",
"output_nc: 3\n",
"output_path: ./output/\n",
"phase: test\n",
"pic_a_path: ./crop_224/gdg.jpg\n",
"pic_b_path: ./crop_224/zrf.jpg\n",
"resize_or_crop: scale_width\n",
"results_dir: ./results/\n",
"semantic_nc: 3\n",
"serial_batches: False\n",
"temp_path: ./temp_results\n",
"tf_log: False\n",
"use_dropout: False\n",
"use_encoded_image: False\n",
"verbose: False\n",
"video_path: ./demo_file/mutil_people_1080p.mp4\n",
"which_epoch: latest\n",
"-------------- End ----------------\n",
"input mean and std: 127.5 127.5\n",
"find model: ./insightface_func/models/antelope/glintr100.onnx recognition\n",
"find model: ./insightface_func/models/antelope/scrfd_10g_bnkps.onnx detection\n",
"set det-size: (640, 640)\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"\r 0%| | 0/594 [00:00<?, ?it/s]"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"(142, 366, 4)\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"100%|██████████| 594/594 [08:45<00:00, 1.13it/s]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"[MoviePy] >>>> Building video ./output/demo.mp4\n",
"[MoviePy] Writing audio in demoTEMP_MPY_wvf_snd.mp3\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"100%|██████████| 438/438 [00:00<00:00, 877.18it/s]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"[MoviePy] Done.\n",
"[MoviePy] Writing video ./output/demo.mp4\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"100%|██████████| 595/595 [00:53<00:00, 11.15it/s]\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"[MoviePy] Done.\n",
"[MoviePy] >>>> Video ready: ./output/demo.mp4 \n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "Rty2GsyZZrI6"
},
"source": [],
"execution_count": null,
"outputs": []
}
]
}