Simswap 512 on Colab #162

New Issue

Open

opened 2021-11-24 17:05:44 +01:00 by Lyntai · 7 comments

Lyntai commented 2021-11-24 17:05:44 +01:00 (Migrated from github.com)

Copy Link

Copy Source

Hello!

The results are awesome, I'm trying to reproduce them on Colab but I'm having issues, the code as it is, seems to output as the original behavior.

If I modify
crop_size = opt.crop_size
to
crop_size = 512
nothing happens.

I I use instead use:
opt.crop_size = 512
crop_size = opt.crop_size

I get these results, they vary on each run:

Hello! The results are awesome, I'm trying to reproduce them on Colab but I'm having issues, the code as it is, seems to output as the original behavior. If I modify crop_size = opt.crop_size to crop_size = 512 nothing happens. I I use instead use: opt.crop_size = 512 crop_size = opt.crop_size I get these results, they vary on each run:   

Lyntai commented 2021-11-24 19:27:27 +01:00 (Migrated from github.com)

Copy Link

Copy Source

I managed to fix it (this may not be perfect and may have bugs), if anyone wants to try it, do this:

1. Download the checkpoints of the Simswap 512 beta version:

!wget https://github.com/neuralchen/SimSwap/releases/download/512_beta/512.zip
!unzip ./512.zip  -d ./checkpoints

2. You can swap with this code:

opt = TestOptions()
opt.initialize()
opt.parser.add_argument('-f') ## dummy arg to avoid bug
opt = opt.parse()
opt.pic_a_path = './demo_file/Iron_man.jpg' ## or replace it with image from your own google drive
opt.video_path = './demo_file/multi_people_1080p.mp4' ## or replace it with video from your own google drive
opt.output_path = './output/demo.mp4'
opt.temp_path = './tmp'
opt.Arc_path = './arcface_model/arcface_checkpoint.tar'
opt.isTrain = False
opt.use_mask = True  ## new feature up-to-date
opt.crop_size = 512
crop_size = opt.crop_size

torch.nn.Module.dump_patches = True
if crop_size == 512:
   opt.which_epoch = 550000
   opt.name = '512'
   mode = 'ffhq'
else:
   mode = 'None'

model = create_model(opt)
model.eval()

app = Face_detect_crop(name='antelope', root='./insightface_func/models')
app.prepare(ctx_id= 0, det_thresh=0.6, det_size=(640,640),mode=mode)

with torch.no_grad():
    pic_a = opt.pic_a_path
    # img_a = Image.open(pic_a).convert('RGB')
    img_a_whole = cv2.imread(pic_a)
    img_a_align_crop, _ = app.get(img_a_whole,crop_size)
    img_a_align_crop_pil = Image.fromarray(cv2.cvtColor(img_a_align_crop[0],cv2.COLOR_BGR2RGB)) 
    img_a = transformer_Arcface(img_a_align_crop_pil)
    img_id = img_a.view(-1, img_a.shape[0], img_a.shape[1], img_a.shape[2])

    # convert numpy to tensor
    img_id = img_id.cuda()

    #create latent id
    img_id_downsample = F.interpolate(img_id, size=(112,112))
    latend_id = model.netArc(img_id_downsample)
    latend_id = latend_id.detach().to('cpu')
    latend_id = latend_id/np.linalg.norm(latend_id,axis=1,keepdims=True)
    latend_id = latend_id.to('cuda')

    video_swap(opt.video_path, latend_id, model, app, opt.output_path, temp_results_dir=opt.temp_path, use_mask=opt.use_mask, crop_size=crop_size)

You can switch between 224 and 512 by modifying opt.crop_size value.

224

512

Congratulations to the authors for doing a wonderful job.

I managed to fix it (this may not be perfect and may have bugs), if anyone wants to try it, do this: 1) Download the checkpoints of the Simswap 512 beta version: ``` !wget https://github.com/neuralchen/SimSwap/releases/download/512_beta/512.zip !unzip ./512.zip -d ./checkpoints ``` 2) You can swap with this code: ``` opt = TestOptions() opt.initialize() opt.parser.add_argument('-f') ## dummy arg to avoid bug opt = opt.parse() opt.pic_a_path = './demo_file/Iron_man.jpg' ## or replace it with image from your own google drive opt.video_path = './demo_file/multi_people_1080p.mp4' ## or replace it with video from your own google drive opt.output_path = './output/demo.mp4' opt.temp_path = './tmp' opt.Arc_path = './arcface_model/arcface_checkpoint.tar' opt.isTrain = False opt.use_mask = True ## new feature up-to-date opt.crop_size = 512 crop_size = opt.crop_size torch.nn.Module.dump_patches = True if crop_size == 512: opt.which_epoch = 550000 opt.name = '512' mode = 'ffhq' else: mode = 'None' model = create_model(opt) model.eval() app = Face_detect_crop(name='antelope', root='./insightface_func/models') app.prepare(ctx_id= 0, det_thresh=0.6, det_size=(640,640),mode=mode) with torch.no_grad(): pic_a = opt.pic_a_path # img_a = Image.open(pic_a).convert('RGB') img_a_whole = cv2.imread(pic_a) img_a_align_crop, _ = app.get(img_a_whole,crop_size) img_a_align_crop_pil = Image.fromarray(cv2.cvtColor(img_a_align_crop[0],cv2.COLOR_BGR2RGB)) img_a = transformer_Arcface(img_a_align_crop_pil) img_id = img_a.view(-1, img_a.shape[0], img_a.shape[1], img_a.shape[2]) # convert numpy to tensor img_id = img_id.cuda() #create latent id img_id_downsample = F.interpolate(img_id, size=(112,112)) latend_id = model.netArc(img_id_downsample) latend_id = latend_id.detach().to('cpu') latend_id = latend_id/np.linalg.norm(latend_id,axis=1,keepdims=True) latend_id = latend_id.to('cuda') video_swap(opt.video_path, latend_id, model, app, opt.output_path, temp_results_dir=opt.temp_path, use_mask=opt.use_mask, crop_size=crop_size) ``` You can switch between 224 and 512 by modifying opt.crop_size value. 224  512  Congratulations to the authors for doing a wonderful job.

👍 2 🎉 2

NNNNAI commented 2021-11-25 04:04:34 +01:00 (Migrated from github.com)

Copy Link

Copy Source

Sorry for the late respond. Grate to see you make it work, I forget to update the colab file to the latest version yesterday, I will update those files today.Have a nice day.

Sorry for the late respond. Grate to see you make it work, I forget to update the colab file to the latest version yesterday, I will update those files today.Have a nice day.

👍 3

woctezuma commented 2022-06-11 14:35:58 +02:00 (Migrated from github.com)

Copy Link

Copy Source

I was contacted about this subject at https://github.com/woctezuma/SimSwap-colab/issues/3 and was a bit confused about the original issue, and where the fix was in the code posted above. So I will try to explain what I understood and see if I got it right after reading both of these issues:

• Simswap 512 on Colab (#162)
• Simswap 512 (beta version) rendering (#198)

As I understand, the issue only arises for people who use Colab notebooks instead of the official Python scripts. In these cases, the value of the crop_size variable is not set via the function argument --crop_size 512, but manually set by the user in the Colab notebook. This is an issue because opt.crop_size is later used by the program when create_model() is called, so the user is supposed to have set it to the non-default value (512) beforehand.

https://github.com/neuralchen/SimSwap/blob/050ed8a00a190aafafe183fdfa6a0eb845ded89d/models/fs_model.py#L53-L56

https://github.com/neuralchen/SimSwap/blob/dd1ecdd2a718636d33977ab3097a69a0ecf080d8/options/test_options.py#L36

The fix consists in setting opt.crop_size (to the same value as crop_size) in the Colab notebook.

I was contacted about this subject at https://github.com/woctezuma/SimSwap-colab/issues/3 and was a bit confused about the original issue, and where the fix was in the code posted above. So I will try to explain what I understood and see if I got it right after reading both of these issues: - #162 - #198 As I understand, the issue only arises for people who use Colab notebooks instead of the official Python scripts. In these cases, the value of the `crop_size` variable is not set via the function argument `--crop_size 512`, but manually set by the user in the Colab notebook. This is an issue because `opt.crop_size` is later used by the program when `create_model()` is called, so the user is supposed to have set it to the non-default value (512) beforehand. https://github.com/neuralchen/SimSwap/blob/050ed8a00a190aafafe183fdfa6a0eb845ded89d/models/fs_model.py#L53-L56 https://github.com/neuralchen/SimSwap/blob/dd1ecdd2a718636d33977ab3097a69a0ecf080d8/options/test_options.py#L36 The fix consists in setting `opt.crop_size` (to the same value as `crop_size`) in the Colab notebook.

mayige commented 2022-07-24 18:16:47 +02:00 (Migrated from github.com)

Copy Link

Copy Source

How is this going?

How is this going? 

woctezuma commented 2022-07-24 19:01:52 +02:00 (Migrated from github.com)

Copy Link

Copy Source

I don't know which script you are trying to run, and you don't show the error message. I cannot help you with that. 😄

I don't know which script you are trying to run, and you don't show the error message. I cannot help you with that. 😄

Luby17 commented 2023-04-13 16:59:03 +02:00 (Migrated from github.com)

Copy Link

Copy Source

I managed to fix it (this may not be perfect and may have bugs), if anyone wants to try it, do this:

1. Download the checkpoints of the Simswap 512 beta version:

!wget https://github.com/neuralchen/SimSwap/releases/download/512_beta/512.zip
!unzip ./512.zip  -d ./checkpoints

2. You can swap with this code:

opt = TestOptions()
opt.initialize()
opt.parser.add_argument('-f') ## dummy arg to avoid bug
opt = opt.parse()
opt.pic_a_path = './demo_file/Iron_man.jpg' ## or replace it with image from your own google drive
opt.video_path = './demo_file/multi_people_1080p.mp4' ## or replace it with video from your own google drive
opt.output_path = './output/demo.mp4'
opt.temp_path = './tmp'
opt.Arc_path = './arcface_model/arcface_checkpoint.tar'
opt.isTrain = False
opt.use_mask = True  ## new feature up-to-date
opt.crop_size = 512
crop_size = opt.crop_size

torch.nn.Module.dump_patches = True
if crop_size == 512:
   opt.which_epoch = 550000
   opt.name = '512'
   mode = 'ffhq'
else:
   mode = 'None'

model = create_model(opt)
model.eval()

app = Face_detect_crop(name='antelope', root='./insightface_func/models')
app.prepare(ctx_id= 0, det_thresh=0.6, det_size=(640,640),mode=mode)

with torch.no_grad():
    pic_a = opt.pic_a_path
    # img_a = Image.open(pic_a).convert('RGB')
    img_a_whole = cv2.imread(pic_a)
    img_a_align_crop, _ = app.get(img_a_whole,crop_size)
    img_a_align_crop_pil = Image.fromarray(cv2.cvtColor(img_a_align_crop[0],cv2.COLOR_BGR2RGB)) 
    img_a = transformer_Arcface(img_a_align_crop_pil)
    img_id = img_a.view(-1, img_a.shape[0], img_a.shape[1], img_a.shape[2])

    # convert numpy to tensor
    img_id = img_id.cuda()

    #create latent id
    img_id_downsample = F.interpolate(img_id, size=(112,112))
    latend_id = model.netArc(img_id_downsample)
    latend_id = latend_id.detach().to('cpu')
    latend_id = latend_id/np.linalg.norm(latend_id,axis=1,keepdims=True)
    latend_id = latend_id.to('cuda')

    video_swap(opt.video_path, latend_id, model, app, opt.output_path, temp_results_dir=opt.temp_path, use_mask=opt.use_mask, crop_size=crop_size)

You can switch between 224 and 512 by modifying opt.crop_size value.

224

512

Congratulations to the authors for doing a wonderful job.

I´ve copied the code and ended up with 256 anyway

> I managed to fix it (this may not be perfect and may have bugs), if anyone wants to try it, do this: > > 1. Download the checkpoints of the Simswap 512 beta version: > > ``` > !wget https://github.com/neuralchen/SimSwap/releases/download/512_beta/512.zip > !unzip ./512.zip -d ./checkpoints > ``` > > 2. You can swap with this code: > > ``` > opt = TestOptions() > opt.initialize() > opt.parser.add_argument('-f') ## dummy arg to avoid bug > opt = opt.parse() > opt.pic_a_path = './demo_file/Iron_man.jpg' ## or replace it with image from your own google drive > opt.video_path = './demo_file/multi_people_1080p.mp4' ## or replace it with video from your own google drive > opt.output_path = './output/demo.mp4' > opt.temp_path = './tmp' > opt.Arc_path = './arcface_model/arcface_checkpoint.tar' > opt.isTrain = False > opt.use_mask = True ## new feature up-to-date > opt.crop_size = 512 > crop_size = opt.crop_size > > torch.nn.Module.dump_patches = True > if crop_size == 512: > opt.which_epoch = 550000 > opt.name = '512' > mode = 'ffhq' > else: > mode = 'None' > > model = create_model(opt) > model.eval() > > app = Face_detect_crop(name='antelope', root='./insightface_func/models') > app.prepare(ctx_id= 0, det_thresh=0.6, det_size=(640,640),mode=mode) > > with torch.no_grad(): > pic_a = opt.pic_a_path > # img_a = Image.open(pic_a).convert('RGB') > img_a_whole = cv2.imread(pic_a) > img_a_align_crop, _ = app.get(img_a_whole,crop_size) > img_a_align_crop_pil = Image.fromarray(cv2.cvtColor(img_a_align_crop[0],cv2.COLOR_BGR2RGB)) > img_a = transformer_Arcface(img_a_align_crop_pil) > img_id = img_a.view(-1, img_a.shape[0], img_a.shape[1], img_a.shape[2]) > > # convert numpy to tensor > img_id = img_id.cuda() > > #create latent id > img_id_downsample = F.interpolate(img_id, size=(112,112)) > latend_id = model.netArc(img_id_downsample) > latend_id = latend_id.detach().to('cpu') > latend_id = latend_id/np.linalg.norm(latend_id,axis=1,keepdims=True) > latend_id = latend_id.to('cuda') > > video_swap(opt.video_path, latend_id, model, app, opt.output_path, temp_results_dir=opt.temp_path, use_mask=opt.use_mask, crop_size=crop_size) > ``` > > You can switch between 224 and 512 by modifying opt.crop_size value. > > 224  > > 512  > > Congratulations to the authors for doing a wonderful job. I´ve copied the code and ended up with 256 anyway

acousticdragon commented 2025-01-06 10:28:43 +01:00 (Migrated from github.com)

Copy Link

Copy Source

Anyone who has a working copy of the colab so you can pass it to me? Please 🥹

Anyone who has a working copy of the colab so you can pass it to me? Please 🥹

😕 1

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

512_beta

1.0

Labels

Clear labels

GIF

GUI

Increase inference time

A solution to increase inference time

bug

Something isn't working

documentation

Improvements or additions to documentation

duplicate

This issue or pull request already exists

enhancement

New feature or request

good first issue

Good for newcomers

help wanted

Extra attention is needed

invalid

This doesn't seem right

question

Further information is requested

wontfix

This will not be worked on

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

CalvinBackup (Calvin Backup)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: CalvinBackup/neuralchen-SimSwap#162