SimSwap without replacing the mouth #413

New Issue

2023-05-24T18:44:33+02:00

SamPse commented

2023-05-24 18:44:33 +02:00

(Migrated from github.com)

Is there a way to use SimSwap without replacing the mouth? only Eyes and nose !

instant-high commented

2023-05-25 09:12:05 +02:00

(Migrated from github.com)

You can play with the face parts in util/reverse2original.py
Remove #1 which is the face
You can find the list of facial parts somewhere in the original github repo

You can play with the face parts in util/reverse2original.py Remove #1 which is the face You can find the list of facial parts somewhere in the original github repo

SamPse commented

2023-05-25 19:14:18 +02:00

(Migrated from github.com)

You can play with the face parts in util/reverse2original.py Remove #1 which is the face You can find the list of facial parts somewhere in the original github repo

Thank you for your answer. I am no expert with opencv. The part to modify is this if I understood correctly.

def encode_segmentation_rgb(segmentation, no_neck=True):
parse = segmentation

face_part_ids = [1, 2, 3, 4, 5, 6, 10, 12, 13] if no_neck else [1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14]
mouth_id = 11
# hair_id = 17
face_map = np.zeros([parse.shape[0], parse.shape[1]])
mouth_map = np.zeros([parse.shape[0], parse.shape[1]])
# hair_map = np.zeros([parse.shape[0], parse.shape[1]])

for valid_id in face_part_ids:
    valid_index = np.where(parse==valid_id)
    face_map[valid_index] = 255
valid_index = np.where(parse==mouth_id)
mouth_map[valid_index] = 255
# valid_index = np.where(parse==hair_id)
# hair_map[valid_index] = 255
#return np.stack([face_map, mouth_map,hair_map], axis=2)
return np.stack([face_map, mouth_map], axis=2)

> You can play with the face parts in util/reverse2original.py Remove #1 which is the face You can find the list of facial parts somewhere in the original github repo Thank you for your answer. I am no expert with opencv. The part to modify is this if I understood correctly. def encode_segmentation_rgb(segmentation, no_neck=True): parse = segmentation face_part_ids = [1, 2, 3, 4, 5, 6, 10, 12, 13] if no_neck else [1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14] mouth_id = 11 # hair_id = 17 face_map = np.zeros([parse.shape[0], parse.shape[1]]) mouth_map = np.zeros([parse.shape[0], parse.shape[1]]) # hair_map = np.zeros([parse.shape[0], parse.shape[1]]) for valid_id in face_part_ids: valid_index = np.where(parse==valid_id) face_map[valid_index] = 255 valid_index = np.where(parse==mouth_id) mouth_map[valid_index] = 255 # valid_index = np.where(parse==hair_id) # hair_map[valid_index] = 255 #return np.stack([face_map, mouth_map,hair_map], axis=2) return np.stack([face_map, mouth_map], axis=2)

instant-high commented

2023-05-25 20:05:11 +02:00

(Migrated from github.com)

Yes. Just remove 1 and ,
you have to use the --use_mask option in your commandline.

Yes. Just remove 1 and , you have to use the --use_mask option in your commandline.

SamPse commented

2023-05-26 11:55:42 +02:00

(Migrated from github.com)

Yes. Just remove 1 and , you have to use the --use_mask option in your commandline.

Thanks !
I removed 1 and , forme this : face_part_ids = [2, 3, 4, 5, 6, 10, 12, 13] if no_neck else [2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14]
but doesn’t work.

> Yes. Just remove 1 and , you have to use the --use_mask option in your commandline. Thanks ! I removed 1 and , forme this : face_part_ids = [2, 3, 4, 5, 6, 10, 12, 13] if no_neck else [2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14] but doesn’t work.

instant-high commented

2023-05-26 16:49:49 +02:00

(Migrated from github.com)

Yes, mask has soft border....
You can cange iterations from 7 to 1 at line 81
smooth_mask = SoftErosion(kernel_size=17, threshold=0.9, iterations=7).cuda()

Look at the image, from left to right:
Mask with #1 , without #1 and iterations 7 , without #1 but iterations 1

EDIT: forgot to say, you can play also with kernel size, only odd numbers I think

Yes, mask has soft border.... You can cange iterations from 7 to 1 at line 81 _smooth_mask = SoftErosion(kernel_size=17, threshold=0.9, iterations=7).cuda()_ Look at the image, from left to right: Mask with #1 , without #1 and iterations 7 , without #1 but iterations 1 ![mask](https://github.com/neuralchen/SimSwap/assets/77229558/127718d5-cf2d-4140-8368-323f1959ad7f) EDIT: forgot to say, you can play also with kernel size, only odd numbers I think

SamPse commented

2023-05-26 21:14:55 +02:00

(Migrated from github.com)

Yes, mask has soft border.... You can cange iterations from 7 to 1 at line 81 smooth_mask = SoftErosion(kernel_size=17, threshold=0.9, iterations=7).cuda()

Look at the image, from left to right: Mask with #1 , without #1 and iterations 7 , without #1 but iterations 1

EDIT: forgot to say, you can play also with kernel size, only odd numbers I think

Very interesting but on my side I have no change. I don't know why, just for information I use the graphical version: test_video_swapsingle_GUI1.py

> Yes, mask has soft border.... You can cange iterations from 7 to 1 at line 81 _smooth_mask = SoftErosion(kernel_size=17, threshold=0.9, iterations=7).cuda()_ > > Look at the image, from left to right: Mask with #1 , without #1 and iterations 7 , without #1 but iterations 1 > > ![mask](https://user-images.githubusercontent.com/77229558/241251706-127718d5-cf2d-4140-8368-323f1959ad7f.png) > > EDIT: forgot to say, you can play also with kernel size, only odd numbers I think Very interesting but on my side I have no change. I don't know why, just for information I use the graphical version: test_video_swapsingle_GUI1.py

instant-high commented

2023-05-26 23:14:59 +02:00

(Migrated from github.com)

Ok. The GUI version is not official. I guess you donwloaded it from my youtube channel?
I will check it the next few days

Ok. The GUI version is not official. I guess you donwloaded it from my youtube channel? I will check it the next few days

SamPse commented

2023-05-27 21:58:28 +02:00

(Migrated from github.com)

Ok. The GUI version is not official. I guess you donwloaded it from my youtube channel? I will check it the next few days

Yes it's from your youtube channel. Thank you!

> Ok. The GUI version is not official. I guess you donwloaded it from my youtube channel? I will check it the next few days Yes it's from your youtube channel. Thank you!

instant-high commented

2023-05-28 10:42:39 +02:00

(Migrated from github.com)

I've tried it myself. It doesn't work as expected.
Maybe it's because they they do masking complety different as I do it in my other projects.
If I remember correctly, in the initial version of simswap (2021) it did work this way?

Sorry, maybe it requires more code changes...

I've tried it myself. It doesn't work as expected. Maybe it's because they they do masking complety different as I do it in my other projects. If I remember correctly, in the initial version of simswap (2021) it did work this way? Sorry, maybe it requires more code changes...

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: CalvinBackup/neuralchen-SimSwap#413