how to adjust swap intensity #34

New Issue

2021-07-03T23:15:00+02:00

molo32 commented

2021-07-03 23:15:00 +02:00

(Migrated from github.com)

I would like to know if it is possible to adjust the force of the face swap.

for example

force = 0
show original face
force = 0.5
original and new face mixed
force = 1
show the new face

I would like to know if it is possible to adjust the force of the face swap. for example force = 0 show original face force = 0.5 original and new face mixed force = 1 show the new face

👍 1

woctezuma commented

2021-07-03 23:18:50 +02:00

(Migrated from github.com)

If it is just a matter of merging the two images (input and output), you could do it by averaging the 2 images with layers in Gimp.

molo32 commented

2021-07-03 23:30:56 +02:00

(Migrated from github.com)

with average, if two faces have different structures, an overlap of faces will be noticed
I was referring more to an interpolation as stylegan.

with average, if two faces have different structures, an overlap of faces will be noticed I was referring more to an interpolation as stylegan.

👍 1

ExponentialML commented

2021-07-04 09:32:25 +02:00

(Migrated from github.com)

with average, if two faces have different structures, an overlap of faces will be noticed
I was referring more to an interpolation as stylegan.

You have two choices. You can either average the two videos together (50% opacity overlay in your favorite video editor or GIIMP), or you can mess with the bellow settings.

fc4b701354/test_video_swapsingle.py (L22-L25)

To make this easier to understand, divide each of the values in the list by half on this line:
fc4b701354/test_video_swapsingle.py (L24)

Then perform a swap. You'll get an idea of how much you should want to tune these parameters, but going too far can lead to some nightmarish results 🙂 .

> with average, if two faces have different structures, an overlap of faces will be noticed > I was referring more to an interpolation as stylegan. You have two choices. You can either average the two videos together (50% opacity overlay in your favorite video editor or GIIMP), or you can mess with the bellow settings. https://github.com/neuralchen/SimSwap/blob/fc4b7013547f023223c83097923aa255a8dd05e7/test_video_swapsingle.py#L22-L25 To make this easier to understand, divide **each** of the values in the list by half on this line: https://github.com/neuralchen/SimSwap/blob/fc4b7013547f023223c83097923aa255a8dd05e7/test_video_swapsingle.py#L24 Then perform a swap. You'll get an idea of how much you should want to tune these parameters, but going too far can lead to some nightmarish results :slightly_smiling_face: .

👍 2

asiboro commented

2021-08-03 16:10:56 +02:00

(Migrated from github.com)

@ftaker887 "To make this easier to understand, divide each of the values in the list by half on this line" I tried halving each value but result video did not change much...

mata3o commented

2021-08-25 00:58:10 +02:00

(Migrated from github.com)

Ive tried swapping those numbers out in all ways and i haven't seen any sort of difference either good or bad. Am i doing something wrong in that regard? I do try to get the most similar facial features but sometimes id like to make the swapped face more pronounced. Any help would be greatly appreciated , Cheers!

woctezuma commented

2021-10-17 08:51:56 +02:00

(Migrated from github.com)

You are not supposed to swap values, only divide all of them by the same number.
I have not tried though.

The first argument is the mean, and the second the standard-deviation.
https://pytorch.org/vision/stable/transforms.html#torchvision.transforms.Normalize

You are not supposed to swap values, only divide all of them by the same number. I have not tried though. The first argument is the mean, and the second the standard-deviation. https://pytorch.org/vision/stable/transforms.html#torchvision.transforms.Normalize

paulescobar commented

2021-10-18 08:23:48 +02:00

(Migrated from github.com)

Okay guys, I did some thorough testing of video conversions tonight. I'm a total newb to this, but a couple of people mentioned they weren't seeing any difference or hadn't tested this. So thought I'd share this for the record.

Thanks to @ftaker887 for sharing his suggestion of editing the specific values. That's what I did tonight, by dividing with a large range/sample of numbers. Here are my key observations...

There are differences - so if you don't see any, it's likely because you're editing the wrong section. I was guilty of this early on. It's really tedious doing the various calculations & you can get confused and paste the new values in the wrong area.
Multiplying seems to do absolutely nothing. So stick to dividing these values.
I tested divisions from many different values, ranging from 1.5 to 8. Ultimately, dividing by any values in the range of 1.1 to 2 gives you clean results that lean more towards the "new face" rather than the "old face". It's a subtle difference, and will likely not satisfy those who wish the "new face" would be more perfectly/dominantly transferred. But I do feel it's an improvement to the default value.
Dividing by any value 2.1 and higher (ie. 3, 4, 5, 6, etc.) is useless because it seems to add weird distortions that resemble "aging" or "wrinkles". The higher the number gets, the more you see bizarre distortions in the forehead (ie. the "nightmarish results" that @ftaker887 alluded to).
I think the single best value to divide by is 2. Anything lesser is weaker and leans more towards the original face. Even a single decimal point higher (2.1, 2.2, 2.3, etc.) begins to add the previously mentioned distortions that resemble aging or wrinkles.

Would be glad to hear from others what their observations were, if they played around with this technique.

Okay guys, I did some thorough testing of video conversions tonight. I'm a total newb to this, but a couple of people mentioned they weren't seeing any difference or hadn't tested this. So thought I'd share this for the record. Thanks to @ftaker887 for sharing his suggestion of editing the specific values. That's what I did tonight, by dividing with a large range/sample of numbers. Here are my key observations... 1) There are differences - so if you don't see any, it's likely because you're editing the wrong section. I was guilty of this early on. It's really tedious doing the various calculations & you can get confused and paste the new values in the wrong area. 2) Multiplying seems to do absolutely nothing. So stick to dividing these values. 3) I tested divisions from many different values, ranging from 1.5 to 8. Ultimately, dividing by any values in the range of 1.1 to 2 gives you clean results that lean more towards the "new face" rather than the "old face". It's a subtle difference, and will likely not satisfy those who wish the "new face" would be more perfectly/dominantly transferred. But I do feel it's an improvement to the default value. 4) Dividing by any value 2.1 and higher (ie. 3, 4, 5, 6, etc.) is useless because it seems to add weird distortions that resemble "aging" or "wrinkles". The higher the number gets, the more you see bizarre distortions in the forehead (ie. the "nightmarish results" that @ftaker887 alluded to). 5) I think the single best value to divide by is 2. Anything lesser is weaker and leans more towards the original face. Even a single decimal point higher (2.1, 2.2, 2.3, etc.) begins to add the previously mentioned distortions that resemble aging or wrinkles. Would be glad to hear from others what their observations were, if they played around with this technique.

🚀 1

mata3o commented

2021-10-22 03:58:17 +02:00

(Migrated from github.com)

@paulescobar sounds like great testing you've done so far! Would you be able to paste your exact settings that worked best for you? I'd like to start where you left off. When i changed the numbers i wasn't paying too close attention but could see those "wrinkles" if i tweaked one too far

😆 1

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: CalvinBackup/neuralchen-SimSwap#34