not an issue - emotion transfer / stylegan2 #40

New Issue

2021-07-05T15:23:12+02:00

johndpope commented

2021-07-05 15:23:12 +02:00

(Migrated from github.com)

I've seen other code that's been added to fsgan to attempt to inject emotion.
24637fcf10

because the architecture is quite simple - presumably - you'd just need to do some neural net surgery on the ID injection layer?

if you have any line numbers to / blended networks to play with.....

Also - I see the pix2pix model is part of the architecture - were there any efforts to tack on stylegan2 into the mix?

I've seen other code that's been added to fsgan to attempt to inject emotion. https://github.com/BigBey/emotion_transfer/commit/24637fcf10a871dc7dc1a9c6681a655509c39c4e because the architecture is quite simple - presumably - you'd just need to do some neural net surgery on the ID injection layer? if you have any line numbers to / blended networks to play with..... Also - I see the pix2pix model is part of the architecture - were there any efforts to tack on stylegan2 into the mix?

👍 1 👀 1 ❤️ 1

neuralchen commented

2021-07-06 03:52:06 +02:00

(Migrated from github.com)

Thanks for your golden suggestion!
Our framework was designed to maintain the original emotion of target face.
We believe that to injection layer is a critical component in manipulating the generation results.
Actually, there are a lot of papers employ the stylegan1/2 scheme to edit face attributes and swap face.

Thanks for your golden suggestion! Our framework was designed to maintain the original emotion of target face. We believe that to injection layer is a critical component in manipulating the generation results. Actually, there are a lot of papers employ the stylegan1/2 scheme to edit face attributes and swap face.

👍 1

johndpope commented

2021-07-16 10:11:13 +02:00

(Migrated from github.com)

fyi ShapeEditer: a StyleGAN Encoder for Face Swapping
https://arxiv.org/pdf/2106.13984v1.pdf

fyi ShapeEditer: a StyleGAN Encoder for Face Swapping https://arxiv.org/pdf/2106.13984v1.pdf

👀 1

woctezuma commented

2021-07-16 10:37:07 +02:00

(Migrated from github.com)

Results are not super impressive to me, as I don't think that "face identity" is preserved.

johndpope commented

2021-07-16 10:46:14 +02:00

(Migrated from github.com)

stylegan2 is a difficult to explore latent space. The Pivotal Tuning paper / code by @danielroich seems most promising with impressive results. https://github.com/danielroich/PTI - maybe @williamyang1991 / the author of paper could leverage this PTI approach to upgrade results.

👍 2 👀 1

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: CalvinBackup/neuralchen-SimSwap#40