Can you provide your training logs? #347

New Issue

Open

opened 2022-11-21 08:55:52 +01:00 by ygtxr1997 · 4 comments

ygtxr1997 commented 2022-11-21 08:55:52 +01:00 (Migrated from github.com)

Copy Link

Your training logs can be very helpful for re-implementing SimSwap on other training datasets.
We really need these logs for reference.
Thank you!

Your training logs can be very helpful for re-implementing SimSwap on other training datasets. We really need these logs for reference. Thank you!

ygtxr1997 commented 2022-11-21 09:21:09 +01:00 (Migrated from github.com)

Copy Link

You said 'Using insightface with different versions is not recommanded. Please use this specific version.'
Can you explain more detailedly why?
What will happen if I use other versions of face recognition model?

You said 'Using insightface with different versions is not recommanded. Please use this specific version.' Can you explain more detailedly why? What will happen if I use other versions of face recognition model?

netrunner-exe commented 2022-11-21 10:17:07 +01:00 (Migrated from github.com)

Copy Link

You said 'Using insightface with different versions is not recommanded. Please use this specific version.' Can you explain more detailedly why? What will happen if I use other versions of face recognition model?

To be honest, it is quite possible to use the latest version of insightface (0.6.2), but for this you need to edit the code a bit. I am not the developer of this repository, I just made an adaptation to work with the latest version of insightface some time ago.

First you need to uninsall old insightface by pip uninstall -y insightface and install latest by pip install insightface==0.6.2 and replace insightface_func/face_detect_crop_multi.py with this one.

# -*- coding: utf-8 -*-
# @Author: netrunner-exe
# @Date:   2022-06-19 18:02:36
# @Last Modified by:   netrunner-exe
# @Last Modified time: 2022-07-29 08:18:31
'''
Author: Naiyuan liu
Github: https://github.com/NNNNAI
Date: 2021-11-23 17:03:58
LastEditors: Naiyuan liu
LastEditTime: 2021-11-24 16:45:41
Description: 
'''
from __future__ import division

import collections
import glob
import os.path as osp

import cv2
import numpy as np
import onnxruntime
from insightface.model_zoo import model_zoo
from insightface.utils import DEFAULT_MP_NAME, ensure_available

from insightface_func.utils import face_align_ffhqandnewarc as face_align

__all__ = ['Face_detect_crop', 'Face']

Face = collections.namedtuple('Face', [
    'bbox', 'kps', 'det_score', 'embedding', 'gender', 'age',
    'embedding_norm', 'normed_embedding',
    'landmark'
])

Face.__new__.__defaults__ = (None, ) * len(Face._fields)


class Face_detect_crop:
    def __init__(self, name=DEFAULT_MP_NAME, root='~/.insightface', allowed_modules=None, **kwargs):
        onnxruntime.set_default_logger_severity(3)
        self.models = {}
        self.model_dir = ensure_available('models', name, root=root)
        onnx_files = glob.glob(osp.join(self.model_dir, '*.onnx'))
        onnx_files = sorted(onnx_files)
        for onnx_file in onnx_files:
            model = model_zoo.get_model(onnx_file, **kwargs)
            if model is None:
                print('model not recognized:', onnx_file)
            elif allowed_modules is not None and model.taskname not in allowed_modules:
                print('model ignore:', onnx_file, model.taskname)
                del model
            elif model.taskname not in self.models and (allowed_modules is None or model.taskname in allowed_modules):
                print('find model:', onnx_file, model.taskname, model.input_shape, model.input_mean, model.input_std)
                self.models[model.taskname] = model
            else:
                print('duplicated model task type, ignore:', onnx_file, model.taskname)
                del model
        assert 'detection' in self.models
        self.det_model = self.models['detection']


    def prepare(self, ctx_id, det_thresh=0.5, det_size=(640, 640), mode ='None'):
        self.det_thresh = det_thresh
        self.mode = mode
        assert det_size is not None
        print('set det-size:', det_size)
        self.det_size = det_size
        for taskname, model in self.models.items():
            if taskname=='detection':
                model.prepare(ctx_id, input_size=det_size, det_thresh=det_thresh)
            else:
                model.prepare(ctx_id)


    def get(self, img, crop_size, max_num=0):
        bboxes, kpss = self.det_model.detect(img,
                                             max_num=max_num,
                                             metric='default')
        if bboxes.shape[0] == 0:
            return None

        ret = []
        align_img_list = []
        M_list = []

        for i in range(bboxes.shape[0]):
            kps = None
            if kpss is not None:
                kps = kpss[i]

            M, _ = face_align.estimate_norm(kps, crop_size, mode = self.mode) 
            align_img = cv2.warpAffine(img, M, (crop_size, crop_size), borderValue=0.0)
            align_img_list.append(align_img)
            M_list.append(M)


        return align_img_list, M_list

Next, you need to replace insightface_func/face_detect_crop_single.py with this one.

# -*- coding: utf-8 -*-
# @Author: netrunner-exe
# @Date:   2022-06-19 18:02:36
# @Last Modified by:   netrunner-exe
# @Last Modified time: 2022-07-29 08:18:24
'''
Author: Naiyuan liu
Github: https://github.com/NNNNAI
Date: 2021-11-23 17:03:58
LastEditors: Naiyuan liu
LastEditTime: 2021-11-24 16:46:04
Description: 
'''
from __future__ import division

import collections
import glob
import os.path as osp

import cv2
import numpy as np
import onnxruntime
from insightface.model_zoo import model_zoo
from insightface.utils import DEFAULT_MP_NAME, ensure_available

from insightface_func.utils import face_align_ffhqandnewarc as face_align

__all__ = ['Face_detect_crop', 'Face']

Face = collections.namedtuple('Face', [
    'bbox', 'kps', 'det_score', 'embedding', 'gender', 'age',
    'embedding_norm', 'normed_embedding',
    'landmark'
])

Face.__new__.__defaults__ = (None, ) * len(Face._fields)


class Face_detect_crop:
    def __init__(self, name=DEFAULT_MP_NAME, root='~/.insightface', allowed_modules=None, **kwargs):
        onnxruntime.set_default_logger_severity(3)
        self.models = {}
        self.model_dir = ensure_available('models', name, root=root)
        onnx_files = glob.glob(osp.join(self.model_dir, '*.onnx'))
        onnx_files = sorted(onnx_files)
        for onnx_file in onnx_files:
            model = model_zoo.get_model(onnx_file, **kwargs)
            if model is None:
                print('model not recognized:', onnx_file)
            elif allowed_modules is not None and model.taskname not in allowed_modules:
                print('model ignore:', onnx_file, model.taskname)
                del model
            elif model.taskname not in self.models and (allowed_modules is None or model.taskname in allowed_modules):
                print('find model:', onnx_file, model.taskname, model.input_shape, model.input_mean, model.input_std)
                self.models[model.taskname] = model
            else:
                print('duplicated model task type, ignore:', onnx_file, model.taskname)
                del model
        assert 'detection' in self.models
        self.det_model = self.models['detection']


    def prepare(self, ctx_id, det_thresh=0.5, det_size=(640, 640), mode ='None'):
        self.det_thresh = det_thresh
        self.mode = mode
        assert det_size is not None
        print('set det-size:', det_size)
        self.det_size = det_size
        for taskname, model in self.models.items():
            if taskname=='detection':
                model.prepare(ctx_id, input_size=det_size, det_thresh=det_thresh)
            else:
                model.prepare(ctx_id)


    def get(self, img, crop_size, max_num=0):
        bboxes, kpss = self.det_model.detect(img,
                                             max_num=max_num,
                                             metric='default')
        if bboxes.shape[0] == 0:
            return None

        det_score = bboxes[..., 4]

        # select the face with the hightest detection score
        best_index = np.argmax(det_score)
        kps = None
        if kpss is not None:
            kps = kpss[best_index]

        M, _ = face_align.estimate_norm(kps, crop_size, mode = self.mode) 
        align_img = cv2.warpAffine(img, M, (crop_size, crop_size), borderValue=0.0)


        return [align_img], [M]

And finally, wherever there is
app = Face_detect_crop(name='antelope', root='./insightface_func/models')

Replace with this:
app = Face_detect_crop(name='antelope', root='./insightface_func', allowed_modules=['detection'], providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])

It also makes it possible to use antelopev2, buffalo_l, etc. instead of antelope. in Face_detect_crop. But accordingly, you need to download them and unzip them into the ./insightface_func folder.

Or you can try to just change the antelope to what you need and, in theory, the model itself should download.
But now it looks like they have problems with the server and therefore you will need to download and unpack it yourself.

> You said 'Using insightface with different versions is not recommanded. Please use this specific version.' Can you explain more detailedly why? What will happen if I use other versions of face recognition model? To be honest, it is quite possible to use the latest version of insightface (0.6.2), but for this you need to edit the code a bit. I am not the developer of this repository, I just made an adaptation to work with the latest version of insightface some time ago. First you need to uninsall old insightface by ` pip uninstall -y insightface` and install latest by `pip install insightface==0.6.2` and replace `insightface_func/face_detect_crop_multi.py` with this one. ``` # -*- coding: utf-8 -*- # @Author: netrunner-exe # @Date: 2022-06-19 18:02:36 # @Last Modified by: netrunner-exe # @Last Modified time: 2022-07-29 08:18:31 ''' Author: Naiyuan liu Github: https://github.com/NNNNAI Date: 2021-11-23 17:03:58 LastEditors: Naiyuan liu LastEditTime: 2021-11-24 16:45:41 Description: ''' from __future__ import division import collections import glob import os.path as osp import cv2 import numpy as np import onnxruntime from insightface.model_zoo import model_zoo from insightface.utils import DEFAULT_MP_NAME, ensure_available from insightface_func.utils import face_align_ffhqandnewarc as face_align __all__ = ['Face_detect_crop', 'Face'] Face = collections.namedtuple('Face', [ 'bbox', 'kps', 'det_score', 'embedding', 'gender', 'age', 'embedding_norm', 'normed_embedding', 'landmark' ]) Face.__new__.__defaults__ = (None, ) * len(Face._fields) class Face_detect_crop: def __init__(self, name=DEFAULT_MP_NAME, root='~/.insightface', allowed_modules=None, **kwargs): onnxruntime.set_default_logger_severity(3) self.models = {} self.model_dir = ensure_available('models', name, root=root) onnx_files = glob.glob(osp.join(self.model_dir, '*.onnx')) onnx_files = sorted(onnx_files) for onnx_file in onnx_files: model = model_zoo.get_model(onnx_file, **kwargs) if model is None: print('model not recognized:', onnx_file) elif allowed_modules is not None and model.taskname not in allowed_modules: print('model ignore:', onnx_file, model.taskname) del model elif model.taskname not in self.models and (allowed_modules is None or model.taskname in allowed_modules): print('find model:', onnx_file, model.taskname, model.input_shape, model.input_mean, model.input_std) self.models[model.taskname] = model else: print('duplicated model task type, ignore:', onnx_file, model.taskname) del model assert 'detection' in self.models self.det_model = self.models['detection'] def prepare(self, ctx_id, det_thresh=0.5, det_size=(640, 640), mode ='None'): self.det_thresh = det_thresh self.mode = mode assert det_size is not None print('set det-size:', det_size) self.det_size = det_size for taskname, model in self.models.items(): if taskname=='detection': model.prepare(ctx_id, input_size=det_size, det_thresh=det_thresh) else: model.prepare(ctx_id) def get(self, img, crop_size, max_num=0): bboxes, kpss = self.det_model.detect(img, max_num=max_num, metric='default') if bboxes.shape[0] == 0: return None ret = [] align_img_list = [] M_list = [] for i in range(bboxes.shape[0]): kps = None if kpss is not None: kps = kpss[i] M, _ = face_align.estimate_norm(kps, crop_size, mode = self.mode) align_img = cv2.warpAffine(img, M, (crop_size, crop_size), borderValue=0.0) align_img_list.append(align_img) M_list.append(M) return align_img_list, M_list ``` Next, you need to replace `insightface_func/face_detect_crop_single.py` with this one. ``` # -*- coding: utf-8 -*- # @Author: netrunner-exe # @Date: 2022-06-19 18:02:36 # @Last Modified by: netrunner-exe # @Last Modified time: 2022-07-29 08:18:24 ''' Author: Naiyuan liu Github: https://github.com/NNNNAI Date: 2021-11-23 17:03:58 LastEditors: Naiyuan liu LastEditTime: 2021-11-24 16:46:04 Description: ''' from __future__ import division import collections import glob import os.path as osp import cv2 import numpy as np import onnxruntime from insightface.model_zoo import model_zoo from insightface.utils import DEFAULT_MP_NAME, ensure_available from insightface_func.utils import face_align_ffhqandnewarc as face_align __all__ = ['Face_detect_crop', 'Face'] Face = collections.namedtuple('Face', [ 'bbox', 'kps', 'det_score', 'embedding', 'gender', 'age', 'embedding_norm', 'normed_embedding', 'landmark' ]) Face.__new__.__defaults__ = (None, ) * len(Face._fields) class Face_detect_crop: def __init__(self, name=DEFAULT_MP_NAME, root='~/.insightface', allowed_modules=None, **kwargs): onnxruntime.set_default_logger_severity(3) self.models = {} self.model_dir = ensure_available('models', name, root=root) onnx_files = glob.glob(osp.join(self.model_dir, '*.onnx')) onnx_files = sorted(onnx_files) for onnx_file in onnx_files: model = model_zoo.get_model(onnx_file, **kwargs) if model is None: print('model not recognized:', onnx_file) elif allowed_modules is not None and model.taskname not in allowed_modules: print('model ignore:', onnx_file, model.taskname) del model elif model.taskname not in self.models and (allowed_modules is None or model.taskname in allowed_modules): print('find model:', onnx_file, model.taskname, model.input_shape, model.input_mean, model.input_std) self.models[model.taskname] = model else: print('duplicated model task type, ignore:', onnx_file, model.taskname) del model assert 'detection' in self.models self.det_model = self.models['detection'] def prepare(self, ctx_id, det_thresh=0.5, det_size=(640, 640), mode ='None'): self.det_thresh = det_thresh self.mode = mode assert det_size is not None print('set det-size:', det_size) self.det_size = det_size for taskname, model in self.models.items(): if taskname=='detection': model.prepare(ctx_id, input_size=det_size, det_thresh=det_thresh) else: model.prepare(ctx_id) def get(self, img, crop_size, max_num=0): bboxes, kpss = self.det_model.detect(img, max_num=max_num, metric='default') if bboxes.shape[0] == 0: return None det_score = bboxes[..., 4] # select the face with the hightest detection score best_index = np.argmax(det_score) kps = None if kpss is not None: kps = kpss[best_index] M, _ = face_align.estimate_norm(kps, crop_size, mode = self.mode) align_img = cv2.warpAffine(img, M, (crop_size, crop_size), borderValue=0.0) return [align_img], [M] ``` And finally, wherever there is `app = Face_detect_crop(name='antelope', root='./insightface_func/models')` Replace with this: `app = Face_detect_crop(name='antelope', root='./insightface_func', allowed_modules=['detection'], providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])` It also makes it possible to use `antelopev2`, `buffalo_l`, etc. instead of `antelope`. in `Face_detect_crop`. But accordingly, you need to download them and unzip them into the `./insightface_func` folder. Or you can try to just change the `antelope` to what you need and, in theory, the model itself should download. But now it looks like they have problems with the server and therefore you will need to download and unpack it yourself.

netrunner-exe commented 2022-11-21 10:27:31 +01:00 (Migrated from github.com)

Copy Link

Looks like the automatic download for buffalo_l is working now!)

Looks like the automatic download for `buffalo_l` is working now!) 

netrunner-exe commented 2022-11-21 10:38:12 +01:00 (Migrated from github.com)

Copy Link

By the way, I noticed that buffalo_l and buffalo_sc give a slightly different output.

buffalo_l (result almost same as default antelope):

and buffalo_sc (very lightweight version of buffalo_l, 16MB vs 326MB of buffalo_l):

By the way, I noticed that `buffalo_l` and `buffalo_sc` give a slightly different output. `buffalo_l` (result almost same as default `antelope`):  and `buffalo_sc` (very lightweight version of `buffalo_l`, 16MB vs 326MB of `buffalo_l`): 

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

512_beta

1.0

Labels

Clear labels

GIF

GUI

Increase inference time

A solution to increase inference time

bug

Something isn't working

documentation

Improvements or additions to documentation

duplicate

This issue or pull request already exists

enhancement

New feature or request

good first issue

Good for newcomers

help wanted

Extra attention is needed

invalid

This doesn't seem right

question

Further information is requested

wontfix

This will not be worked on

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

CalvinBackup

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: CalvinBackup/neuralchen-SimSwap#347