Skimage similaritytransform does not work

Question

I am trying to apply a similarity transform to align faces:

from skimage.transform import SimilarityTransform, ProjectiveTransform
from skimage import transform
from scipy.misc import imshow

# face image detected facial landmarks
src = np.float32([left_eye_center, right_eye_center, mouth_center, nose_center])
# face template landmarks, see second image below
dst = np.float32([template['left_eye'], template['right_eye'], template['mouth'], template['nose']])

# see result in the third image below
tf = SimilarityTransform()
tf.estimate(src, dst)
result = transform.warp(image, inverse_map=tf.inverse)
imshow(result)

To test that other kinds of transformations do work I also try with a projective transform:

# see result in the fourth image below
tf = ProjectiveTransform()
tf.estimate(src, dst)
result = transform.warp(image, inverse_map=tf.inverse)
imshow(result)

Images of the original image,template, similarity transform and projective transform respectively:

As you can see, there is something wrong with the similarity transform, but I have no clue what it is. The projective transform seems to work fine, with the eyes, mouth and nose lining up with the points in the template.

What's going on? What am I not getting?

score 0 · Answer 1 · answered Jan 10 '18 at 14:13

I solved my problem by using a better template with 68 landmarks instead of 4. Also I use img_as_ubyte which help with displaying the array as image using imshow. from skimage import transform from scipy.misc import imshow from skimage import img_as_ubyte

# points match with the template below
template_landmarks = (68, 2) array with landmarks
# I use dlib to get detect landmarks from my image
detected_landmarks = (68, 2) array with landmarks

tf = transform.estimate_transform('similarity', detected_landmarks, template_landmarks)
result = img_as_ubyte(transform.warp(image, inverse_map=tf.inverse, output_shape=(198, 198, 3)))

# overlay template landmarks on result in green
for p in template_landmarks:
    x, y = p
    result[y, x] = [0, 255, 0]
imshow(result)

Below from left to right: original image, template, result overlaid with template.

Another result showing alignment results in this videoclip.

, ,

Skimage similaritytransform does not work

1 Answers1