2

I have a dataset of images with faces. I also have for each face within the dataset a set of 66 2D points that correspond to my face landmarks(nose, eyes, shape of my face, mouth).

So basically I have the shape of my face in terms of 2D points from my image.

Do you know any algorithm that I can use and that can rotate my shape so that the face shape is straight? Let's say that the pan angle is 30 degrees and I want it rotated to 30 degrees so that it is positioned at 0 degrees on the pan angle. I have illustrated bellow what I want to say.

enter image description here

enter image description here

Basically you can consider the above illustrated shapes outlines for my images, which are represented in 2D. I want to rotate my first shape points so that they can look like the second shape. A shape is made out of a set of 66 2D points which are basically pixel coordinates. All I want to do is to find the correspondence of each of those 66 points so that the new shape is rotated with a certain degree on the pan angle.

Simon
  • 4,999
  • 21
  • 69
  • 97
  • Do you know the depth of each 2d point? The only reason your rotation looks like that is because the nose and mouth are actually in front of the jawline. – Hammer Aug 29 '12 at 14:52
  • Do you want to rotate the shape point coordinates (i.e. the 66 2D landmarks), so you will have a frontal shape (outline) or to warp an image (that corresponds to those 2D points) to a frontal looking pose? To rephrase that: do you want to transform the point coordinates ("shape") or warp image intensity values ("texture"). – gevang Aug 29 '12 at 15:03

1 Answers1

6

From your question, I can assume you either have the rotation parameters (e.g. degrees in x,y) or the point correspondences (since you have a database of matched points). Thus you either need to apply or estimate (and apply) a 2D similarity transformation for image alignment/registration. See also the response on this question: face alignment algorithm on images

From rotation angle and to new point locations: You can define a 2D rotation matrix R and transform your point coordinates with it.

From point correspondences between shape A and Shape B to rotation: Estimate a 2D similarity transform (image alignment) using 3 or more matching points.

From either rotation or point correspondences to warped image: From the similarity transform, map image values (accounting for interpolation or non-values) using the underlying coordinate transformation for the entire image grid.

point-constrained image warping (image courtesy of Denis Simakov, AAM Slides)

Most of these are already implemented in OpenCV and MATLAB. See also the background and relevant methods around Active Shape and Active Appearance Models (Tim Cootes page includes binaries and background material).

Community
  • 1
  • 1
gevang
  • 4,994
  • 25
  • 33