Introduction
In this project, I explored facial morphing techniques to transform one face into another, compute the average face of a population, and create caricatures by extrapolating from the mean. These techniques have practical applications in computer graphics, animation, and facial recognition, allowing for smooth transitions between faces and deeper analysis of facial features.
Part 1: Defining Correspondences
To perform face morphing from me to my friend Lily, I need to first align the two images by labeling the corresponding points. I cropped the images and resized them to be exact same size, and used this online tool to label the points. I then calculated the Delunay triangulation of the points using the average of two sets of points.
Part 2: The "Mid-way Face"
To compute the mid-way face:
- 1. Compute the average shape by taking the average of corresponding points between the two images.
- 2. For each triangle in the average shape, compute the inverse affine transformation between the original triangle and the mean triangle. Using polygon from scikit image to select the relevant points, inverse warp to find out the relevant pixels in the original image (use bilinear interpolation)
- 3. Cross dissolve to find the average color between two triangles.
Part 3: The Morph Sequence
Using the same method as last part, I created a morph function that takes two fraction parameters warp_frac and dissolve_frac, which respectively control the amount of warping and cross dissolving. Both parameters lies within [0,1]. Using a list of different parameters, with 0 being the original start image and 1 being the target image, I generated 45 frames of a morph sequence, and saved them as a gif.
Part 4: The Mean Face of A Population
In this part, I used the FEI dataset (https://fei.edu.br/~cet/facedatabase.html) to compute the average face shape of the population. The FEI contains two subsets of the population: one with expression and one without expression. Below are the average appearance (without morphing) and the average shape of the dataset (for no expression subpopulation).
Next, I morphed all faces from the dataset into the average shape calculated. Here are some examples of faces morphed into the average shape, with the original on the left and the morphed result on the right.
I computed the mean face for the dataset without expression by taking the average of all images warped into the average shape.
Below is my face morphed into the average face geometry without expression, as well as the average face warped into my geometry.
Part 5: Caricatures: Extrapolating from the Mean
In this part, I created caricatures by extrapolating from the mean no-expression face from the previous part. I used different alphas, with negative alphas (less than -1) aiming to exaggerate characteristics from the no-expression dataset. Specifically, I aimed to "decrease expression" from my face, and we can see that my smile was indeed reduced when alpha is less than -1.
Part 6: Bells and Whistles
Change of Gender (BW1)
In this part, I attempted to change the gender of my friend Lily. I used the average Chinese male face from here. I have tried to blend only the shape, only the appearance, and both shape and appearance. For blending both the shape and appearances, I played around with several different parameters contolling how much shape and appearance should be mixed. It turns out result seem better when I use slightly more shape from average chinese male and a little bit more appearance from lily. Below are the results, "both 3" seem to the best one. (warp frac 0.7, dissolve frac 0.5)
PCA Basis and Eigenfaces (BW2)
In this part, I applied Principal Component Analysis (PCA) to derive the key eigenfaces. I then used the eigenfaces and a randomly initialized set of weights to reconstruct a random new face from the basis. Inspired by this.
