Automatic Generation and Stylization of 3D Facial Rigs

PubDate: August 2019

Teams: Techna；Universiteit Utrecht；ESIR

Writers: Fabien Danieau; Ilja Gubins; Nicolas Olivier; Olivier Dumas; Bernard Denis; Thomas Lopez; Nicolas Mollet; Brian Frager; Quentin Avril

PDF: Automatic Generation and Stylization of 3D Facial Rigs

Abstract

In this paper, we present a fully automatic pipeline for generating and stylizing high geometric and textural quality facial rigs. They are automatically rigged with facial blendshapes for animation, and can be used across platforms for applications including virtual reality, augmented reality, remote collaboration, gaming and more. From a set of input facial photos, our approach is to be able to create a photorealistic, fully rigged character in less than seven minutes. The facial mesh reconstruction is based on state-of-the art photogrammetry approaches. Automatic landmarking coupled with ICP registration with regularization provide direct correspondence and registration from a given generic mesh to the acquired facial mesh. Then, using deformation transfer, existing blendshapes are transferred from the generic to the reconstructed facial mesh. The reconstructed face is then fit to the full body generic mesh. Extra geometry such as jaws, teeth and nostrils are retargeted and transferred to the character. An automatic iris color extraction algorithm is performed to colorize a separate eye texture, animated with dynamic UVs. Finally, an extra step applies a style to the photorealis-tic face to enable blending of personalized facial features into any other character. The user’s face can then be adapted to any human or non-human generic mesh. A pilot user study was performed to evaluate the utility of our approach. Up to 65% of the participants were successfully able to discern the presence of one’s unique facial features when the style was not too far from a humanoid shape.