3D Face Reconstruction from Occluded Images

Abstract

Current methods for reconstructing 3D faces from regular images have some problems. They struggle to create realistic animated faces because they don’t account for how wrinkles change with different expressions. They also have trouble working with images taken in real-world conditions as images are likely to be occluded and exposed to extreme condition. A new approach is implemented that can predict the shape of a face in 3D which has been blocked by different factor like hand, eye glass, mask etc. Context based learning knowledge distillation is used to transfer the knowledge from main DECA model to learner model. The learner model is also trained to handle different occlusion thus helping in constructing 3D faces. This is done from normal pictures without needing special 3D information and it works really well, producing accurate results. It achieves state-of-the-art shape reconstruction accuracy on NoW benchmarks.