Item

Abstract

Implicit functions represented as deep learning approximations are powerful
for reconstructing 3D surfaces. However, they can only produce static surfaces
that are not controllable, which provides limited ability to modify the
resulting model by editing its pose or shape parameters. Nevertheless, such
features are essential in building flexible models for both computer graphics
and computer vision. In this work, we present methodology that combines
detail-rich implicit functions and parametric representations in order to
reconstruct 3D models of people that remain controllable and accurate even in
the presence of clothing. Given sparse 3D point clouds sampled on the surface
of a dressed person, we use an Implicit Part Network (IP-Net)to jointly predict
the outer 3D surface of the dressed person, the and inner body surface, and the
semantic correspondences to a parametric body model. We subsequently use
correspondences to fit the body model to our inner surface and then non-rigidly
deform it (under a parametric body + displacement model) to the outer surface
in order to capture garment, face and hair detail. In quantitative and
qualitative experiments with both full body data and hand scans we show that
the proposed methodology generalizes, and is effective even given incomplete
point clouds collected from single-view depth images. Our models and code can
be downloaded from http://virtualhumans.mpi-inf.mpg.de/ipnet.