English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT

Released

Paper

LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration

MPS-Authors

Bhatnagar,  Bharat Lal
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society;

/persons/resource/persons45610

Theobalt,  Christian       
Computer Graphics, MPI for Informatics, Max Planck Society;

Pons-Moll,  Gerard
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society;

External Resource
No external resources are shared
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)

arXiv:2010.12447.pdf
(Preprint), 11MB

Supplementary Material (public)
There is no public supplementary material available
Citation

Bhatnagar, B. L., Sminchisescu, C., Theobalt, C., & Pons-Moll, G. (2020). LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration. Retrieved from https://arxiv.org/abs/2010.12447.


Cite as: https://hdl.handle.net/21.11116/0000-0007-E91C-4
Abstract
We address the problem of fitting 3D human models to 3D scans of dressed
humans. Classical methods optimize both the data-to-model correspondences and
the human model parameters (pose and shape), but are reliable only when
initialized close to the solution. Some methods initialize the optimization
based on fully supervised correspondence predictors, which is not
differentiable end-to-end, and can only process a single scan at a time. Our
main contribution is LoopReg, an end-to-end learning framework to register a
corpus of scans to a common 3D human model. The key idea is to create a
self-supervised loop. A backward map, parameterized by a Neural Network,
predicts the correspondence from every scan point to the surface of the human
model. A forward map, parameterized by a human model, transforms the
corresponding points back to the scan based on the model parameters (pose and
shape), thus closing the loop. Formulating this closed loop is not
straightforward because it is not trivial to force the output of the NN to be
on the surface of the human model - outside this surface the human model is not
even defined. To this end, we propose two key innovations. First, we define the
canonical surface implicitly as the zero level set of a distance field in R3,
which in contrast to morecommon UV parameterizations, does not require cutting
the surface, does not have discontinuities, and does not induce distortion.
Second, we diffuse the human model to the 3D domain R3. This allows to map the
NN predictions forward,even when they slightly deviate from the zero level set.
Results demonstrate that we can train LoopRegmainly self-supervised - following
a supervised warm-start, the model becomes increasingly more accurate as
additional unlabelled raw scans are processed. Our code and pre-trained models
can be downloaded for research.