ausblenden:
Schlagwörter:
Computer Science, Computer Vision and Pattern Recognition, cs.CV
Zusammenfassung:
Photo-real digital human avatars are of enormous importance in graphics, as
they enable immersive communication over the globe, improve gaming and
entertainment experiences, and can be particularly beneficial for AR and VR
settings. However, current avatar generation approaches either fall short in
high-fidelity novel view synthesis, generalization to novel motions,
reproduction of loose clothing, or they cannot render characters at the high
resolution offered by modern displays. To this end, we propose HDHumans, which
is the first method for HD human character synthesis that jointly produces an
accurate and temporally coherent 3D deforming surface and highly
photo-realistic images of arbitrary novel views and of motions not seen at
training time. At the technical core, our method tightly integrates a classical
deforming character template with neural radiance fields (NeRF). Our method is
carefully designed to achieve a synergy between classical surface deformation
and NeRF. First, the template guides the NeRF, which allows synthesizing novel
views of a highly dynamic and articulated character and even enables the
synthesis of novel motions. Second, we also leverage the dense pointclouds
resulting from NeRF to further improve the deforming surface via 3D-to-3D
supervision. We outperform the state of the art quantitatively and
qualitatively in terms of synthesis quality and resolution, as well as the
quality of 3D surface reconstruction.
