Item

Abstract

Capturing general deforming scenes is crucial for many computer graphics and
vision applications, and it is especially challenging when only a monocular RGB
video of the scene is available. Competing methods assume dense point tracks,
3D templates, large-scale training datasets, or only capture small-scale
deformations. In contrast to those, our method, Ub4D, makes none of these
assumptions while outperforming the previous state of the art in challenging
scenarios. Our technique includes two new, in the context of non-rigid 3D
reconstruction, components, i.e., 1) A coordinate-based and implicit neural
representation for non-rigid scenes, which enables an unbiased reconstruction
of dynamic scenes, and 2) A novel dynamic scene flow loss, which enables the
reconstruction of larger deformations. Results on our new dataset, which will
be made publicly available, demonstrate the clear improvement over the state of
the art in terms of surface reconstruction accuracy and robustness to large
deformations. Visit the project page https://4dqv.mpi-inf.mpg.de/Ub4D/.