hide
Free keywords:
Computer Science, Computer Vision and Pattern Recognition, cs.CV
Abstract:
While dense non-rigid structure from motion (NRSfM) has been extensively
studied from the perspective of the reconstructability problem over the recent
years, almost no attempts have been undertaken to bring it into the practical
realm. The reasons for the slow dissemination are the severe ill-posedness,
high sensitivity to motion and deformation cues and the difficulty to obtain
reliable point tracks in the vast majority of practical scenarios. To fill this
gap, we propose a hybrid approach that extracts prior shape knowledge from an
input sequence with NRSfM and uses it as a dynamic shape prior for sequential
surface recovery in scenarios with recurrence. Our Dynamic Shape Prior
Reconstruction (DSPR) method can be combined with existing dense NRSfM
techniques while its energy functional is optimised with stochastic gradient
descent at real-time rates for new incoming point tracks. The proposed
versatile framework with a new core NRSfM approach outperforms several other
methods in the ability to handle inaccurate and noisy point tracks, provided we
have access to a representative (in terms of the deformation variety) image
sequence. Comprehensive experiments highlight convergence properties and the
accuracy of DSPR under different disturbing effects. We also perform a joint
study of tracking and reconstruction and show applications to shape compression
and heart reconstruction under occlusions. We achieve state-of-the-art metrics
(accuracy and compression ratios) in different scenarios.
