Item

Abstract

This paper proposes a new marker-less approach to capturing human performances
from multi-view video. Our algorithm can jointly reconstruct spatio-temporally
coherent geometry, motion and textural surface appearance of actors that
perform complex and rapid moves. Furthermore, since our algorithm is purely
meshbased and makes as few as possible prior assumptions about the type of
subject being tracked, it can even capture performances of people wearing wide
apparel, such as a dancer wearing a skirt. To serve this purpose our method
efficiently and effectively combines the power of surface- and volume-based
shape deformation techniques with a new mesh-based analysis-through-synthesis
framework. This framework extracts motion constraints from video and makes the
laser-scan of the tracked subject mimic the recorded performance. Also
small-scale time-varying shape detail is recovered by applying model-guided
multi-view stereo to refine the model surface. Our method delivers captured
performance data at higher level of detail, is highly versatile, and is
applicable to many complex types of scenes that could not be handled by
alternative marker-based or marker-free recording techniques.