Item

Abstract

We present a novel method for multi-view depth estimation from a single
video, which is a critical task in various applications, such as perception,
reconstruction and robot navigation. Although previous learning-based methods
have demonstrated compelling results, most works estimate depth maps of
individual video frames independently, without taking into consideration the
strong geometric and temporal coherence among the frames. Moreover, current
state-of-the-art (SOTA) models mostly adopt a fully 3D convolution network for
cost regularization and therefore require high computational cost, thus
limiting their deployment in real-world applications. Our method achieves
temporally coherent depth estimation results by using a novel Epipolar
Spatio-Temporal (EST) transformer to explicitly associate geometric and
temporal correlation with multiple estimated depth maps. Furthermore, to reduce
the computational cost, inspired by recent Mixture-of-Experts models, we design
a compact hybrid network consisting of a 2D context-aware network and a 3D
matching network which learn 2D context information and 3D disparity cues
separately. Extensive experiments demonstrate that our method achieves higher
accuracy in depth estimation and significant speedup than the SOTA methods.