English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT

Released

Conference Paper

DECO: Dense Estimation of 3D Human-Scene Contact in the Wild

MPS-Authors
/persons/resource/persons277652

Tripathi,  Shashank
Dept. Perceiving Systems, Max Planck Institute for Intelligent Systems, Max Planck Society;

/persons/resource/persons306330

Chatterjee,  Agniv
Dept. Perceiving Systems, Max Planck Institute for Intelligent Systems, Max Planck Society;

/persons/resource/persons232314

Passy,  Jean-Claude
Central Scientific Facility Software Workshop, Max Planck Institute for Intelligent Systems, Max Planck Society;

/persons/resource/persons288947

Yi,  Hongwei
Dept. Perceiving Systems, Max Planck Institute for Intelligent Systems, Max Planck Society;

/persons/resource/persons75293

Black,  Michael J.       
Dept. Perceiving Systems, Max Planck Institute for Intelligent Systems, Max Planck Society;

Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)
There are no public fulltexts stored in PuRe
Supplementary Material (public)
There is no public supplementary material available
Citation

Tripathi, S., Chatterjee, A., Passy, J.-C., Yi, H., Tzionas, D., & Black, M. J. (2024). DECO: Dense Estimation of 3D Human-Scene Contact in the Wild. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 7967-7979). New York, NY: IEEE. doi:10.1109/ICCV51070.2023.00735.


Cite as: https://hdl.handle.net/21.11116/0000-0010-9C6F-3
Abstract
Understanding how humans use physical contact to interact with the world is key to enabling human-centric artificial intelligence. While inferring 3D contact is crucial for modeling realistic and physically-plausible human-object interactions, existing methods either focus on 2D, consider body joints rather than the surface, use coarse 3D body regions, or do not generalize to in-the-wild images. In contrast, we focus on inferring dense, 3D contact between the full body surface and objects in arbitrary images. To achieve this, we first collect DAMON, a new dataset containing dense vertex-level contact annotations paired with RGB images containing complex human-object and human-scene contact. Second, we train DECO, a novel 3D contact detector that uses both body-part-driven and scene-context-driven attention to estimate vertex-level contact on the SMPL body. DECO builds on the insight that human observers recognize contact by reasoning about the contacting body parts, their proximity to scene objects, and the surrounding scene context. We perform extensive evaluations of our detector on DAMON as well as on the RICH and BEHAVE datasets. We significantly outperform existing SOTA methods across all benchmarks. We also show qualitatively that DECO generalizes well to diverse and challenging real-world human interactions in natural images. The code, data, and models are available at https://deco.is.tue.mpg.de/login.php.