English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement

Zhou, K., Lal Bhatnagar, B., Lenssen, J. E., & Pons-Moll, G. (2022). TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement. Retrieved from https://arxiv.org/abs/2205.07982.

Item is

Files

show Files
hide Files
:
arXiv:2205.07982.pdf (Preprint), 10MB
Name:
arXiv:2205.07982.pdf
Description:
File downloaded from arXiv at 2022-07-05 07:43
OA-Status:
Visibility:
Public
MIME-Type / Checksum:
application/pdf / [MD5]
Technical Metadata:
Copyright Date:
-
Copyright Info:
-

Locators

show

Creators

show
hide
 Creators:
Zhou, Keyang1, Author           
Lal Bhatnagar, Bharat1, Author           
Lenssen, Jan Eric1, Author           
Pons-Moll, Gerard1, Author           
Affiliations:
1Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society, ou_1116547              

Content

show
hide
Free keywords: Computer Science, Computer Vision and Pattern Recognition, cs.CV
 Abstract: We present TOCH, a method for refining incorrect 3D hand-object interaction sequences using a data prior. Existing hand trackers, especially those that rely on very few cameras, often produce visually unrealistic results with hand-object intersection or missing contacts. Although correcting such errors requires reasoning about temporal aspects of interaction, most previous work focus on static grasps and contacts. The core of our method are TOCH fields, a novel spatio-temporal representation for modeling correspondences between hands and objects during interaction. The key component is a point-wise object-centric representation which encodes the hand position relative to the object. Leveraging this novel representation, we learn a latent manifold of plausible TOCH fields with a temporal denoising auto-encoder. Experiments demonstrate that TOCH outperforms state-of-the-art (SOTA) 3D hand-object interaction models, which are limited to static grasps and contacts. More importantly, our method produces smooth interactions even before and after contact. Using a single trained TOCH model, we quantitatively and qualitatively demonstrate its usefulness for 1) correcting erroneous reconstruction results from off-the-shelf RGB/RGB-D hand-object reconstruction methods, 2) de-noising, and 3) grasp transfer across objects. We will release our code and trained model on our project page at http://virtualhumans.mpi-inf.mpg.de/toch/

Details

show
hide
Language(s): eng - English
 Dates: 2022-05-162022
 Publication Status: Published online
 Pages: 19 p.
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: arXiv: 2205.07982
URI: https://arxiv.org/abs/2205.07982
BibTex Citekey: Zhou_2205.07982
 Degree: -

Event

show

Legal Case

show

Project information

show

Source

show