English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT

Released

Paper

IMoS: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions

MPS-Authors
/persons/resource/persons265930

Dabral,  Rishabh
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society;

/persons/resource/persons239654

Golyanik,  Vladislav
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society;

/persons/resource/persons45610

Theobalt,  Christian       
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society;

External Resource
No external resources are shared
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)

arXiv:2212.07555.pdf
(Preprint), 5MB

Supplementary Material (public)
There is no public supplementary material available
Citation

Ghosh, A., Dabral, R., Golyanik, V., Theobalt, C., & Slusallek, P. (2022). IMoS: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions. Retrieved from https://arxiv.org/abs/2212.07555.


Cite as: https://hdl.handle.net/21.11116/0000-000C-72C6-4
Abstract
Can we make virtual characters in a scene interact with their surrounding
objects through simple instructions? Is it possible to synthesize such motion
plausibly with a diverse set of objects and instructions? Inspired by these
questions, we present the first framework to synthesize the full-body motion of
virtual human characters performing specified actions with 3D objects placed
within their reach. Our system takes as input textual instructions specifying
the objects and the associated intentions of the virtual characters and outputs
diverse sequences of full-body motions. This is in contrast to existing work,
where full-body action synthesis methods generally do not consider object
interactions, and human-object interaction methods focus mainly on synthesizing
hand or finger movements for grasping objects. We accomplish our objective by
designing an intent-driven full-body motion generator, which uses a pair of
decoupled conditional variational autoencoders (CVAE) to learn the motion of
the body parts in an autoregressive manner. We also optimize for the positions
of the objects with six degrees of freedom (6DoF) such that they plausibly fit
within the hands of the synthesized characters. We compare our proposed method
with the existing methods of motion synthesis and establish a new and stronger
state-of-the-art for the task of intent-driven motion synthesis. Through a user
study, we further show that our synthesized full-body motions appear more
realistic to the participants in more than 80% of scenarios compared to the
current state-of-the-art methods, and are perceived to be as good as the ground
truth on several occasions.