Help Privacy Policy Disclaimer
  Advanced SearchBrowse





Face2Face: Real-time Face Capture and Reenactment of RGB Videos


Zollhöfer,  Michael
Computer Graphics, MPI for Informatics, Max Planck Society;


Theobalt,  Christian       
Computer Graphics, MPI for Informatics, Max Planck Society;

External Resource
No external resources are shared
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)

(Preprint), 7MB

Supplementary Material (public)
There is no public supplementary material available

Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2020). Face2Face: Real-time Face Capture and Reenactment of RGB Videos. Retrieved from https://arxiv.org/abs/2007.14808.

Cite as: https://hdl.handle.net/21.11116/0000-0007-E8E9-D
We present Face2Face, a novel approach for real-time facial reenactment of a
monocular target video sequence (e.g., Youtube video). The source sequence is
also a monocular video stream, captured live with a commodity webcam. Our goal
is to animate the facial expressions of the target video by a source actor and
re-render the manipulated output video in a photo-realistic fashion. To this
end, we first address the under-constrained problem of facial identity recovery
from monocular video by non-rigid model-based bundling. At run time, we track
facial expressions of both source and target video using a dense photometric
consistency measure. Reenactment is then achieved by fast and efficient
deformation transfer between source and target. The mouth interior that best
matches the re-targeted expression is retrieved from the target sequence and
warped to produce an accurate fit. Finally, we convincingly re-render the
synthesized target face on top of the corresponding video stream such that it
seamlessly blends with the real-world illumination. We demonstrate our method
in a live setup, where Youtube videos are reenacted in real time.