English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  Deep Video Portraits

Kim, H., Garrido, P., Tewari, A., Xu, W., Thies, J., Nießner, M., et al. (2018). Deep Video Portraits. Retrieved from http://arxiv.org/abs/1805.11714.

Item is

Files

show Files
hide Files
:
arXiv:1805.11714.pdf (Preprint), 6MB
Name:
arXiv:1805.11714.pdf
Description:
File downloaded from arXiv at 2018-10-19 09:07 SIGGRAPH 2018, Video: https://www.youtube.com/watch?v=qc5P2bvfl44
Visibility:
Public
MIME-Type / Checksum:
application/pdf / [MD5]
Technical Metadata:
Copyright Date:
-
Copyright Info:
-

Locators

show

Creators

show
hide
 Creators:
Kim, Hyeongwoo1, Author              
Garrido, Pablo2, Author              
Tewari, Ayush1, Author              
Xu, Weipeng1, Author              
Thies, Justus2, Author              
Nießner, Matthias2, Author              
Pérez, Patrick2, Author
Richardt, Christian2, Author              
Zollhöfer, Michael2, Author              
Theobalt, Christian1, Author              
Affiliations:
1Computer Graphics, MPI for Informatics, Max Planck Society, ou_40047              
2External Organizations, ou_persistent22              

Content

show
hide
Free keywords: Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Artificial Intelligence, cs.AI,Computer Science, Graphics, cs.GR
 Abstract: We present a novel approach that enables photo-realistic re-animation of portrait videos using only an input video. In contrast to existing approaches that are restricted to manipulations of facial expressions only, we are the first to transfer the full 3D head position, head rotation, face expression, eye gaze, and eye blinking from a source actor to a portrait video of a target actor. The core of our approach is a generative neural network with a novel space-time architecture. The network takes as input synthetic renderings of a parametric face model, based on which it predicts photo-realistic video frames for a given target actor. The realism in this rendering-to-video transfer is achieved by careful adversarial training, and as a result, we can create modified target videos that mimic the behavior of the synthetically-created input. In order to enable source-to-target video re-animation, we render a synthetic target video with the reconstructed head animation parameters from a source video, and feed it into the trained network -- thus taking full control of the target. With the ability to freely recombine source and target parameters, we are able to demonstrate a large variety of video rewrite applications without explicitly modeling hair, body or background. For instance, we can reenact the full head using interactive user-controlled editing, and realize high-fidelity visual dubbing. To demonstrate the high quality of our output, we conduct an extensive series of experiments and evaluations, where for instance a user study shows that our video edits are hard to detect.

Details

show
hide
Language(s): eng - English
 Dates: 2018-05-292018
 Publication Status: Published online
 Pages: 14 p.
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: arXiv: 1805.11714
URI: http://arxiv.org/abs/1805.11714
BibTex Citekey: Kim_arXiv1805.11714
 Degree: -

Event

show

Legal Case

show

Project information

show

Source

show