Item

Abstract

We present a new pose transfer method for synthesizing a human animation from
a single image of a person controlled by a sequence of body poses. Existing
pose transfer methods exhibit significant visual artifacts when applying to a
novel scene, resulting in temporal inconsistency and failures in preserving the
identity and textures of the person. To address these limitations, we design a
compositional neural network that predicts the silhouette, garment labels, and
textures. Each modular network is explicitly dedicated to a subtask that can be
learned from the synthetic data. At the inference time, we utilize the trained
network to produce a unified representation of appearance and its labels in UV
coordinates, which remains constant across poses. The unified representation
provides an incomplete yet strong guidance to generating the appearance in
response to the pose change. We use the trained network to complete the
appearance and render it with the background. With these strategies, we are
able to synthesize human animations that can preserve the identity and
appearance of the person in a temporally coherent way without any fine-tuning
of the network on the testing scene. Experiments show that our method
outperforms the state-of-the-arts in terms of synthesis quality, temporal
coherence, and generalization ability.