Item

Abstract

Human re-rendering from a single image is a starkly under-constrained
problem, and state-of-the-art algorithms often exhibit undesired artefacts,
such as over-smoothing, unrealistic distortions of the body parts and garments,
or implausible changes of the texture. To address these challenges, we propose
a new method for neural re-rendering of a human under a novel user-defined pose
and viewpoint, given one input image. Our algorithm represents body pose and
shape as a parametric mesh which can be reconstructed from a single image and
easily reposed. Instead of a colour-based UV texture map, our approach further
employs a learned high-dimensional UV feature map to encode appearance. This
rich implicit representation captures detailed appearance variation across
poses, viewpoints, person identities and clothing styles better than learned
colour texture maps. The body model with the rendered feature maps is fed
through a neural image-translation network that creates the final rendered
colour image. The above components are combined in an end-to-end-trained neural
network architecture that takes as input a source person image, and images of
the parametric body model in the source pose and desired target pose.
Experimental evaluation demonstrates that our approach produces higher quality
single image re-rendering results than existing methods.