English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT

Released

Conference Paper

GAN2X: Non-Lambertian Inverse Rendering of Image GANs

MPS-Authors
/persons/resource/persons282904

Pan,  Xingang
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society;

/persons/resource/persons206546

Tewari,  Ayush
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society;

/persons/resource/persons226679

Liu,  Lingjie
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society;

/persons/resource/persons45610

Theobalt,  Christian       
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society;

External Resource
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)

arXiv:2206.09244.pdf
(Preprint), 4MB

Supplementary Material (public)
There is no public supplementary material available
Citation

Pan, X., Tewari, A., Liu, L., & Theobalt, C. (2023). GAN2X: Non-Lambertian Inverse Rendering of Image GANs. In International Conference on 3D Vision (pp. 711-721). Piscataway, NJ: IEEE. doi:10.1109/3DV57658.2022.00081.


Cite as: https://hdl.handle.net/21.11116/0000-000B-9C8A-A
Abstract
2D images are observations of the 3D physical world depicted with the
geometry, material, and illumination components. Recovering these underlying
intrinsic components from 2D images, also known as inverse rendering, usually
requires a supervised setting with paired images collected from multiple
viewpoints and lighting conditions, which is resource-demanding. In this work,
we present GAN2X, a new method for unsupervised inverse rendering that only
uses unpaired images for training. Unlike previous Shape-from-GAN approaches
that mainly focus on 3D shapes, we take the first attempt to also recover
non-Lambertian material properties by exploiting the pseudo paired data
generated by a GAN. To achieve precise inverse rendering, we devise a
specularity-aware neural surface representation that continuously models the
geometry and material properties. A shading-based refinement technique is
adopted to further distill information in the target image and recover more
fine details. Experiments demonstrate that GAN2X can accurately decompose 2D
images to 3D shape, albedo, and specular properties for different object
categories, and achieves the state-of-the-art performance for unsupervised
single-view 3D face reconstruction. We also show its applications in downstream
tasks including real image editing and lifting 2D GANs to decomposed 3D GANs.