hide
Free keywords:
Computer Science, Graphics, cs.GR,Computer Science, Computer Vision and Pattern Recognition, cs.CV
Abstract:
Foveated image reconstruction recovers full image from a sparse set of
samples distributed according to the human visual system's retinal sensitivity
that rapidly drops with eccentricity. Recently, the use of Generative
Adversarial Networks was shown to be a promising solution for such a task as
they can successfully hallucinate missing image information. Like for other
supervised learning approaches, also for this one, the definition of the loss
function and training strategy heavily influences the output quality. In this
work, we pose the question of how to efficiently guide the training of foveated
reconstruction techniques such that they are fully aware of the human visual
system's capabilities and limitations, and therefore, reconstruct visually
important image features. Due to the nature of GAN-based solutions, we
concentrate on the human's sensitivity to hallucination for different input
sample densities. We present new psychophysical experiments, a dataset, and a
procedure for training foveated image reconstruction. The strategy provides
flexibility to the generator network by penalizing only perceptually important
deviations in the output. As a result, the method aims to preserve perceived
image statistics rather than natural image statistics. We evaluate our strategy
and compare it to alternative solutions using a newly trained objective metric
and user experiments.
