Help Privacy Policy Disclaimer
  Advanced SearchBrowse


  Intents and Preferences Prediction Based on Implicit Human Cues

Sattar, H. (2019). Intents and Preferences Prediction Based on Implicit Human Cues. PhD Thesis, Universität des Saarlandes, Saarbrücken. doi:10.22028/D291-28192.

Item is


show Files




Sattar, Hosnieh1, 2, Author              
Fritz, Mario1, Advisor              
Schiele, Bernt1, Referee              
Sugano, Yusuke1, Referee              
1Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society, ou_1116547              
2International Max Planck Research School, MPI for Informatics, Max Planck Society, ou_1116551              


Free keywords: -
 Abstract: Visual search is an important task, and it is part of daily human life. Thus, it has been a long-standing goal in Computer Vision to develop methods aiming at analysing human search intent and preferences. As the target of the search only exists in mind of the person, search intent prediction remains challenging for machine perception. In this thesis, we focus on advancing techniques for search target and preference prediction from implicit human cues. First, we propose a search target inference algorithm from human fixation data recorded during visual search. In contrast to previous work that has focused on individual instances as a search target in a closed world, we propose the first approach to predict the search target in open-world settings by learning the compatibility between observed fixations and potential search targets. Second, we further broaden the scope of search target prediction to categorical classes, such as object categories and attributes. However, state of the art models for categorical recognition, in general, require large amounts of training data, which is prohibitive for gaze data. To address this challenge, we propose a novel Gaze Pooling Layer that integrates gaze information into CNN-based architectures as an attention mechanism – incorporating both spatial and temporal aspects of human gaze behaviour. Third, we go one step further and investigate the feasibility of combining our gaze embedding approach, with the power of generative image models to visually decode, i.e. create a visual representation of, the search target. Forth, for the first time, we studied the effect of body shape on people preferences of outfits. We propose a novel and robust multi-photo approach to estimate the body shapes of each user and build a conditional model of clothing categories given body-shape. We demonstrate that in real-world data, clothing categories and body-shapes are correlated. We show that our approach estimates a realistic looking body shape that captures a user’s weight group and body shape type, even from a single image of a clothed person. However, an accurate depiction of the naked body is considered highly private and therefore, might not be consented by most people. First, we studied the perception of such technology via a user study. Then, in the last part of this thesis, we ask if the automatic extraction of such information can be effectively evaded. In summary, this thesis addresses several different tasks that aims to enable the vision system to analyse human search intent and preferences in real-world scenarios. In particular, the thesis proposes several novel ideas and models in visual search target prediction from human fixation data, for the first time studied the correlation between shape and clothing categories opening a new direction in clothing recommendation systems, and introduces a new topic in privacy and computer vision, aimed at preventing automatic 3D shape extraction from images.


Language(s): eng - English
 Dates: 2019-07-022019-08-092019
 Publication Status: Published in print
 Pages: X, 136 p.
 Publishing info: Saarbrücken : Universität des Saarlandes
 Table of Contents: -
 Rev. Type: -
 Identifiers: BibTex Citekey: Sattar_PhD2019
DOI: 10.22028/D291-28192
URN: urn:nbn:de:bsz:291--ds-281920
Other: hdl:20.500.11880/27625
 Degree: PhD



Legal Case


Project information