アイテム詳細

要旨

The main aim of this study is to unravel the mechanisms responsible for classification and memorization of visual stimuli in the human brain using methods from machine learning. We perform a visual gender discrimination task of images from a human face database. Man and machine are studied using an embedded feedback loop methodology as explained below. In a first classification experiment the human subjects are asked to classify faces from a subset of the database according to their gender. Their responses, i.e. estimated gender, reaction time and confidence rating are recorded. The gender estimated by the subject together with a low-dimensional representation obtained using Principal Component Analysis of the presented stimuli then form the subject’s personal dataset. On this dataset, various linear separating hyperplane algorithms obtained from machine learning, Support Vector Machines, Relevance Vector Machines, mean-of-class prototype learners and K-means clustering combined with a nearest-neigbour classifier, are trained. The separating hyperplane (SH) is then computed for each of these algorithms using a possibly sparse subset of the subject’s dataset: the representations. We observe that stimuli far from the SH are classified more accurately, faster and with higher confidence than those near to the SH. A new dataset, consisting of stimuli drawn from the representations, from the dataset seen by the subject and from a subset of the database which has not been presented to the subject is created on-line. In a subsequent memory experiment the subject is asked to classify the stimuli from this dataset as seen or unseen in the first classification experiment. This step actually constitutes the feedback loop: the results of the subject are used by the machine to create a novel set of stimuli which are then presented to the subject. Here we assess the usefulness of the representations given by the learning algorithms by studying whether a given representation was actually useful for man during classification. In this case we assume this representation to be activated during classification and thus possibly memorized better and hence recognized when presented a second time. Finally, in the second classification experiment, the subject is asked to perform a gender classification experiment identical to the first one, except that all the stimuli from the database and all the representations are presented. This experiment allows to verify the essential assumption that the internal representation of the faces in the subjects is stable and, if not, to see whether the “unstable” ones lie close to the SH of one of the algorithms.