Abstract
Understanding how semantic information is represented in the brain has been an important research
focus of neuroscience in the past few years. Unlike 'traditional' neural (de)coding approaches, which
study the relationship between stimulus and neural response, we are interested in higher-order relational
coding: we ask how perceived relationships between stimuli (e.g. similarity) are connected to
corresponding relationships in the neural activity. Our approach addresses the semantical problem, i.e.
how terms (here stimuli) come to have their (possibly subjective) meaning, from the perspective of the
network theory of semantics (Churchland 1984). This theory posits that meaning arises from the network
of concepts within which a given term is embedded.
We showed previously (Endres et al 2010, AMAI) that Formal Concept Analysis (FCA, (Ganter & Wille
1999)) can reveal interpretable semantic information (e.g. specialization hierarchies, or feature-based
representation) from electrophysiological data. Unlike other analysis methods (e.g. hierarchical
clustering), FCA does not impose inappropriate structure on the data. FCA is a mathematical formulation
of the explicit coding hypothesis (Foldiak, 2009, Curr. Biol.)
Here, we investigate whether similar findings can be obtained from fMRI BOLD responses recorded from
human subjects. While the BOLD response provides only an indirect measure of neural activity on a
much coarser spatio-temporal scale than electrophysiological recordings, it has the advantage that it can
be recorded from humans, which can be questioned about their perceptions during the experiment,
thereby obviating the need of interpreting animal behavioural responses. Furthermore, the BOLD signal
can be recorded from the whole brain simultaneously.
In our experiment, a single human subject was scanned while viewing 72 grayscale pictures of animate
and inanimate objects in a target detection task (Siemens Trio 3T scanner, GE-EPI, TE=40ms, 38 axial
slices, TR=3.08s, 48 sessions, amounting to a total of 10,176 volume images). These pictures comprise
the formal objects for FCA. We computed formal attributes by learning a hierarchical Bayesian classifier,
which maps BOLD responses onto binary features, and these features onto object labels. The
connectivity matrix between the binary features and the object labels can then serve as the formal
context.
In line with previous reports, FCA revealed a clear dissociation between animate and inanimate objects in
a high-level visual area (inferior temporal cortex, IT), with the inanimate category including plants. The
inanimate category was subdivided into plants and non-plants when we increased the number of
attributes extracted from the fMRI responses. FCA also highlighted organizational differences between
the IT and the primary visual cortex, V1. We show that subjective familiarity and similarity ratings are
strongly correlated with the attribute structure computed from the fMRI signal (Endres et al. 2012,
ICFCA).
