hide
Free keywords:
-
Abstract:
For the purpose of object recognition, the combination of complementary information derived from the different sensory systems (vision and touch) should result in a rich representation of
the object in memory and may consequently enhance recognition performance.
The purpose of this study is to investigate the haptic exploration behavior during bimodal
object recognition. Specifically, we asked whether the exploration behavior for recognition
depends on what modality was used during learning of an object.
Analog to [1], we designed an old/new object recognition task using novel objects built
each from 6 Lego bricks. In each condition subjects had to learn 4 novel object shapes either
haptically (H), visually (V), or combined haptically and visually (VH). In a following recognition
phase we added 4 distractor objects and subjects had to decide whether the presented
object was known (old) or unknown (new) from the learning phase. The recognition phase was
always bimodal (VH).
We tested 12 subjects in all three conditions (V-to-VH, H-to-VH, VH-to-VH) measuring
recognition performance. Subjects exploration behavior was recorded using video tape. For
technical reasons, the recordings of only 8 subjects was used for the video analysis. Therefore,
the video was cut into single clips each showing the exploration of one stimulus. These clips
were replayed in randomized order to raters, who judged the ‘hapticallity’ of subjects’ exploration
behavior on a scale with 9 possible answers between ‘subject just held and turned the
stimulus’ to ‘subject explored the stimulus with fingers’. We found ‘hapticallity’ during visualhaptic
recognition was judged largest for the exploration following haptic learning (H-to-VH),
smallest for the one following visual learning (V-to-VH), and intermediate for visuo-haptic
learning (VH-to-VH).
This suggests that subjects use a strategy when recognizing the objects in a way that they
match the exploration behavior between learning and recognition test. They use the haptic
modality for recognition when the object was learned haptically. When the object was learned
visually they predominantly use the visual modality during recognition ignoring touch. Also
when the object was learned bimodally (VH), the exploration strategy was matched between
learning and recognition test here indicating that the haptic modality was used almost exclusively
for actively manipulating the object but not for exploring its shape.
