Learning Multisensory Representations

Abstract : Peoples everyday experiences are multisensory. For example, while eating breakfast, we both see and grasp our coffee cup. Moreover, multisensory perception is critical in some highly important situations, as when a TSA agent searches a passengers bag by both looking inside the bag and touching its contents or when a police officer frisks a person using both visual and tactile inspection. In brief, our research program uses experimental and computational methodologies to study how people acquire multisensory representations and how the use of these representations influences perceptual judgements and decision making. The program focuses on peoples performances in visual-haptic and visual-auditory environments. Funding from this grant supported research reported in 7 journal publications and 3 conference publications (manuscripts based on 2 of the conference publications are currently being prepared for journal submission).