Document Actions
Multimodal Expertise
Lots of multimodal research have looked at the how one modality can bias another modality (such as the ventriloquist effect where the thrown voice is attibuted to the dummy since vision trumphs audition) or random pairings of modalities (number of perceived flashes can be biased by number of auditory beeps). Multimodal expertise seems to be of another flavor. Is the processing different when we hear the voice/see the face of our mother?
Lots of objects in our lives can be experienced in multiple modalities: we recognize an ambulance by sight or sound; we know our softest sweater by touch or by sight; we know our favorite meal by smell or by taste. How is the information from different modalities about the same object represented neurally? One proposal is that sensory information is processed in unimodal areas (visual processing stream, auditory processing stream, etc) and the modalities only converge later in the processing streams. Alternatively, early sensory information could bias the processing of another modality. Thus, the heart of the issue is the nature of the representation. Do we have a polysensory representation for any given object or do we have separate storehouses of auditory object components and visual object components?
The more specifc question about the role of expertise reflects the finding by von Kriegstein and colleagues that familiar voices activates an area of cortex just outside of the fusiform cortex selective for faces, as well as data from a recent Ph.D. graduate of the lab, Chun-Chia Kung, that found fusiform activation in bird experts for bird song that was just outside the area of fusiform cortex selective for bird pictures. Does special processing occur for multimodal objects for object domains of expertise?
von Kriegstein, K., Kleinschmidt, A., Sterzer, P., & Giraud, A. (2005). Interaction of face and voice areas during speaker recognition. Journal of Cognitive Neuroscience, 17(3), 367-376.
Methods.
Combination of behavioral methods to define multimodal expertise and fMRI (and perhaps ERP) to look for neural correlates of the behavioral markers.
Results and Discussion.
This project is in the planning stages. Currently, we are completely a mix of literature reviewing and some scripting.
Current and future work.
N/A
People.
Jean, Sophie, & Julia
Congrats to Jessie and Nigel!