Poster C76, Sunday, March 26, 5:00 – 7:00 pm, Pacific Concourse
The neural basis of the integration of speech and gesture: A brain stimulation approach
wanying zhao1, Kevin Riggs1, Igor Schindler1, Henning Hollo1; 1University of Hull
Previous neuro imaging studies have reported activity of the left middle temporal gyrus (pMTG) and the left Inferior frontal gyrus (LIFG) during co-speech gesture comprehension. It is currently unclear, however, which of these brain areas play a causal role in gesture-speech integration. The current study builds on the paradigm of Kelly et al., (2010) which provides a reaction time index of gesture-speech integration. Briefly, participants were presented with co-speech gestures (e.g., typing while saying 'write'), with gender and semantic congruency of audio-visual stimuli being manipulated. Based on a naming study, gestures were split into pantomimes (that can be easily understand without the involvement of speech) and iconics (that need speech to be disambiguated). In the first experiment, 5 pulses of repetitive Transcranial Magnetic Stimulation (rTMS) were delivered ‘on-line’ to left pMTG during each video item. The participant’s task was to respond to the gender of the speaker as quickly and accurately as possible. Results show pMTG is crucial for the integration of speech and pantomimes. In the second experiment, ’offline’ 40 seconds theta-burst TMS (TBS) was either applied to the left IFG, the left pMTG or a control area, followed by the experimental task. Results show that LIFG is also involved in the integration of speech and pantomime. This suggests that LIFG and pMTG both have a role of unifying meaning from different modalities when both gesture and speech have clearly defined meanings in isolation.
Topic Area: LANGUAGE: Semantic