Poster B51, Sunday, March 25, 8:00-10:00 am, Exhibit Hall C
Semantic processing of self-adaptors, emblems, and iconic gestures: An ERP study
Kawai Chui1, Chia-Ying Lee2,3, Kanyu Yeh1, Pei-Chun Chao3; 1National Chengchi University, Taiwan, 2Academia Sinica, Taiwan, 3National Yang-Ming University, Taiwan
The study investigates how the brain processes self-adaptors, emblems, and iconic gestures along with speech. The three types of gestures give rise to a continuum of semantic distinctions in relation to the accompanying speech. The overall N400 component occurred between 500 and 800 msec after the simultaneous gesture and speech onsets. In comparison to the speech-only condition, the reduced N400 evidenced the facilitation effect of iconic gestures at the centro-parietal sites. The meaningful yet non-speech-related emblems elicited enhanced N400s at the left frontal-parietal sites; the meaningless self-adaptors produced the largest N400 effect over the scalp at the frontal-parietal sites. Self-adaptors had a larger negativity of N400 than emblems did at the centro-parietal regions. The results evidence the automatic integration of gesture and speech, and the diverse influence of gesture on processing. Only iconic gestures facilitate the semantic integration with speech. For a linguistic meaning to integrate with a speech-unrelated emblem is less effortful than with a meaningless self-adaptor, as pragmatic or world knowledge would be needed for the processing of self-adaptors.
Topic Area: LANGUAGE: Semantic