Switching modalities in a sentence verification task: ERP evidence for embodied language processing

• Main Authors: Lea Ann Hald, Julie-Ann eMarshall, Dirk Paul Janssen, Alan eGarnham
• Format: Article
• Language: English
• Published: Frontiers Media S.A. 2011-03-01
• Series: Frontiers in Psychology
• Subjects: ERP; language processing; embodiment; N400; modality; modality switch effect
• Online Access: http://journal.frontiersin.org/Journal/10.3389/fpsyg.2011.00045/full

In an ERP experiment using written language materials only, we investigated a potential modulation of the N400 by the Modality Switch Effect. The Modality Switch Effect occurs when a first sentence, describing a fact grounded in one modality, is followed by a second sentence describing a second fact grounded in a different modality. For example, A cellar is dark (visual), was preceded by either another visual property Ham is pink or by a tactile property A mitten is soft. We also investigated whether the Modality Switch Effect occurs for false sentences (A cellar is light). We found that, for true sentences, the ERP at the underlined critical word dark elicited a significantly greater frontal, early N400-like effect (270-370 ms) when there was a modality mismatch than when there was a modality match. This pattern was not found for the false sentences. Results similar to the frontal negativity were obtained in a late time window (500-700 ms). The obtained ERP effect is similar to one previously obtained for pictures. We conclude that in this paradigm we obtained fast access to conceptual properties for modality matched pairs, which leads to embodiment effects similar to those previously obtained with pictorial stimuli.