Matching pictures and signs: An ERP study of the effects of iconic structural alignment in American sign language

• Main Authors: Emmorey, K. (Author), Holcomb, P.J (Author), Massa, N. (Author), McGarry, M.E (Author), Midgley, K.J (Author), Mott, M. (Author)
• Format: Article
• Language: English
• Published: Elsevier Ltd 2021
• Subjects: accuracy; adult; Article; clinical article; comprehension; controlled study; Deaf; Deafness; decision making; electroencephalography; Electroencephalography; English (language); ERPs; event related potential; Evoked Potentials; evoked response; female; Female; hearing impaired person; hearing impairment; human; Humans; Iconicity; language; Language; language processing; male; Male; reaction time; semantics; Semantics; sign language; Sign language; Sign Language; Structural alignment; United States
• Online Access: View Fulltext in Publisher

 Event-related potentials (ERPs) were used to explore the effects of iconicity and structural visual alignment between a picture-prime and a sign-target in a picture-sign matching task in American Sign Language (ASL). Half the targets were iconic signs and were presented after a) a matching visually-aligned picture (e.g., the shape and location of the hands in the sign COW align with the depiction of a cow with visible horns), b) a matching visually-nonaligned picture (e.g., the cow's horns were not clearly shown), and c) a non-matching picture (e.g., a picture of a swing instead of a cow). The other half of the targets were filler signs. Trials in the matching condition were responded to faster than those in the non-matching condition and were associated with smaller N400 amplitudes in deaf ASL signers. These effects were also observed for hearing non-signers performing the same task with spoken-English targets. Trials where the picture-prime was aligned with the sign target were responded to faster than non-aligned trials and were associated with a reduced P3 amplitude rather than a reduced N400, suggesting that picture-sign alignment facilitated the decision process, rather than lexical access. These ERP and behavioral effects of alignment were found only for the ASL signers. The results indicate that iconicity effects on sign comprehension may reflect a task-dependent strategic use of iconicity, rather than facilitation of lexical access. © 2021 The Authors