Are Facial Motion Cues Sufficient for Recognizing Facial Expressions?

Poster Session B - Sunday, April 14, 2024, 8:00 – 10:00 am EDT, Sheraton Hall ABC

Natalia K. Pallis-Hassani¹ (natalia.pallis-hassani@nih.gov), Shruti Japee, Maryam Vaziri-Pashkam, Christopher I. Baker; ¹National Institutes of Health

Our ability to extract meaning from facial expressions is critical for our social interactions. Recent work has proposed a third visual pathway specialized for motion processing, which may be vital for understanding facial expressions. While prior research on facial expression perception has typically used static facial expressions, dynamic facial motion, particularly of eyes and mouth, has recently been shown to improve recognition of facial expressions. Thus, we are interested in examining whether facial motion cues—in the absence of underlying facial features—are sufficient for recognizing facial expressions. To answer this question, we converted dynamic video stimuli of various facial expressions into random dot kinematograms (RDKs) using the underlying optic flow information in the videos. These videos include seven facial expressions—happy, sad, angry, disgusted, fearful, surprised, and neutral—across 22 actors. In a pilot experiment, participants were shown each of the 154 RDK stimuli and asked to label the facial expression. Preliminary analyses of these data revealed above-chance labeling accuracy for most expressions. Results from this study will provide insight into how we recognize facial expressions from facial motion cues in the absence of facial features. In a future experiment, RDK videos will also be used to examine the decoding of movement in facial expressions using functional magnetic resonance imaging (fMRI) or magnetoencephalography (MEG).

Topic Area: EMOTION & SOCIAL: Person perception