The effect of affect: modulation of facial expression representations by affective scenes

Poster Session F - Tuesday, April 16, 2024, 8:00 – 10:00 am EDT, Sheraton Hall ABC

Shaofeng Liu¹ (shawnliu7@gmail.com), Tyler Roberts¹, Jonathan Cant¹, Adrian Nestor¹; ¹University of Toronto

Prior research has demonstrated the perceptual benefits of viewing certain facial expressions within relevant affective contexts. However, a systematic investigation of how the affective context provided by different scenes impacts a wider range of expressions remains to be undertaken. Here, we address this challenge by assessing the structure of expression face space in the context of scenes varying in valence and level of arousal. To this aim, adult participants (n=114) with normal expression recognition ability rated the pairwise similarity of 15 different facial expressions of two male actors (e.g., happy satiated, smiling sardonic, sad, arrogant). Facial images were displayed against scenes from five affective categories (i.e., amusement, awe, disgust, fear and neutral). Further, in a control experiment, participants rated the same expression pairs against versions of the same scene images transformed to eliminate the semantic content of the scenes while preserving low-level image properties. Chiefly, our results revealed that fear-evoking contexts systematically impacted the structure of face space relative to a neutral context. While the size of this effect varied as a function of facial identity (i.e., specific male actor), the effect was traced to a common subset of expressions (i.e.., insecurity and disgust) whose representations were significantly modulated by fear contexts. Further, the effect was eliminated after removing the semantic content of scene images. In summary, our results shed light on the impact of affective context on emotional expression representations and, notably, provide evidence for the role of fear in face perception.

Topic Area: PERCEPTION & ACTION: Vision