Cross-Modal MVPA Reveals Common Brain Representations of Action and Perception of Newly Learned Melodies

Poster Session D - Monday, April 15, 2024, 8:00 – 10:00 am EDT, Sheraton Hall ABC

Fiona (Yu-Hsin) Chang^1,2, Fredrik Ullén^1,2, Örjan de Manzano^1,2; ¹Max Planck Institute for Empirical Aesthetics, ²Karolinska Institutet

Playing a musical instrument requires not only the control of finger movements but also coordination with auditory feedback. Studies indicate that with training, a seemingly inseparable relationship can be established between action and perception. This is known as action-perception coupling. Supporting evidence shows that when musicians observe a piano-playing hand silently, there are nonetheless activations in the auditory cortex. Moreover, our own previous work illustrated that when non-musicians listened to melodies they had learned to play, the corresponding patterns of activity in the superior temporal gyrus and dorsal premotor cortex could be used to distinguish between them. To extend on these previous results, we here used fMRI to investigate whether brain representations are similar when non-musicians (a) play two learned melodies on the piano while imagining the sound and (b) listen to the same melodies while imagining the finger movements. In total, 22 participants with no previous musical training participated in the experiment. Conjunction analysis revealed that both auditory and motor-related regions were active in both conditions. Multivariate pattern analysis (MVPA) showed that a classifier could differentiate between the brain patterns associated with the melodies in both the superior temporal gyrus and premotor cortex during the listening condition. However, only activity patterns in the premotor cortex distinguished the two melodies when the participants were playing. Additionally, cross-modal classification showed that representations generalized across conditions in the right premotor cortex. Notably, there were substantial individual differences in classification accuracies across regions and tasks.

Topic Area: PERCEPTION & ACTION: Multisensory