Poster Session B, Sunday, March 24, 8:00 – 10:00 am, Pacific Concourse
Drawing Sounds: Translational features across domains
Sara Hill1, Ferrinne Spector2; 1Edgewood College
The purpose of this study is to further understand how sensory information is translated across sensory domains. Specifically, whether basic properties of an auditory stimulus consistently associate to features in a visual image created in response to the sound. This is one of the first two studies of audio-visual perception using a newly developed sound stimulus set consisting of 64 sound stimuli, each created to manipulate a single property of sound (pitch, volume, ascension, tempo, continuity), with each sound property represented across four musical instruments to capture differences in timbre. Participants listened to 30 sounds drawn randomly from the auditory stimulus set and drew their visual interpretation of each sound using crayons on white paper. Using systematic inter-rater coding, we examined each visual representation for the presence of numerous visual elements, including color lightness, angularity, continuity and reduplication of shape and lines. Overall, we found that sound features are consistently associated with related visual features, with some auditory features leading to more consistent audio-visual translation. For example, a fast tempo is frequently represented using reduplicated clusters of lines (proportion = .92), high pitch is strongly related to the color lightness (proportion = .83), and legato sounds were frequently represented with continuous lines (proportion = .82). These results highlight the value of using a standard sound set for comparison across experimental and descriptive studies and provide insight into the consistent translation of information across the visual and auditory domains.
Topic Area: PERCEPTION & ACTION: Multisensory