Listeners as spectators? Audio-visual integration improves music performer identification

Helen Mitchell, Raymond MacDonald

Research output: Contribution to journalArticlepeer-review

Abstract

Listeners take for granted their capacity to distinguish between musical instruments, and their ability to discriminate between performers playing the same instrument by their sound alone. Sound perception is usually considered a purely auditory process, but there is significant debate on how auditory and visual information are integrated during listening. Two experiments examined how listeners perceive individual performers. Saxophonists (n = 5) performed three jazz standards for an audio and video recording. Experiment 1 investigated listeners’ ability to identify individual saxophonists by ear. Listeners heard one saxophonist, and were later asked to identify him from a line- up of saxophonists. Only 52% of listeners could correctly identify their target. Experiment 2 explored the integration of cross-modal sensory experiences (audio and visual) in saxophonist identification. Participants either watched a silent video clip of a saxophonist playing and matched it to an audio clip of the same performer, or heard an audio clip of a saxophonist and matched it to a silent video clip. Listener/viewers reliably identified their target saxophonists, and results suggest that listeners combine cross-modal perceptions to identify individual performers, and can use the information about a performer in one modality and match it to the same performer in another modality.
Original languageEnglish
Pages (from-to)112-127
Number of pages15
JournalPsychology of Music
Volume42
Issue number1
DOIs
Publication statusPublished - 2014

Keywords

  • audiovisual integration
  • auditory recognition
  • music perception
  • music performance
  • visual recognition

Fingerprint

Dive into the research topics of 'Listeners as spectators? Audio-visual integration improves music performer identification'. Together they form a unique fingerprint.

Cite this