Cross-modal associations and synaesthesia: Categorical perception and structure in vowel-colour mappings in a large online sample

Christine Cuskley, Mark Dingemanse, Simon Kirby, Tessa M van Leeuwen

Research output: Contribution to journalArticlepeer-review

Abstract

We report associations between vowel sounds, graphemes, and colours collected online 20 from over 1000 Dutch speakers. We provide open materials including a Pythonimplementation of the structure measure, and code for a single page web application to 22 run simple cross-modal tasks. We also provide a full dataset of colour-vowel associations 23 from 1164 participants, including over 200 syanesthetes identified using consistency measures. Our analysis reveals salient patterns in cross-modal associations, and introduces a novel measure of isomorphism in cross-modal mappings. We find that while acoustic features of vowels significantly predict certain mappings (replicating prior 27 work), both vowel phoneme category and grapheme category are even better predictors 28 of colour choice. Phoneme category is the best predictor of colour choice overall,pointing to the importance of phonological representations in addition to acoustic cues. 30 Generally, high/front vowels are lighter, more green, and more yellow than low/back 31 vowels. Synaesthetes respond more strongly on some dimensions, choosing lighter and 32 more yellow colours for high and mid front vowels than non-synaesthetes. We alsopresent a novel measure of cross-modal mappings adapted from ecology, which uses a 34 simulated distribution of mappings to measure the extent to which participants’ actual 35 mappings are structured isomorphically across modalities. Synaesthetes have mappings 36 that tend to be more structured than non-synaesthetes, and more consistent colour choices across trials correlate with higher structure scores. Nevertheless, the large majority (~70%) of participants produce structured mappings, indicating that the capacity to make isomorphically structured mappings across distinct modalities is shared to a large extent, even if the exact nature of mappings varies across individuals. 41 Overall, this novel structure measure suggests a distribution of structured cross-modal 42 association in the population, with synaesthetes on one extreme and participants with 43 unstructured associations on the other.
Original languageEnglish
Pages (from-to)1651–1675
JournalBehavior Research Methods
Volume51
Issue number4
Early online date3 Apr 2019
DOIs
Publication statusPublished - Aug 2019

Keywords

  • synaesthesia
  • cross-modal correspondences
  • language
  • sound symbolism

Fingerprint

Dive into the research topics of 'Cross-modal associations and synaesthesia: Categorical perception and structure in vowel-colour mappings in a large online sample'. Together they form a unique fingerprint.

Cite this