Edinburgh Research Explorer

Cross-modal associations and synaesthesia: Categorical perception and structure in vowel-colour mappings in a large online sample

Research output: Contribution to journalArticle

Related Edinburgh Organisations

Open Access permissions

Open

Documents

  • Download as Adobe PDF

    Final published version, 3 MB, PDF-document

    Licence: Creative Commons: Attribution (CC-BY)

Original languageEnglish
Pages (from-to)1651–1675
JournalBehavior Research Methods
Volume51
Issue number4
Early online date3 Apr 2019
DOIs
Publication statusPublished - Aug 2019

Abstract

We report associations between vowel sounds, graphemes, and colours collected online 20 from over 1000 Dutch speakers. We provide open materials including a Pythonimplementation of the structure measure, and code for a single page web application to 22 run simple cross-modal tasks. We also provide a full dataset of colour-vowel associations 23 from 1164 participants, including over 200 syanesthetes identified using consistency measures. Our analysis reveals salient patterns in cross-modal associations, and introduces a novel measure of isomorphism in cross-modal mappings. We find that while acoustic features of vowels significantly predict certain mappings (replicating prior 27 work), both vowel phoneme category and grapheme category are even better predictors 28 of colour choice. Phoneme category is the best predictor of colour choice overall,pointing to the importance of phonological representations in addition to acoustic cues. 30 Generally, high/front vowels are lighter, more green, and more yellow than low/back 31 vowels. Synaesthetes respond more strongly on some dimensions, choosing lighter and 32 more yellow colours for high and mid front vowels than non-synaesthetes. We alsopresent a novel measure of cross-modal mappings adapted from ecology, which uses a 34 simulated distribution of mappings to measure the extent to which participants’ actual 35 mappings are structured isomorphically across modalities. Synaesthetes have mappings 36 that tend to be more structured than non-synaesthetes, and more consistent colour choices across trials correlate with higher structure scores. Nevertheless, the large majority (~70%) of participants produce structured mappings, indicating that the capacity to make isomorphically structured mappings across distinct modalities is shared to a large extent, even if the exact nature of mappings varies across individuals. 41 Overall, this novel structure measure suggests a distribution of structured cross-modal 42 association in the population, with synaesthetes on one extreme and participants with 43 unstructured associations on the other.

    Research areas

  • synaesthesia, cross-modal correspondences, language, sound symbolism

ID: 81130570