The Visual and Beyond: Characterizing Experiences with Auditory, Haptic and Visual Data Representations

Trevor Hogan, Uta Hinrichs, Eva Hornecker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Research in sonification and physicalization have expanded data representation techniques to include senses beyond the visual. Yet, little is known of how people interpret and make sense of haptic and sonic compared to visual representations. We have conducted two phenomenologically oriented comparative studies (applying the Repertory Grid and the Micro-phenomenological interview technique) to gather in-depth accounts of people's interpretation and experience of different representational modalities that included auditory, haptic and visual variations. Our findings show a rich characterization of these different representational modalities: our visually oriented representations engage through their familiarity, accuracy and easy interpretation, while our representations that stimulated auditory and haptic interpretation were experienced as more ambiguous, yet stimulated an engaging interpretation of data that involved the whole body. We describe and discuss in detail participants' processes of making sense and generating meaning using the modalities' unique characteristics, individually and as a group. Our research informs future research in the area of multimodal data representations from both a design and methodological perspective.
Original languageEnglish
Title of host publicationProceedings of the 2017 Conference on Designing Interactive Systems
Place of PublicationNew York, NY, USA
PublisherACM Association for Computing Machinery
Pages797–809
ISBN (Electronic)9781450349222
DOIs
Publication statusPublished - 10 Jun 2017
EventACM SIGCHI Conference on Designing Interactive Systems: Space, Place and Interface: Bridging knowledge, connecting people - Edinburgh, United Kingdom
Duration: 10 Jun 201714 Jun 2017

Conference

ConferenceACM SIGCHI Conference on Designing Interactive Systems
Abbreviated titleDIS'17
Country/TerritoryUnited Kingdom
CityEdinburgh
Period10/06/1714/06/17

Keywords / Materials (for Non-textual outputs)

  • micro-phenomenological interview
  • phenomenology
  • sonification
  • qualitative evaluation
  • embodiment
  • physicalisation
  • repgrid
  • data interpretation

Fingerprint

Dive into the research topics of 'The Visual and Beyond: Characterizing Experiences with Auditory, Haptic and Visual Data Representations'. Together they form a unique fingerprint.

Cite this