Touch design and narrative interpretation: A social semiotic approach to picture book apps

Sumin Zhao, Len Unsworth

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract / Description of output

Interactivity is often considered as the defining feature of a book app. However, what constitutes interactivity in the context of apps is not always clear. We postulate in this chapter a multimodal social semiotic theory of interactivity, which considers interactivity as not simply a function of technology, but also a resource for meaning making. We distinguish two basic types of interactivity—intra-text and extra-text—incorporated in the touch design, and explore the different functions they perform in a broad range of picture book apps. In particular, we look at the app version of The Heart and the Bottle in depth, and illustrate how interactive design elements help to create an interpretative possibility of the story. We suggest that a better understanding of interactive touch design would promote more effective adult-child interactions around mobile applications.
Original languageEnglish
Title of host publicationApps, Technology and Younger Learners
EditorsNatalia Kucirkova, Garry Falloon
Place of PublicationUnited Kingdom
Number of pages12
ISBN (Electronic)9781315682204
ISBN (Print)9781138927889, 9781138927872
Publication statusPublished - 2016

Keywords / Materials (for Non-textual outputs)

  • picturebooks
  • narrative forms
  • interactivity
  • mobile applications


Dive into the research topics of 'Touch design and narrative interpretation: A social semiotic approach to picture book apps'. Together they form a unique fingerprint.

Cite this