Touch design and narrative interpretation: A social semiotic approach to picture book apps

Interactivity is often considered as the defining feature of a book app. However, what constitutes interactivity in the context of apps is not always clear. We postulate in this chapter a multimodal social semiotic theory of interactivity, which considers interactivity as not simply a function of technology, but also a resource for meaning making. We distinguish two basic types of interactivity—intra-text and extra-text—incorporated in the touch design, and explore the different functions they perform in a broad range of picture book apps. In particular, we look at the app version of The Heart and the Bottle in depth, and illustrate how interactive design elements help to create an interpretative possibility of the story. We suggest that a better understanding of interactive touch design would promote more effective adult-child interactions around mobile applications.