A Formal Semantic Analysis of Gesture

The gestures that speakers use in tandem with speech include not only conventionalized actions with identifiable meanings (so-called narrow gloss gestures or emblems) but also productive iconic and deictic gestures whose form and meanings seem largely improvised in context. In this paper, we bridge the descriptive tradition with formal models of reference and discourse structure so as to articulate an approach to the interpretation of these productive gestures. Our model captures gestures' partial and incomplete meanings as derived from form and accounts for the more specific interpretations they derive in context. Our work emphasizes the commonality of the pragmatic mechanisms for interpreting both language and gesture, and the place of formal methods in discovering the principles and knowledge that those mechanisms rely on.