The roles of haptic-ostensive referring expressions in cooperative, task-based human-robot dialogue

Mary Ellen Foster, Ellen Gurman Bard, Markus Guhe, Robin L. Hill, Jon Oberlander, Alois Knoll

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Generating referring expressions is a task that has received a great deal of attention in the natural-language generation community, with an increasing amount of recent effort targeted at the generation of multimodal referring expressions. However, most implemented systems tend to assume very little shared knowledge between the speaker and the hearer, and therefore must generate fully-elaborated linguistic references. Some systems do include a representation of the physical context or the dialogue context; however, other sources of contextual information are not normally used. Also, the generated references normally consist only of language and, possibly, deictic pointing gestures.

When referring to objects in the context of a task-based interaction involving jointly manipulating objects, a much richer notion of context is available, which permits a wider range of referring options. In particular, when conversational partners cooperate on a mutual task in a shared environment, objects can be made accessible simply by manipulating them as part of the task. We demonstrate that such expressions are common in a corpus of human-human dialogues based on constructing virtual objects, and then describe how this type of reference can be incorporated into the output of a humanoid robot that engages in similar joint construction dialogues with a human partner.
Original languageEnglish
Title of host publicationProceedings of the 3rd ACM/IEEE international conference on Human Robot Interaction (HRI '08)
Place of PublicationNew York, NY, USA
PublisherACM
Pages295-302
Number of pages8
ISBN (Print)978-1-60558-017-3
DOIs
Publication statusPublished - 2008

Keywords / Materials (for Non-textual outputs)

  • multimodal dialogue
  • referring expressions

Fingerprint

Dive into the research topics of 'The roles of haptic-ostensive referring expressions in cooperative, task-based human-robot dialogue'. Together they form a unique fingerprint.

Cite this