Data collection and synchronisation: Towards a Multiperspective multimodal dialogue system with metacognitive abilities

Fasih Haider*, Saturnino Luz, Nick Campbell

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingChapter (peer-reviewed)peer-review

Abstract

This article describes the data collection system and methods adopted in the METALOGUE (Multiperspective Multimodal Dialogue System with Metacognitive Abilities) project. The ultimate goal of theMETALOGUE project is to develop a multimodal dialogue system with abilities to deliver instructional advice by interacting with humans in a natural way. The data we are collecting will facilitate the development of a dialogue system which will exploit metacognitive reasoning in order to deliver feedback on the user’s performance in debates and negotiations. The initial data collection scenario consists of debates where two students are exchanging views and arguments on a social issue, such as a proposed ban on smoking in public areas, and delivering their presentations in front of an audience. Approximately 3 hours of data has been recorded to date, and all recorded streams have been precisely synchronized and pre-processed for statistical learning. The data consists of audio, video and 3-dimensional skeletal movement information of the participants. This datawill be used in the development of cognitive dialogue and discourse models to underpin educational interventions in public speaking training.

Original languageEnglish
Title of host publicationDialogues with Social Robots - Enablements, Analyses, and Evaluation
PublisherSpringer
Pages245-256
Number of pages12
Volume999 LNEE
ISBN (Print)9789811025846
DOIs
Publication statusPublished - 25 Dec 2016
Event7th International Workshop on Spoken Dialogue Systems, IWSDS 2016 - Saariselka, Finland
Duration: 13 Jan 201616 Jan 2016

Publication series

NameLecture Notes in Electrical Engineering
Volume999 LNEE
ISSN (Print)18761100
ISSN (Electronic)18761119

Conference

Conference7th International Workshop on Spoken Dialogue Systems, IWSDS 2016
Country/TerritoryFinland
CitySaariselka
Period13/01/1616/01/16

Keywords / Materials (for Non-textual outputs)

  • Dialogue systems
  • Instructional advice
  • Metacognition
  • Multimodal data
  • Presentation quality

Fingerprint

Dive into the research topics of 'Data collection and synchronisation: Towards a Multiperspective multimodal dialogue system with metacognitive abilities'. Together they form a unique fingerprint.

Cite this