Abstract / Description of output
The goal of this project was to create a multimodal dialogue system which provides some of the advantages of a human tutor, not normally encountered in self-study material and systems. A human tutor aids learners by:
•Providing a framework of tasks suitable to the learner’s needs
•Continuously monitoring learner progress and adapting task content and delivery style
•Providing a source of speaking practice and motivation
MILLA is a prototype language tuition system comprising tuition management, learner state monitoring, and an adaptable curriculum, all mediated through speech. The system enrols and monitors learners via a spoken dialogue interface, provides pronunciation practice and automatic error correction in two modalities, grammar exercises, and two custom speech-to-speech chat bots for spoken interaction practice. The focus on speech in the tutor’s output and in the learning modules addresses the current deficit in spoken interaction practice in Computer Aided Language Learning (CALL) applications, with different text-to-speech (TTS) voices used to provide a variety of speech models across the different modules. The system monitors learner engagement using Kinect sensors and checks pronunciation and responds to dialogue using automatic speech recognition (ASR).A learner record is used in conjunction with the curriculum to provide activities relevant to the learner’s current abilities and first language, and to monitor and record progress.
•Providing a framework of tasks suitable to the learner’s needs
•Continuously monitoring learner progress and adapting task content and delivery style
•Providing a source of speaking practice and motivation
MILLA is a prototype language tuition system comprising tuition management, learner state monitoring, and an adaptable curriculum, all mediated through speech. The system enrols and monitors learners via a spoken dialogue interface, provides pronunciation practice and automatic error correction in two modalities, grammar exercises, and two custom speech-to-speech chat bots for spoken interaction practice. The focus on speech in the tutor’s output and in the learning modules addresses the current deficit in spoken interaction practice in Computer Aided Language Learning (CALL) applications, with different text-to-speech (TTS) voices used to provide a variety of speech models across the different modules. The system monitors learner engagement using Kinect sensors and checks pronunciation and responds to dialogue using automatic speech recognition (ASR).A learner record is used in conjunction with the curriculum to provide activities relevant to the learner’s current abilities and first language, and to monitor and record progress.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of eNTERFACE’14 |
| Subtitle of host publication | 10th International Summer Workshop on Multimodal Interfaces |
| Editors | Daniel Erro, Inma Hernáez |
| Place of Publication | Bilbao, Spain |
| Pages | 164-166 |
| Publication status | Published - 2014 |