Abstract
Infants acquire language in distinct stages, starting from single gestures and single words, and through utilising gestures, they learn multi-word combinations. To achieve this language development on artificial agents, we propose a multi-modal computational model for single to multi-word transition through gesture-word combinations. Our approach relies on advancements in deep models for feature extraction and on casting the supplementary word generation problem into a matrix completion task. Experimental evaluation is carried out on a dataset recorded directly from the humanoid iCub’s cameras, comprising the deictic gesture of pointing and real-world objects. Illustrated by our results, the proposed architecture further strengthens its potential to model early stage language acquisition.
Original language | English |
---|---|
Title of host publication | 2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob) |
Publisher | Institute of Electrical and Electronics Engineers |
Pages | 284-289 |
Number of pages | 6 |
ISBN (Electronic) | 978-1-5386-8128-2 |
ISBN (Print) | 978-1-5386-8129-9 |
DOIs | |
Publication status | Published - 30 Sept 2019 |
Event | 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics - Oslo, Norway Duration: 19 Aug 2019 → 22 Aug 2019 https://icdl-epirob2019.org/ |
Publication series
Name | |
---|---|
Publisher | Institute of Electrical and Electronics Engineers (IEEE) |
ISSN (Print) | 2161-9484 |
ISSN (Electronic) | 2161-9484 |
Conference
Conference | 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics |
---|---|
Abbreviated title | ICDL-EPIROB 2019 |
Country/Territory | Norway |
City | Oslo |
Period | 19/08/19 → 22/08/19 |
Internet address |
Keywords / Materials (for Non-textual outputs)
- control engineering computing
- feature extraction
- gesture recognition
- humanoid robots
- matrix algebra
- natural language processing
- language development
- multimodal computational model
- gesture-word combinations
- matrix completion task
- deictic gesture
- language acquisition
- word generation problem
- single word transition
- humanoid iCub cameras
- multi-word combinations
- multi-word transition
- Computational modeling
- Object detection
- Computer architecture
- Pose estimation
- Wrist
- Humanoid robots
- language computational modelling
- early language acquisition
- weakly supervised learning