Modelling the Single Word to Multi-Word Transition Using Matrix Completion

Gabriella Pizzuto, Timothy M. Hospedales, Olga Capirci, Angelo Cangelosi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Infants acquire language in distinct stages, starting from single gestures and single words, and through utilising gestures, they learn multi-word combinations. To achieve this language development on artificial agents, we propose a multi-modal computational model for single to multi-word transition through gesture-word combinations. Our approach relies on advancements in deep models for feature extraction and on casting the supplementary word generation problem into a matrix completion task. Experimental evaluation is carried out on a dataset recorded directly from the humanoid iCub’s cameras, comprising the deictic gesture of pointing and real-world objects. Illustrated by our results, the proposed architecture further strengthens its potential to model early stage language acquisition.
Original languageEnglish
Title of host publication2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages284-289
Number of pages6
ISBN (Electronic)978-1-5386-8128-2
ISBN (Print)978-1-5386-8129-9
DOIs
Publication statusPublished - 30 Sep 2019
Event9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics - Oslo, Norway
Duration: 19 Aug 201922 Aug 2019
https://icdl-epirob2019.org/

Publication series

Name
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
ISSN (Print)2161-9484
ISSN (Electronic)2161-9484

Conference

Conference9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics
Abbreviated titleICDL-EPIROB 2019
CountryNorway
CityOslo
Period19/08/1922/08/19
Internet address

Keywords

  • control engineering computing
  • feature extraction
  • gesture recognition
  • humanoid robots
  • matrix algebra
  • natural language processing
  • language development
  • multimodal computational model
  • gesture-word combinations
  • matrix completion task
  • deictic gesture
  • language acquisition
  • word generation problem
  • single word transition
  • humanoid iCub cameras
  • multi-word combinations
  • multi-word transition
  • Computational modeling
  • Object detection
  • Computer architecture
  • Pose estimation
  • Wrist
  • Humanoid robots
  • language computational modelling
  • early language acquisition
  • weakly supervised learning

Fingerprint

Dive into the research topics of 'Modelling the Single Word to Multi-Word Transition Using Matrix Completion'. Together they form a unique fingerprint.

Cite this