Modelling the Single Word to Multi-Word Transition Using Matrix Completion

Gabriella Pizzuto; Timothy M. Hospedales; Olga Capirci; Angelo Cangelosi

doi:10.1109/DEVLRN.2019.8850708

Modelling the Single Word to Multi-Word Transition Using Matrix Completion

Gabriella Pizzuto, Timothy M. Hospedales, Olga Capirci, Angelo Cangelosi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Infants acquire language in distinct stages, starting from single gestures and single words, and through utilising gestures, they learn multi-word combinations. To achieve this language development on artificial agents, we propose a multi-modal computational model for single to multi-word transition through gesture-word combinations. Our approach relies on advancements in deep models for feature extraction and on casting the supplementary word generation problem into a matrix completion task. Experimental evaluation is carried out on a dataset recorded directly from the humanoid iCub’s cameras, comprising the deictic gesture of pointing and real-world objects. Illustrated by our results, the proposed architecture further strengthens its potential to model early stage language acquisition.

Original language	English
Title of host publication	2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob)
Publisher	Institute of Electrical and Electronics Engineers
Pages	284-289
Number of pages	6
ISBN (Electronic)	978-1-5386-8128-2
ISBN (Print)	978-1-5386-8129-9
DOIs	https://doi.org/10.1109/DEVLRN.2019.8850708
Publication status	Published - 30 Sept 2019
Event	9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics - Oslo, Norway Duration: 19 Aug 2019 → 22 Aug 2019 https://icdl-epirob2019.org/

Publication series

Name
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
ISSN (Print)	2161-9484
ISSN (Electronic)	2161-9484

Conference

Conference	9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics
Abbreviated title	ICDL-EPIROB 2019
Country/Territory	Norway
City	Oslo
Period	19/08/19 → 22/08/19
Internet address	https://icdl-epirob2019.org/

Keywords / Materials (for Non-textual outputs)

control engineering computing
feature extraction
gesture recognition
humanoid robots
matrix algebra
natural language processing
language development
multimodal computational model
gesture-word combinations
matrix completion task
deictic gesture
language acquisition
word generation problem
single word transition
humanoid iCub cameras
multi-word combinations
multi-word transition
Computational modeling
Object detection
Computer architecture
Pose estimation
Wrist
Humanoid robots
language computational modelling
early language acquisition
weakly supervised learning

Access to Document

10.1109/DEVLRN.2019.8850708Licence: All Rights Reserved

https://ieeexplore.ieee.org/document/8850708Licence: All Rights Reserved

Cite this

@inproceedings{9afd8d37e5a74f279261d2d424272a74,

title = "Modelling the Single Word to Multi-Word Transition Using Matrix Completion",

abstract = "Infants acquire language in distinct stages, starting from single gestures and single words, and through utilising gestures, they learn multi-word combinations. To achieve this language development on artificial agents, we propose a multi-modal computational model for single to multi-word transition through gesture-word combinations. Our approach relies on advancements in deep models for feature extraction and on casting the supplementary word generation problem into a matrix completion task. Experimental evaluation is carried out on a dataset recorded directly from the humanoid iCub{\textquoteright}s cameras, comprising the deictic gesture of pointing and real-world objects. Illustrated by our results, the proposed architecture further strengthens its potential to model early stage language acquisition.",

keywords = "control engineering computing, feature extraction, gesture recognition, humanoid robots, matrix algebra, natural language processing, language development, multimodal computational model, gesture-word combinations, matrix completion task, deictic gesture, language acquisition, word generation problem, single word transition, humanoid iCub cameras, multi-word combinations, multi-word transition, Computational modeling, Object detection, Computer architecture, Pose estimation, Wrist, Humanoid robots, language computational modelling, early language acquisition, weakly supervised learning",

author = "Gabriella Pizzuto and Hospedales, {Timothy M.} and Olga Capirci and Angelo Cangelosi",

year = "2019",

month = sep,

day = "30",

doi = "10.1109/DEVLRN.2019.8850708",

language = "English",

isbn = "978-1-5386-8129-9",

publisher = "Institute of Electrical and Electronics Engineers",

pages = "284--289",

booktitle = "2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob)",

address = "United States",

note = "9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, ICDL-EPIROB 2019 ; Conference date: 19-08-2019 Through 22-08-2019",

url = "https://icdl-epirob2019.org/",

}

Pizzuto, G, Hospedales, TM, Capirci, O & Cangelosi, A 2019, Modelling the Single Word to Multi-Word Transition Using Matrix Completion. in 2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob). Institute of Electrical and Electronics Engineers, pp. 284-289, 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, Oslo, Norway, 19/08/19. https://doi.org/10.1109/DEVLRN.2019.8850708

Modelling the Single Word to Multi-Word Transition Using Matrix Completion. / Pizzuto, Gabriella; Hospedales, Timothy M.; Capirci, Olga et al.
2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob). Institute of Electrical and Electronics Engineers, 2019. p. 284-289.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Modelling the Single Word to Multi-Word Transition Using Matrix Completion

AU - Pizzuto, Gabriella

AU - Hospedales, Timothy M.

AU - Capirci, Olga

AU - Cangelosi, Angelo

PY - 2019/9/30

Y1 - 2019/9/30

N2 - Infants acquire language in distinct stages, starting from single gestures and single words, and through utilising gestures, they learn multi-word combinations. To achieve this language development on artificial agents, we propose a multi-modal computational model for single to multi-word transition through gesture-word combinations. Our approach relies on advancements in deep models for feature extraction and on casting the supplementary word generation problem into a matrix completion task. Experimental evaluation is carried out on a dataset recorded directly from the humanoid iCub’s cameras, comprising the deictic gesture of pointing and real-world objects. Illustrated by our results, the proposed architecture further strengthens its potential to model early stage language acquisition.

AB - Infants acquire language in distinct stages, starting from single gestures and single words, and through utilising gestures, they learn multi-word combinations. To achieve this language development on artificial agents, we propose a multi-modal computational model for single to multi-word transition through gesture-word combinations. Our approach relies on advancements in deep models for feature extraction and on casting the supplementary word generation problem into a matrix completion task. Experimental evaluation is carried out on a dataset recorded directly from the humanoid iCub’s cameras, comprising the deictic gesture of pointing and real-world objects. Illustrated by our results, the proposed architecture further strengthens its potential to model early stage language acquisition.

KW - control engineering computing

KW - feature extraction

KW - gesture recognition

KW - humanoid robots

KW - matrix algebra

KW - natural language processing

KW - language development

KW - multimodal computational model

KW - gesture-word combinations

KW - matrix completion task

KW - deictic gesture

KW - language acquisition

KW - word generation problem

KW - single word transition

KW - humanoid iCub cameras

KW - multi-word combinations

KW - multi-word transition

KW - Computational modeling

KW - Object detection

KW - Computer architecture

KW - Pose estimation

KW - Wrist

KW - Humanoid robots

KW - language computational modelling

KW - early language acquisition

KW - weakly supervised learning

U2 - 10.1109/DEVLRN.2019.8850708

DO - 10.1109/DEVLRN.2019.8850708

M3 - Conference contribution

SN - 978-1-5386-8129-9

SP - 284

EP - 289

BT - 2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob)

PB - Institute of Electrical and Electronics Engineers

T2 - 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics

Y2 - 19 August 2019 through 22 August 2019

ER -

Modelling the Single Word to Multi-Word Transition Using Matrix Completion

Abstract

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Cite this