DCU-UvA Multimodal MT System Report

Iacer Calixto; Desmond Elliott; Stella Frank

doi:10.18653/v1/W16-2359

DCU-UvA Multimodal MT System Report

Iacer Calixto, Desmond Elliott, Stella Frank

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We present a doubly-attentive multimodal machine translation model. Our model learns to attend to source language and spatial-preserving CONV5,4 visual features as separate attention mechanisms in a neural translation model. In image description translation experiments (Task 1), we find an improvement of 2.3 Meteor points compared to initialising the hidden state of the decoder with only the FC7 features and 2.9 Meteor points compared to a text-only neural machine translation baseline, confirming the useful nature of attending to the CONV5,4 features.

Original language	English
Title of host publication	Proceedings of the First Conference on Machine Translation, WMT 2016, colocated with ACL 2016, August 11-12, Berlin, Germany
Publisher	Association for Computational Linguistics (ACL)
Pages	634-638
Number of pages	5
DOIs	https://doi.org/10.18653/v1/W16-2359
Publication status	Published - 12 Aug 2016
Event	First Conference on Machine Translation - Berlin, Germany Duration: 11 Aug 2016 → 12 Aug 2016 http://www.statmt.org/wmt16/

Conference

Conference	First Conference on Machine Translation
Abbreviated title	WMT16
Country/Territory	Germany
City	Berlin
Period	11/08/16 → 12/08/16
Internet address	http://www.statmt.org/wmt16/

Access to Document

10.18653/v1/W16-2359Licence: Creative Commons: Attribution (CC-BY)

W16-2359Final published version, 206 KBLicence: Creative Commons: Attribution (CC-BY)

https://www.aclweb.org/anthology/W16-2359/Licence: Creative Commons: Attribution (CC-BY)

1 Finished

QT21: Quality Translation 21
Koehn, P., Haddow, B. & Huck, M.
EU government bodies
1/02/15 → 31/01/18
Project: Research

Cite this

@inproceedings{1057dde54c204c26bb3c242821c13d12,

title = "DCU-UvA Multimodal MT System Report",

abstract = "We present a doubly-attentive multimodal machine translation model. Our model learns to attend to source language and spatial-preserving CONV5,4 visual features as separate attention mechanisms in a neural translation model. In image description translation experiments (Task 1), we find an improvement of 2.3 Meteor points compared to initialising the hidden state of the decoder with only the FC7 features and 2.9 Meteor points compared to a text-only neural machine translation baseline, confirming the useful nature of attending to the CONV5,4 features.",

author = "Iacer Calixto and Desmond Elliott and Stella Frank",

year = "2016",

month = aug,

day = "12",

doi = "10.18653/v1/W16-2359",

language = "English",

pages = "634--638",

booktitle = "Proceedings of the First Conference on Machine Translation, WMT 2016, colocated with ACL 2016, August 11-12, Berlin, Germany",

publisher = "Association for Computational Linguistics (ACL)",

note = "First Conference on Machine Translation, WMT16 ; Conference date: 11-08-2016 Through 12-08-2016",

url = "http://www.statmt.org/wmt16/",

}

Calixto, I, Elliott, D & Frank, S 2016, DCU-UvA Multimodal MT System Report. in Proceedings of the First Conference on Machine Translation, WMT 2016, colocated with ACL 2016, August 11-12, Berlin, Germany. Association for Computational Linguistics (ACL), pp. 634-638, First Conference on Machine Translation, Berlin, Germany, 11/08/16. https://doi.org/10.18653/v1/W16-2359

TY - GEN

T1 - DCU-UvA Multimodal MT System Report

AU - Calixto, Iacer

AU - Elliott, Desmond

AU - Frank, Stella

PY - 2016/8/12

Y1 - 2016/8/12

N2 - We present a doubly-attentive multimodal machine translation model. Our model learns to attend to source language and spatial-preserving CONV5,4 visual features as separate attention mechanisms in a neural translation model. In image description translation experiments (Task 1), we find an improvement of 2.3 Meteor points compared to initialising the hidden state of the decoder with only the FC7 features and 2.9 Meteor points compared to a text-only neural machine translation baseline, confirming the useful nature of attending to the CONV5,4 features.

AB - We present a doubly-attentive multimodal machine translation model. Our model learns to attend to source language and spatial-preserving CONV5,4 visual features as separate attention mechanisms in a neural translation model. In image description translation experiments (Task 1), we find an improvement of 2.3 Meteor points compared to initialising the hidden state of the decoder with only the FC7 features and 2.9 Meteor points compared to a text-only neural machine translation baseline, confirming the useful nature of attending to the CONV5,4 features.

U2 - 10.18653/v1/W16-2359

DO - 10.18653/v1/W16-2359

M3 - Conference contribution

SP - 634

EP - 638

BT - Proceedings of the First Conference on Machine Translation, WMT 2016, colocated with ACL 2016, August 11-12, Berlin, Germany

PB - Association for Computational Linguistics (ACL)

T2 - First Conference on Machine Translation

Y2 - 11 August 2016 through 12 August 2016

ER -

DCU-UvA Multimodal MT System Report

Abstract

Conference

Access to Document

Fingerprint

Projects

QT21: Quality Translation 21

Cite this