Document-level Neural MT: A Systematic Comparison

António V. Lopes; M. Amin Farajian; Rachel Bawden; Michael Zhang; André F. T. Martins

Document-level Neural MT: A Systematic Comparison

António V. Lopes, M. Amin Farajian, Rachel Bawden, Michael Zhang, André F. T. Martins

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this paper we provide a systematic comparison of existing and new document-level neural machine translation solutions. As part of this comparison, we introduce and evaluate a document-level variant of the recently proposed Star Transformer architecture. In addition to using the traditional metric BLEU, we report the accuracy of the models in handling anaphoric pronoun translation as well as coherence and cohesion using contrastive test sets. Finally, we report the results of human evaluation in terms of Multidimensional Quality Metrics (MQM) and analyse the correlation of the results obtained by the automatic metrics with human judgments.

Original language	English
Title of host publication	Proceedings of the 22nd Annual Conference of the European Association for Machine Translation
Place of Publication	Lisboa, Portugal
Publisher	European Association for Machine Translation
Pages	225–234
Number of pages	10
ISBN (Electronic)	978-989-33-0589-8
Publication status	Published - 6 May 2020
Event	22nd Annual Conference of the European Association for Machine Translation - Online Conference in place of Instituto Superior Técnico, Lisbon, Portugal Duration: 3 Nov 2020 → 5 Nov 2020 https://eamt2020.inesc-id.pt/

Conference

Conference	22nd Annual Conference of the European Association for Machine Translation
Abbreviated title	EAMT 2020
Period	3/11/20 → 5/11/20
Internet address	https://eamt2020.inesc-id.pt/

Keywords / Materials (for Non-textual outputs)

Machine translation
Document-level machine translation
Neural machine translation
Context
Evaluation
anaphora
lexical choice

Access to Document

EAMT20_Document_level_NMT-correctedAccepted author manuscript, 249 KBLicence: Creative Commons: Attribution No Derivatives (CC-BY-ND)

https://eamt2020.inesc-id.pt/proceedings-eamt2020.pdfLicence: Creative Commons: Attribution No Derivatives (CC-BY-ND)

Global Under-Resourced MEdia Translation
Birch-Mayne, A. (Principal Investigator) & Haddow, B. (Co-investigator)
EU government bodies
1/01/19 → 30/06/22
Project: Research
MTStretch: Low-resource Machine Translation
Birch-Mayne, A. (Principal Investigator)
EPSRC
29/06/18 → 28/12/21
Project: Research

Cite this

@inproceedings{79a87a8b833846dd869d0283f8db0b2c,

title = "Document-level Neural MT: A Systematic Comparison",

abstract = "In this paper we provide a systematic comparison of existing and new document-level neural machine translation solutions. As part of this comparison, we introduce and evaluate a document-level variant of the recently proposed Star Transformer architecture. In addition to using the traditional metric BLEU, we report the accuracy of the models in handling anaphoric pronoun translation as well as coherence and cohesion using contrastive test sets. Finally, we report the results of human evaluation in terms of Multidimensional Quality Metrics (MQM) and analyse the correlation of the results obtained by the automatic metrics with human judgments.",

keywords = "Machine translation, Document-level machine translation, Neural machine translation, Context, Evaluation, anaphora, lexical choice",

author = "Lopes, {Ant{\'o}nio V.} and Farajian, {M. Amin} and Rachel Bawden and Michael Zhang and Martins, {Andr{\'e} F. T.}",

year = "2020",

month = may,

day = "6",

language = "English",

pages = "225–234",

booktitle = "Proceedings of the 22nd Annual Conference of the European Association for Machine Translation",

publisher = "European Association for Machine Translation",

note = "22nd Annual Conference of the European Association for Machine Translation, EAMT 2020 ; Conference date: 03-11-2020 Through 05-11-2020",

url = "https://eamt2020.inesc-id.pt/",

}

Lopes, AV, Farajian, MA, Bawden, R, Zhang, M & Martins, AFT 2020, Document-level Neural MT: A Systematic Comparison. in Proceedings of the 22nd Annual Conference of the European Association for Machine Translation. European Association for Machine Translation, Lisboa, Portugal, pp. 225–234, 22nd Annual Conference of the European Association for Machine Translation, 3/11/20. <https://eamt2020.inesc-id.pt/proceedings-eamt2020.pdf>

Document-level Neural MT: A Systematic Comparison. / Lopes, António V.; Farajian, M. Amin; Bawden, Rachel et al.
Proceedings of the 22nd Annual Conference of the European Association for Machine Translation. Lisboa, Portugal: European Association for Machine Translation, 2020. p. 225–234.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Document-level Neural MT: A Systematic Comparison

AU - Lopes, António V.

AU - Farajian, M. Amin

AU - Bawden, Rachel

AU - Zhang, Michael

AU - Martins, André F. T.

PY - 2020/5/6

Y1 - 2020/5/6

N2 - In this paper we provide a systematic comparison of existing and new document-level neural machine translation solutions. As part of this comparison, we introduce and evaluate a document-level variant of the recently proposed Star Transformer architecture. In addition to using the traditional metric BLEU, we report the accuracy of the models in handling anaphoric pronoun translation as well as coherence and cohesion using contrastive test sets. Finally, we report the results of human evaluation in terms of Multidimensional Quality Metrics (MQM) and analyse the correlation of the results obtained by the automatic metrics with human judgments.

AB - In this paper we provide a systematic comparison of existing and new document-level neural machine translation solutions. As part of this comparison, we introduce and evaluate a document-level variant of the recently proposed Star Transformer architecture. In addition to using the traditional metric BLEU, we report the accuracy of the models in handling anaphoric pronoun translation as well as coherence and cohesion using contrastive test sets. Finally, we report the results of human evaluation in terms of Multidimensional Quality Metrics (MQM) and analyse the correlation of the results obtained by the automatic metrics with human judgments.

KW - Machine translation

KW - Document-level machine translation

KW - Neural machine translation

KW - Context

KW - Evaluation

KW - anaphora

KW - lexical choice

M3 - Conference contribution

SP - 225

EP - 234

BT - Proceedings of the 22nd Annual Conference of the European Association for Machine Translation

PB - European Association for Machine Translation

CY - Lisboa, Portugal

T2 - 22nd Annual Conference of the European Association for Machine Translation

Y2 - 3 November 2020 through 5 November 2020

ER -

Document-level Neural MT: A Systematic Comparison

Abstract

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Projects

Global Under-Resourced MEdia Translation

MTStretch: Low-resource Machine Translation

Cite this