Projects per year
Abstract
For machine translation to tackle discourse phenomena, models must have access to extrasentential linguistic context. There has been recent interest in modelling context in neural machine translation (NMT), but models have been principally evaluated with standard automatic metrics, poorly adapted to evaluating discourse phenomena. In this article, we present hand-crafted, discourse test sets, designed to test the models’ ability to exploit previous source and target sentences. We investigate the performance of recently proposed multi-encoder NMT models trained on subtitles for English to French. We also explore a novel way of exploiting context from the previous sentence. Despite gains using BLEU, multi-encoder models give limited improvement in the handling of discourse phenomena: 50% accuracy on our coreference test set and 53.5% for coherence/cohesion (compared to a non-contextual baseline of 50%). A simple strategy of decoding the concatenation of the previous and current sentence leads to good performance, and our novel strategy of multiencoding and decoding of two sentences leads to the best performance (72.5% for coreference and 57% for coherence/cohesion), highlighting the importance of target-side context.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) |
Place of Publication | New Orleans, Louisiana |
Publisher | Association for Computational Linguistics (ACL) |
Pages | 1304-1313 |
Number of pages | 10 |
ISBN (Electronic) | 978-1-948087-27-8 |
DOIs | |
Publication status | Published - 6 Jun 2018 |
Event | 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Hyatt Regency New Orleans Hotel, New Orleans, United States Duration: 1 Jun 2018 → 6 Jun 2018 http://naacl2018.org/ |
Conference
Conference | 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies |
---|---|
Abbreviated title | NAACL HLT 2018 |
Country/Territory | United States |
City | New Orleans |
Period | 1/06/18 → 6/06/18 |
Internet address |
Fingerprint
Dive into the research topics of 'Evaluating Discourse Phenomena in Neural Machine Translation'. Together they form a unique fingerprint.Projects
- 2 Finished
-
Translation for Massive Open Online Courses- TraMooc
Koehn, P. & Birch-Mayne, A.
1/02/15 → 31/01/18
Project: Research
-
HimL: Health in my Language
Haddow, B., Birch-Mayne, A. & Webber, B.
1/02/15 → 31/01/18
Project: Research
Profiles
-
Alexandra Birch-Mayne
- School of Informatics - Reader in Natural Language Processing
- Institute of Language, Cognition and Computation
- Language, Interaction and Robotics
Person: Academic: Research Active (Research Assistant)