Projects per year
Abstract
Differently from the traditional statistical MT that decomposes the translation task into distinct separately learned components, neural machine translation uses a single neural network to model the entire translation process. Despite neural machine translation being de-facto standard, it is still not clear how NMT models acquire different competences over the course of training, and how this mirrors the different models in traditional SMT. In this work, we look at the competences related to three core SMT components and find that during training, NMT first focuses on learning target-side language modeling, then improves translation quality approaching word-by-word translation, and finally learns more complicated reordering patterns. We show that this behavior holds for several models and language pairs. Additionally, we explain how such an understanding of the training process can be useful in practice and, as an example, show how it can be used to improve vanilla non-autoregressive neural machine translation by guiding teacher model selection.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing |
Place of Publication | Online and Punta Cana, Dominican Republic |
Publisher | Association for Computational Linguistics |
Pages | 8478-8491 |
Number of pages | 14 |
ISBN (Electronic) | 978-1-955917-09-4 |
Publication status | Published - 7 Nov 2021 |
Event | 2021 Conference on Empirical Methods in Natural Language Processing - Punta Cana, Dominican Republic Duration: 7 Nov 2021 → 11 Nov 2021 https://2021.emnlp.org/ |
Conference
Conference | 2021 Conference on Empirical Methods in Natural Language Processing |
---|---|
Abbreviated title | EMNLP 2021 |
Country/Territory | Dominican Republic |
City | Punta Cana |
Period | 7/11/21 → 11/11/21 |
Internet address |
Fingerprint
Dive into the research topics of 'Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT'. Together they form a unique fingerprint.Projects
- 2 Finished
-
Global Under-Resourced MEdia Translation
Birch-Mayne, A. (Principal Investigator) & Haddow, B. (Co-investigator)
1/01/19 → 30/06/22
Project: Research
-
BroadSem-Induction of Broad-Coverage Semantic Parsers
Titov, I. (Principal Investigator)
1/05/17 → 30/04/22
Project: Research