Domain Robustness in Neural Machine Translation

Mathias Müller, Annette Rios, Rico Sennrich

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Translating text that diverges from the training domain is a key challenge for machine translation. Domain robustness—the generalization of models to unseen test domains—is low for both statistical (SMT) and neural machine translation (NMT). In this paper, we study the performance of SMT and NMT models on out-of-domain test sets. We find that in unknown domains, SMT and NMT suffer from very different problems: SMT systems are mostly adequate but not fluent, while NMT systems are mostly fluent, but not adequate. For NMT, we identify such hallucinations (translations that are fluent but unrelated to the source) as a key reason for low domain robustness. To mitigate this problem, we empirically compare methods that are reported to improve adequacy or in-domain robustness in terms of their effectiveness at improving domain robustness. In experiments on German→English OPUS data, and German→Romansh (a low-resource setting) we find that several methods improve domain robustness. While those methods do lead to higher BLEU scores overall, they only slightly increase the adequacy of translations compared to SMT.
Original languageEnglish
Title of host publicationProceedings of the 14th Conference of the Association for Machine Translation in the Americas (AMTA 2020)
Place of PublicationVirtual
PublisherAssociation for Machine Translation in the Americas, AMTA
Pages151-164
Number of pages14
Publication statusPublished - 6 Oct 2020
Event14th Conference of the Association for Machine Translation in the Americas - Virtual Conference
Duration: 6 Oct 20209 Oct 2020
https://amtaweb.org/

Conference

Conference14th Conference of the Association for Machine Translation in the Americas
Abbreviated titleAMTA 2020
CityVirtual Conference
Period6/10/209/10/20
Internet address

Fingerprint

Dive into the research topics of 'Domain Robustness in Neural Machine Translation'. Together they form a unique fingerprint.

Cite this