Regularized Training Objective for Continued Training for Domain Adaptation in Neural Machine Translation

Huda Khayrallah, Brian Thompson, Kevin Duh, Philipp Koehn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Supervised domain adaptation—where a large generic corpus and a smaller indomain corpus are both available for training—is a challenge for neural machine translation (NMT). Standard practice is to train a generic model and use it to initialize a second model, then continue training the second model on in-domain data to produce an in-domain model. We add an auxiliary term to the training objective during continued training that minimizes the cross entropy between the indomain model’s output word distribution and that of the out-of-domain model to prevent the model’s output from differing too much from the original out-of-domain model. We perform experiments on EMEA (descriptions of medicines) and TED (rehearsed presentations), initialized from a general domain (WMT) model. Our method shows improvements over standard continued training by up to 1.5 BLEU.
Original languageEnglish
Title of host publicationProceedings of the 2nd Workshop on Neural Machine Translation and Generation
Place of PublicationMelbourne, Australia
PublisherAssociation for Computational Linguistics
Pages36-44
Number of pages9
Publication statusPublished - 20 Jul 2018
Event2nd Workshop on Neural Machine Translation and Generation - Melbourne, Australia
Duration: 15 Jul 201820 Jul 2018
https://sites.google.com/site/wnmt18/home
https://sites.google.com/site/wnmt18/

Conference

Conference2nd Workshop on Neural Machine Translation and Generation
Abbreviated titleWNMT 2018
Country/TerritoryAustralia
CityMelbourne
Period15/07/1820/07/18
Internet address

Fingerprint

Dive into the research topics of 'Regularized Training Objective for Continued Training for Domain Adaptation in Neural Machine Translation'. Together they form a unique fingerprint.

Cite this