Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings

Annette Rios, Laura Mascarell, Rico Sennrich

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Word sense disambiguation is necessary in translation because different word senses often have different translations. Neural machine translation models learn different senses of words as part of an end-to-end translation task, and their capability to perform word sense disambiguation has so far not been quantified. We exploit the fact that neural translation models can score arbitrary translations to design a novel cross-lingual word sense disambiguation task that is tailored towards evaluating neural machine translation models. We present a test set of 7,200 lexical ambiguities for German!English, and 6,700 for German!French, and report baseline results. With 70% of lexical ambiguities correctly disambiguated, we find that word sense disambiguation remains a challenging problem for neural machine translation, especially for rare word senses. To improve word sense disambiguation in neural machine translation, we experiment with two methods to integrate sense embeddings. In a first approach we pass sense embeddings as additional input to the neural machine translation system. For the second experiment, we extract lexical chains based on sense embeddings from the document and integrate this information into the NMT model. While a baseline NMT system disambiguates frequent word senses quite reliably, the annotation with both sense labels and lexical chains improves the neural models’ performance on rare word senses.
Original languageEnglish
Title of host publicationProceedings of the Conference on Machine Translation (WMT), Volume 1: Research Papers
Place of PublicationCopenhagen, Denmark
PublisherAssociation for Computational Linguistics
Number of pages9
ISBN (Print)978-1-945626-96-8
Publication statusPublished - 11 Sep 2017
Event2017 Conference on Machine Translation - Copenhagen, Denmark
Duration: 7 Sep 20178 Sep 2017


Conference2017 Conference on Machine Translation


Dive into the research topics of 'Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings'. Together they form a unique fingerprint.

Cite this