Climbing Mont BLEU: The Strange World of Reachable High-BLEU Translations

Aaron Smith, Christian Hardmeier, Joerg Tiedemann

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a method for finding oracle BLEU translations in phrase-based statistical machine translation using exact document-level scores. Experiments are presented where the BLEU score of a candidate translation is directly optimised in order to examine the properties of reachable translations with very high BLEU scores. This is achieved by running the document-level decoder Docent in BLEU-decoding mode, where proposed changes to the translation of a document are only accepted if they increase BLEU. The results confirm that the reference translation cannot in most cases be reached by the decoder, which is limited by the set of phrases in the phrase table, and demonstrate that high-BLEU translations are often of poor quality.
Original languageEnglish
Title of host publicationProceedings of the 19th Annual Conference of the European Association for Machine Translation
PublisherAssociation for Computational Linguistics
Pages269-281
Number of pages13
Publication statusPublished - 1 Jun 2016
Event19th annual conference of the European Association for Machine Translation (EAMT) - Riga, Latvia
Duration: 30 May 20161 Jun 2016
Conference number: 19
http://eamt2016.tilde.com/

Conference

Conference19th annual conference of the European Association for Machine Translation (EAMT)
Abbreviated titleEAMT 2016
Country/TerritoryLatvia
CityRiga
Period30/05/161/06/16
Internet address

Keywords

  • Statistical machine translation
  • oracle decoding
  • BLEU
  • Docent

Fingerprint

Dive into the research topics of 'Climbing Mont BLEU: The Strange World of Reachable High-BLEU Translations'. Together they form a unique fingerprint.

Cite this