Results of the WMT15 Metrics Shared Task

Milos Stanojevic, Amir Kamran, Philipp Koehn, Ondrej Bojar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

This paper presents the results of the WMT15 Metrics Shared Task. We asked participants of this task to score the outputs of the MT systems involved in the WMT15 Shared Translation Task. We collected scores of 46 metrics from 11 research groups. In addition to that, we computed scores of 7 standard metrics(BLEU, SentBLEU, NIST, WER, PER,TER and CDER) as baselines. The collected scores were evaluated in terms of system level correlation (how well each metric’s scores correlate with WMT15 official manual ranking of systems) and in terms of segment level correlation (how often a metric agrees with humans in comparing two translations of a particular sentence).
Original languageEnglish
Title of host publicationProceedings of the Tenth Workshop on Statistical Machine Translation, 2015
Place of PublicationLisbon, Portugal
PublisherAssociation for Computational Linguistics
Number of pages18
Publication statusPublished - 2015
EventTenth Workshop on Statistical Machine Translation - Lisbon, Portugal
Duration: 17 Sept 201518 Sept 2015


ConferenceTenth Workshop on Statistical Machine Translation
Abbreviated titleEMNLP 2015
Internet address


Dive into the research topics of 'Results of the WMT15 Metrics Shared Task'. Together they form a unique fingerprint.

Cite this