Results of the WMT15 Metrics Shared Task

Milos Stanojevic, Amir Kamran, Philipp Koehn, Ondrej Bojar

Research output: Chapter in Book/Report/Conference proceedingConference contribution


This paper presents the results of the WMT15 Metrics Shared Task. We asked participants of this task to score the outputs of the MT systems involved in the WMT15 Shared Translation Task. We collected scores of 46 metrics from 11 research groups. In addition to that, we computed scores of 7 standard metrics(BLEU, SentBLEU, NIST, WER, PER,TER and CDER) as baselines. The collected scores were evaluated in terms of system level correlation (how well each metric’s scores correlate with WMT15 official manual ranking of systems) and in terms of segment level correlation (how often a metric agrees with humans in comparing two translations of a particular sentence).
Original languageEnglish
Title of host publicationProceedings of the Tenth Workshop on Statistical Machine Translation, 2015
Place of PublicationLisbon, Portugal
PublisherAssociation for Computational Linguistics
Number of pages18
Publication statusPublished - 2015
EventTenth Workshop on Statistical Machine Translation - Lisbon, Portugal
Duration: 17 Sep 201518 Sep 2015


ConferenceTenth Workshop on Statistical Machine Translation
Abbreviated titleEMNLP 2015
Internet address

Cite this