Simulating human judgment in machine translation evaluation campaigns

Philipp Koehn

Simulating human judgment in machine translation evaluation campaigns

Philipp Koehn

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We present a Monte Carlo model to simulate human judgments in machine translation evaluation campaigns, such as WMT or IWSLT. We use the model to compare different ranking methods and to give guidance on the number of judgments that need to be collected to obtain sufficiently significant distinctions between systems

Original language	English
Title of host publication	2012 International Workshop on Spoken Language Translation, IWSLT 2012, Hong Kong, December 6-7, 2012
Publisher	ISCA
Pages	179-184
Number of pages	6
Publication status	Published - 2012

Access to Document

Koehn 2012 Simulating human judgment in machine translation evaluation campaignsAccepted author manuscript, 661 KB

http://www.isca-speech.org/archive/iwslt_12/sltc_179.html

Cite this

@inproceedings{bb60267d7b3944439b33c0fe52896e56,

title = "Simulating human judgment in machine translation evaluation campaigns",

abstract = "We present a Monte Carlo model to simulate human judgments in machine translation evaluation campaigns, such as WMT or IWSLT. We use the model to compare different ranking methods and to give guidance on the number of judgments that need to be collected to obtain sufficiently significant distinctions between systems",

author = "Philipp Koehn",

year = "2012",

language = "English",

pages = "179--184",

booktitle = "2012 International Workshop on Spoken Language Translation, IWSLT 2012, Hong Kong, December 6-7, 2012",

publisher = "ISCA",

}

TY - GEN

T1 - Simulating human judgment in machine translation evaluation campaigns

AU - Koehn, Philipp

PY - 2012

Y1 - 2012

N2 - We present a Monte Carlo model to simulate human judgments in machine translation evaluation campaigns, such as WMT or IWSLT. We use the model to compare different ranking methods and to give guidance on the number of judgments that need to be collected to obtain sufficiently significant distinctions between systems

AB - We present a Monte Carlo model to simulate human judgments in machine translation evaluation campaigns, such as WMT or IWSLT. We use the model to compare different ranking methods and to give guidance on the number of judgments that need to be collected to obtain sufficiently significant distinctions between systems

M3 - Conference contribution

SP - 179

EP - 184

BT - 2012 International Workshop on Spoken Language Translation, IWSLT 2012, Hong Kong, December 6-7, 2012

PB - ISCA

ER -