Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation

Chris Callison-Burch; Philipp Koehn; Christof Monz; Kay Peterson; Mark Przybocki; Omar F. Zaidan

Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation

Chris Callison-Burch, Philipp Koehn, Christof Monz, Kay Peterson, Mark Przybocki, Omar F. Zaidan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

This paper presents the results of the WMT10 and MetricsMATR10 shared tasks, which included a translation task, a system combination task, and an evaluation task. We conducted a large-scale manual evaluation of 104 machine translation systems and 41 system combination entries. We used the ranking of these systems to measure how strongly automatic metrics correlate with human judgments of translation quality for 26 metrics. This year we also investigated increasing the number of human judgments by hiring non-expert annotators through Amazon's Mechanical Turk.

Original language	English
Title of host publication	Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
Place of Publication	Stroudsburg, PA, USA
Publisher	Association for Computational Linguistics
Pages	17-53
Number of pages	37
Publication status	Published - 2010

Access to Document

http://dl.acm.org/citation.cfm?id=1868850.1868853

Cite this

Callison-Burch, C., Koehn, P., Monz, C., Peterson, K., Przybocki, M., & Zaidan, O. F. (2010). Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR (pp. 17-53). Association for Computational Linguistics. http://dl.acm.org/citation.cfm?id=1868850.1868853

@inproceedings{2aef69a7ad8245c3b411696f9bf24f58,

title = "Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation",

abstract = "This paper presents the results of the WMT10 and MetricsMATR10 shared tasks, which included a translation task, a system combination task, and an evaluation task. We conducted a large-scale manual evaluation of 104 machine translation systems and 41 system combination entries. We used the ranking of these systems to measure how strongly automatic metrics correlate with human judgments of translation quality for 26 metrics. This year we also investigated increasing the number of human judgments by hiring non-expert annotators through Amazon's Mechanical Turk.",

author = "Chris Callison-Burch and Philipp Koehn and Christof Monz and Kay Peterson and Mark Przybocki and Zaidan, {Omar F.}",

year = "2010",

language = "English",

pages = "17--53",

booktitle = "Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR",

publisher = "Association for Computational Linguistics",

}

Callison-Burch, C, Koehn, P, Monz, C, Peterson, K, Przybocki, M & Zaidan, OF 2010, Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation. in Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR. Association for Computational Linguistics, Stroudsburg, PA, USA, pp. 17-53. <http://dl.acm.org/citation.cfm?id=1868850.1868853>

Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation. / Callison-Burch, Chris; Koehn, Philipp; Monz, Christof et al.
Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR. Stroudsburg, PA, USA: Association for Computational Linguistics, 2010. p. 17-53.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation

AU - Callison-Burch, Chris

AU - Koehn, Philipp

AU - Monz, Christof

AU - Peterson, Kay

AU - Przybocki, Mark

AU - Zaidan, Omar F.

PY - 2010

Y1 - 2010

N2 - This paper presents the results of the WMT10 and MetricsMATR10 shared tasks, which included a translation task, a system combination task, and an evaluation task. We conducted a large-scale manual evaluation of 104 machine translation systems and 41 system combination entries. We used the ranking of these systems to measure how strongly automatic metrics correlate with human judgments of translation quality for 26 metrics. This year we also investigated increasing the number of human judgments by hiring non-expert annotators through Amazon's Mechanical Turk.

AB - This paper presents the results of the WMT10 and MetricsMATR10 shared tasks, which included a translation task, a system combination task, and an evaluation task. We conducted a large-scale manual evaluation of 104 machine translation systems and 41 system combination entries. We used the ranking of these systems to measure how strongly automatic metrics correlate with human judgments of translation quality for 26 metrics. This year we also investigated increasing the number of human judgments by hiring non-expert annotators through Amazon's Mechanical Turk.

M3 - Conference contribution

SP - 17

EP - 53

BT - Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR

PB - Association for Computational Linguistics

CY - Stroudsburg, PA, USA

ER -

Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation

Abstract / Description of output

Access to Document

Fingerprint

Cite this