Using MT-ComparEval

Roman Sudarikov; Martin Popel; Ondřej Bojar; Aljoscha Burchardt; Ondrej Klejch

Using MT-ComparEval

Roman Sudarikov, Martin Popel, Ondřej Bojar, Aljoscha Burchardt, Ondrej Klejch

Research output: Contribution to conference › Paper › peer-review

Abstract

The paper showcases the MT-ComparEval tool for qualitative evaluation of machine translation (MT). MT-ComparEval is an open-source tool that has been designed in order to help MT developers by providing a graphical user interface that allows the comparisonand evaluation of different MT engines/experiments and settings. The tool implements several measures that represent the current best practice of automatic evaluation. It also provides guidance in the targeted inspection of examples that show a certain behavior in terms of n-gram similarity/dissimilarity with alternative translations or the reference translation. In this paper, we provide an applied, “hands-on” perspective on the actual usage of MT-ComparEval. In a case study, we use it to compare and analyze several systems submitted to the WMT 2015 shared task.

Original language	English
Pages	76-82
Number of pages	7
Publication status	Published - 24 May 2016
Event	LREC 2016 Workshop “Translation Evaluation – From Fragmented Tools and Data Sets to an Integrated Ecosystem” - Portorož, Slovenia Duration: 24 May 2016 → 24 May 2016 http://www.lrec-conf.org/proceedings/lrec2016/workshops.html

Workshop

Workshop	LREC 2016 Workshop “Translation Evaluation – From Fragmented Tools and Data Sets to an Integrated Ecosystem”
Country/Territory	Slovenia
City	Portorož
Period	24/05/16 → 24/05/16
Internet address	http://www.lrec-conf.org/proceedings/lrec2016/workshops.html

Keywords / Materials (for Non-textual outputs)

Machine Translation Evaluation
Analysis of MT Output

Access to Document

http://www.lrec-conf.org/proceedings/lrec2016/workshops/LREC2016Workshop-MT%20Evaluation_Proceedings.pdfLicence: All Rights Reserved

Cite this

@conference{b046f6a4017c4790a7897c7094689606,

title = "Using MT-ComparEval",

abstract = "The paper showcases the MT-ComparEval tool for qualitative evaluation of machine translation (MT). MT-ComparEval is an open-source tool that has been designed in order to help MT developers by providing a graphical user interface that allows the comparisonand evaluation of different MT engines/experiments and settings. The tool implements several measures that represent the current best practice of automatic evaluation. It also provides guidance in the targeted inspection of examples that show a certain behavior in terms of n-gram similarity/dissimilarity with alternative translations or the reference translation. In this paper, we provide an applied, “hands-on” perspective on the actual usage of MT-ComparEval. In a case study, we use it to compare and analyze several systems submitted to the WMT 2015 shared task.",

keywords = "Machine Translation Evaluation, Analysis of MT Output",

author = "Roman Sudarikov and Martin Popel and Ond{\v r}ej Bojar and Aljoscha Burchardt and Ondrej Klejch",

year = "2016",

month = may,

day = "24",

language = "English",

pages = "76--82",

note = "LREC 2016 Workshop “Translation Evaluation – From Fragmented Tools and Data Sets to an Integrated Ecosystem” ; Conference date: 24-05-2016 Through 24-05-2016",

url = "http://www.lrec-conf.org/proceedings/lrec2016/workshops.html",

}

Sudarikov, R, Popel, M, Bojar, O, Burchardt, A & Klejch, O 2016, 'Using MT-ComparEval', Paper presented at LREC 2016 Workshop “Translation Evaluation – From Fragmented Tools and Data Sets to an Integrated Ecosystem”, Portorož, Slovenia, 24/05/16 - 24/05/16 pp. 76-82. <http://www.lrec-conf.org/proceedings/lrec2016/workshops/LREC2016Workshop-MT%20Evaluation_Proceedings.pdf>

TY - CONF

T1 - Using MT-ComparEval

AU - Sudarikov, Roman

AU - Popel, Martin

AU - Bojar, Ondřej

AU - Burchardt, Aljoscha

AU - Klejch, Ondrej

PY - 2016/5/24

Y1 - 2016/5/24

N2 - The paper showcases the MT-ComparEval tool for qualitative evaluation of machine translation (MT). MT-ComparEval is an open-source tool that has been designed in order to help MT developers by providing a graphical user interface that allows the comparisonand evaluation of different MT engines/experiments and settings. The tool implements several measures that represent the current best practice of automatic evaluation. It also provides guidance in the targeted inspection of examples that show a certain behavior in terms of n-gram similarity/dissimilarity with alternative translations or the reference translation. In this paper, we provide an applied, “hands-on” perspective on the actual usage of MT-ComparEval. In a case study, we use it to compare and analyze several systems submitted to the WMT 2015 shared task.

AB - The paper showcases the MT-ComparEval tool for qualitative evaluation of machine translation (MT). MT-ComparEval is an open-source tool that has been designed in order to help MT developers by providing a graphical user interface that allows the comparisonand evaluation of different MT engines/experiments and settings. The tool implements several measures that represent the current best practice of automatic evaluation. It also provides guidance in the targeted inspection of examples that show a certain behavior in terms of n-gram similarity/dissimilarity with alternative translations or the reference translation. In this paper, we provide an applied, “hands-on” perspective on the actual usage of MT-ComparEval. In a case study, we use it to compare and analyze several systems submitted to the WMT 2015 shared task.

KW - Machine Translation Evaluation

KW - Analysis of MT Output

M3 - Paper

SP - 76

EP - 82

T2 - LREC 2016 Workshop “Translation Evaluation – From Fragmented Tools and Data Sets to an Integrated Ecosystem”

Y2 - 24 May 2016 through 24 May 2016

ER -

Using MT-ComparEval

Abstract

Workshop

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Cite this