A Graphical Pronoun Analysis Tool for the PROTEST Pronoun Evaluation Test Suite

Christian Hardmeier, Liane Guillou

Research output: Contribution to journalArticlepeer-review

Abstract

We present a graphical pronoun analysis tool and a set of guidelines for manual evaluation to be used with the PROTEST pronoun test suite for machine translation (MT). The tool provides a means for researchers to evaluate the performance of their MT systems and browse individual pronoun translations. MT systems may be evaluated automatically by comparing the translation of the test suite pronoun tokens in the MT output with those in the reference translation. Those translations that do not match the reference are referred for manual evaluation, which is supported by the graphical pronoun analysis tool and its accompanying annotation guidelines. By encouraging the manual examination and evaluation of individual pronoun tokens, we hope to understand better how well MT systems perform when translating different categories of pronouns, and gain insights as to where MT systems perform poorly and why.
Original languageEnglish
Pages (from-to)318-330
Number of pages13
JournalBaltic Journal of Modern Computing
Volume4
Issue number2
Publication statusPublished - 30 May 2016
Event19th annual conference of the European Association for Machine Translation (EAMT) - Riga, Latvia
Duration: 30 May 20161 Jun 2016
Conference number: 19
http://eamt2016.tilde.com/

Cite this