Projects per year
We evaluate the output of 16 English-to-German MT systems with respect to the translation of pronouns in the context of the WMT 2018 competition. We work with a test suite specifically designed to assess system quality in various fine-grained categories known to be problematic. The main evaluation scores come from a semi-automatic process, combining automatic reference matching with extensive manual annotation of uncertain cases. We find that current NMT systems are good at translating pronouns with intra-sentential reference, but the inter-sentential cases remain difficult. NMT systems are also good at the translation of event pronouns, unlike systems from the phrase-based SMT paradigm. No single system performs best at translating all types of anaphoric pronouns, suggesting unexplained random effects influencing the translation of pronouns with NMT.
|Title of host publication||Proceedings of the Third Conference on Machine Translation: Shared Task Papers|
|Place of Publication||Belgium, Brussels|
|Publisher||Association for Computational Linguistics|
|Number of pages||8|
|Publication status||Published - 1 Nov 2018|
|Event||EMNLP 2018 Third Conference on Machine Translation (WMT18) - Brussels, Belgium|
Duration: 31 Oct 2018 → 1 Nov 2018
|Workshop||EMNLP 2018 Third Conference on Machine Translation (WMT18)|
|Period||31/10/18 → 1/11/18|
FingerprintDive into the research topics of 'A Pronoun Test Suite Evaluation of the English--German MT Systems at WMT 2018'. Together they form a unique fingerprint.
- 1 Active
1/08/17 → 31/01/23