Human Evaluation of Grammatical Error Correction Systems

Roman Grundkiewicz, Marcin Junczys-Dowmunt, Edward Gillian

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The paper presents the results of the first large-scale human evaluation of automatic grammatical error correction (GEC) systems. Twelve participating systems and the unchanged input of the CoNLL-2014 shared task have been reassessed in a WMT-inspired human evaluation procedure. Methods introduced for the Workshop of Machine Translation evaluation campaigns have been adapted to GEC and extended where necessary. The produced rankings are used to evaluate standard metrics for grammatical error correction in terms of correlation with human judgment.
Original languageEnglish
Title of host publicationProceedings of the 2015 Conference on Empirical Methods in Natural Language Processing
Place of PublicationLisbon, Portugal
PublisherAssociation for Computational Linguistics
Pages461-470
Number of pages10
Publication statusPublished - 1 Sep 2015
EventConference on Empirical Methods in Natural Language Processing - Lisbon, Portugal
Duration: 17 Sep 201521 Sep 2015
http://www.emnlp2015.org/

Conference

ConferenceConference on Empirical Methods in Natural Language Processing
Abbreviated titleEMNLP 2015
Country/TerritoryPortugal
CityLisbon
Period17/09/1521/09/15
Internet address

Fingerprint

Dive into the research topics of 'Human Evaluation of Grammatical Error Correction Systems'. Together they form a unique fingerprint.

Cite this