Evaluating Historical Text Normalization Systems: How Well Do They Generalize?

Alexander Robertson, Sharon Goldwater

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

We highlight several issues in the evaluation of historical text normalization systems that make it hard to tell how well these systems would actually work in practice—i.e., for new datasets or languages; in comparison to more naïve systems; or as a preprocessing step for downstream NLP tools. We illustrate these issues and exemplify our proposed evaluation practices by comparing two neural models against a naïve baseline system. We show that the neural models generalize well to unseen words in tests on five languages; nevertheless, they provide no clear benefit over the naïve baseline for downstream POS tagging of an English historical collection. We conclude that future work should include more rigorous evaluation, including both intrinsic and extrinsic measures where possible.
Original languageEnglish
Title of host publication16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Place of PublicationNew Orleans, Louisiana
PublisherAssociation for Computational Linguistics (ACL)
Pages720-725
Number of pages6
DOIs
Publication statusPublished - 6 Jun 2018
Event16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Hyatt Regency New Orleans Hotel, New Orleans, United States
Duration: 1 Jun 20186 Jun 2018
http://naacl2018.org/

Conference

Conference16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Abbreviated titleNAACL HLT 2018
Country/TerritoryUnited States
CityNew Orleans
Period1/06/186/06/18
Internet address

Fingerprint

Dive into the research topics of 'Evaluating Historical Text Normalization Systems: How Well Do They Generalize?'. Together they form a unique fingerprint.

Cite this