SIGMORPHON 2021 Shared Task on Morphological Reinflection: Generalization Across Languages

Tiago Pimentel, Maria Ryskina, Sabrina J. Mielke, Shijie Wu, Eleanor Chodroff, Brian Leonard, Garrett Nicolai, Yustinus Ghanggo Ate, Salam Khalifa, Nizar Habash, Charbel El-Khaissi, Omer Goldman, Michael Gasser, William Lane, Matt Coler, Arturo Oncevay, Jaime Rafael Montoya Samame, Gema Celeste Silva Villegas, Adam Ek, Jean-Philippe BernardyAndrey Shcherbakov, Aziyana Bayyr-ool, Karina Sheifer, Sofya Ganieva, Matvey Plugaryov, Elena Klyachko, Ali Salehi, Andrew Krizhanovsky, Natalia Krizhanovsky, Clara Vania, Sardana Ivanova, Aelita Salchak, Christopher Straughn, Zoey Liu, Jonathan North Washington, Duygu Ataman, Witold Kieraś, Marcin Woliński, Totok Suhardijanto, Niklas Stoehr, Zahroh Nuriah, Shyam Ratan, Francis M. Tyers, Edoardo M. Ponti, Grant Aiton, Richard J. Hatcher, Emily Prud'hommeaux, Ritesh Kumar, Mans Hulden, Botond Barta, Dorina Lakatos, Gábor Szolnok, Judit Ács, Mohit Raj, David Yarowsky, Ryan Cotterell, Ben Ambridge, Ekaterina Vylomova

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

This year's iteration of the SIGMORPHON Shared Task on morphological reinflection focuses on typological diversity and cross-lingual variation of morphosyntactic features. In terms of the task, we enrich UniMorph with new data for 32 languages from 13 language families, with most of them being under-resourced: Kunwinjku, Classical Syriac, Arabic (Modern Standard, Egyptian, Gulf), Hebrew, Amharic, Aymara, Magahi, Braj, Kurdish (Central, Northern, Southern), Polish, Karelian, Livvi, Ludic, Veps, Võro, Evenki, Xibe, Tuvan, Sakha, Turkish, Indonesian, Kodi, Seneca, Asháninka, Yanesha, Chukchi, Itelmen, Eibela. We evaluate six systems on the new data and conduct an extensive error analysis of the systems' predictions. Transformer-based models generally demonstrate superior performance on the majority of languages, achieving textgreater90% accuracy on 65% of them. The languages on which systems yielded low accuracy are mainly under-resourced, with a limited amount of data. Most errors made by the systems are due to allomorphy, honorificity, and form variation. In addition, we observe that systems especially struggle to inflect multiword lemmas. The systems also produce misspelled forms or end up in repetitive loops (e.g., RNN-based models). Finally, we report a large drop in systems' performance on previously unseen lemmas.
Original languageEnglish
Title of host publicationProceedings of the 18th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology
EditorsGarrett Nicolai, Kyle Gorman, Ryan Cotterell
Place of PublicationStroudsburg, PA, USA
PublisherAssociation for Computational Linguistics
Pages229-259
Number of pages31
ISBN (Electronic)978-1-954085-62-6
DOIs
Publication statusPublished - 5 Aug 2021
EventThe 18th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology - Online
Duration: 5 Aug 20215 Aug 2021
Conference number: 18
https://sigmorphon.github.io/workshops/2021/

Workshop

WorkshopThe 18th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology
Abbreviated titleSIGMORPHON 2021
Period5/08/215/08/21
Internet address

Fingerprint

Dive into the research topics of 'SIGMORPHON 2021 Shared Task on Morphological Reinflection: Generalization Across Languages'. Together they form a unique fingerprint.

Cite this