A Multilingual Parallel Corpora Collection Effort for Indian Languages

Shashank Siripragada, Jerin Philip, Vinay P. Namboodiri, C V Jawahar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present sentence aligned parallel corpora across 10 Indian Languages - Hindi, Telugu, Tamil, Malayalam, Gujarati, Urdu, Bengali, Oriya, Marathi, Punjabi, and English - many of which are categorized as low resource. The corpora are compiled from online sources which have content shared across languages. The corpora presented significantly extends present resources that are either not large enough or are restricted to a specific domain (such as health). We also provide a separate test corpus compiled from an independent online source that can be independently used for validating the performance in 10 Indian languages. Alongside, we report on the methods of constructing such corpora using tools enabled by recent advances in machine translation and cross-lingual retrieval using deep neural network based methods.
Original languageEnglish
Title of host publicationProceedings of the 12th Language Resources and Evaluation Conference
Place of PublicationMarseille, France
PublisherEuropean Language Resources Association (ELRA)
Pages3743-3751
Number of pages9
ISBN (Print)979-10-95546-34-4
Publication statusPublished - 11 May 2020
Event12th Language Resources and Evaluation Conference - Le Palais du Pharo, Marseille, France
Duration: 11 May 202016 May 2020
Conference number: 12
https://lrec2020.lrec-conf.org/en/

Conference

Conference12th Language Resources and Evaluation Conference
Abbreviated titleLREC 2020
Country/TerritoryFrance
CityMarseille
Period11/05/2016/05/20
Internet address

Keywords

  • Machine Translation
  • Parallel Corpus
  • Indian languages

Fingerprint

Dive into the research topics of 'A Multilingual Parallel Corpora Collection Effort for Indian Languages'. Together they form a unique fingerprint.

Cite this