Abstract / Description of output
We propose a novel adapter layer formalism for adapting multilingual models. They are more parameter-efficient than existing adapter layers while obtaining as good or better performance. The layers are specific to one language (as opposed to bilingual adapters) allowing to compose them and generalize to unseen language-pairs. In this zero-shot setting, they obtain a median improvement of +2.77 BLEU points over a strong 20-language multilingual Transformer baseline trained on TED talks.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) |
Place of Publication | Online |
Publisher | Association for Computational Linguistics |
Pages | 4465-4470 |
Number of pages | 6 |
ISBN (Electronic) | 978-1-952148-60-6 |
DOIs | |
Publication status | Published - 16 Nov 2020 |
Event | The 2020 Conference on Empirical Methods in Natural Language Processing - Online Duration: 16 Nov 2020 → 20 Nov 2020 https://2020.emnlp.org/ |
Conference
Conference | The 2020 Conference on Empirical Methods in Natural Language Processing |
---|---|
Abbreviated title | EMNLP 2020 |
Period | 16/11/20 → 20/11/20 |
Internet address |