How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM

Shaoxiong Ji; Pinzhen Chen

How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM

Shaoxiong Ji, Pinzhen Chen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Instruction tuning a large language model with multiple languages can prepare it for multilingual downstream tasks. Nonetheless, it is yet to be determined whether having a handful of languages is sufficient, or whether the benefits increase with the inclusion of more. By fine-tuning large multilingual models on 1 to 52 languages, we present a case study on BLOOM to understand three pertinent factors affecting performance: the number of languages, language exposure, and similarity between training and test languages. Overall we found that 1) expanding language coverage in multilingual instruction tuning proves to be beneficial; 2) accuracy often significantly boots if the test language appears in the instruction mixture; 3) languages' genetic features correlate with cross-lingual transfer more than merely the number of language but different languages benefit to various degrees.

Original language	English
Title of host publication	Proceedings of the 31st International Conference on Computational Linguistics
Editors	Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert
Place of Publication	Abu Dhabi, UAE
Publisher	Association for Computational Linguistics
Pages	2575-2581
Number of pages	7
ISBN (Print)	9798891761964
Publication status	Published - 1 Jan 2025
Event	The 31st International Conference on Computational Linguistics - Abu Dhabi, United Arab Emirates Duration: 19 Jan 2025 → 24 Jan 2025 Conference number: 31 http://www.wikicfp.com/cfp/servlet/event.showcfp?copyownerid=90704&eventid=180678

Publication series

Name	Proceedings – International Conference on Computational Linguistics
Publisher	ACM
ISSN (Print)	2951-2093

Conference

Conference	The 31st International Conference on Computational Linguistics
Abbreviated title	COLING 2025
Country/Territory	United Arab Emirates
City	Abu Dhabi
Period	19/01/25 → 24/01/25
Internet address	http://www.wikicfp.com/cfp/servlet/event.showcfp?copyownerid=90704&eventid=180678

Access to Document

JiChenCOLING25HowManyLanguagesMakeaGoodMultilingualInstructionTuningFinal published version, 649 KBLicence: Creative Commons: Attribution (CC-BY)

https://aclanthology.org/2025.coling-main.175/Licence: Creative Commons: Attribution (CC-BY)

Cite this

Ji, S., & Chen, P. (2025). How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM. In O. Rambow, L. Wanner, M. Apidianaki, H. Al-Khalifa, B. D. Eugenio, & S. Schockaert (Eds.), Proceedings of the 31st International Conference on Computational Linguistics (pp. 2575-2581). (Proceedings – International Conference on Computational Linguistics). Association for Computational Linguistics. https://aclanthology.org/2025.coling-main.175/

Ji, Shaoxiong ; Chen, Pinzhen. / How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM. Proceedings of the 31st International Conference on Computational Linguistics. editor / Owen Rambow ; Leo Wanner ; Marianna Apidianaki ; Hend Al-Khalifa ; Barbara Di Eugenio ; Steven Schockaert. Abu Dhabi, UAE : Association for Computational Linguistics, 2025. pp. 2575-2581 (Proceedings – International Conference on Computational Linguistics).

@inproceedings{418445bd6c684fc8bb3a88bc2d89bf9c,

title = "How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM",

abstract = "Instruction tuning a large language model with multiple languages can prepare it for multilingual downstream tasks. Nonetheless, it is yet to be determined whether having a handful of languages is sufficient, or whether the benefits increase with the inclusion of more. By fine-tuning large multilingual models on 1 to 52 languages, we present a case study on BLOOM to understand three pertinent factors affecting performance: the number of languages, language exposure, and similarity between training and test languages. Overall we found that 1) expanding language coverage in multilingual instruction tuning proves to be beneficial; 2) accuracy often significantly boots if the test language appears in the instruction mixture; 3) languages' genetic features correlate with cross-lingual transfer more than merely the number of language but different languages benefit to various degrees.",

author = "Shaoxiong Ji and Pinzhen Chen",

year = "2025",

month = jan,

day = "1",

language = "English",

isbn = "9798891761964",

series = "Proceedings – International Conference on Computational Linguistics",

publisher = "Association for Computational Linguistics",

pages = "2575--2581",

editor = "Owen Rambow and Leo Wanner and Marianna Apidianaki and Hend Al-Khalifa and Eugenio, \{Barbara Di\} and Steven Schockaert",

booktitle = "Proceedings of the 31st International Conference on Computational Linguistics",

note = "The 31st International Conference on Computational Linguistics, COLING 2025 ; Conference date: 19-01-2025 Through 24-01-2025",

url = "http://www.wikicfp.com/cfp/servlet/event.showcfp?copyownerid=90704\&eventid=180678",

}

Ji, S & Chen, P 2025, How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM. in O Rambow, L Wanner, M Apidianaki, H Al-Khalifa, BD Eugenio & S Schockaert (eds), Proceedings of the 31st International Conference on Computational Linguistics. Proceedings – International Conference on Computational Linguistics, Association for Computational Linguistics, Abu Dhabi, UAE, pp. 2575-2581, The 31st International Conference on Computational Linguistics, Abu Dhabi, United Arab Emirates, 19/01/25. <https://aclanthology.org/2025.coling-main.175/>

How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM. / Ji, Shaoxiong; Chen, Pinzhen.
Proceedings of the 31st International Conference on Computational Linguistics. ed. / Owen Rambow; Leo Wanner; Marianna Apidianaki; Hend Al-Khalifa; Barbara Di Eugenio; Steven Schockaert. Abu Dhabi, UAE: Association for Computational Linguistics, 2025. p. 2575-2581 (Proceedings – International Conference on Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM

AU - Ji, Shaoxiong

AU - Chen, Pinzhen

N1 - Conference code: 31

PY - 2025/1/1

Y1 - 2025/1/1

N2 - Instruction tuning a large language model with multiple languages can prepare it for multilingual downstream tasks. Nonetheless, it is yet to be determined whether having a handful of languages is sufficient, or whether the benefits increase with the inclusion of more. By fine-tuning large multilingual models on 1 to 52 languages, we present a case study on BLOOM to understand three pertinent factors affecting performance: the number of languages, language exposure, and similarity between training and test languages. Overall we found that 1) expanding language coverage in multilingual instruction tuning proves to be beneficial; 2) accuracy often significantly boots if the test language appears in the instruction mixture; 3) languages' genetic features correlate with cross-lingual transfer more than merely the number of language but different languages benefit to various degrees.

AB - Instruction tuning a large language model with multiple languages can prepare it for multilingual downstream tasks. Nonetheless, it is yet to be determined whether having a handful of languages is sufficient, or whether the benefits increase with the inclusion of more. By fine-tuning large multilingual models on 1 to 52 languages, we present a case study on BLOOM to understand three pertinent factors affecting performance: the number of languages, language exposure, and similarity between training and test languages. Overall we found that 1) expanding language coverage in multilingual instruction tuning proves to be beneficial; 2) accuracy often significantly boots if the test language appears in the instruction mixture; 3) languages' genetic features correlate with cross-lingual transfer more than merely the number of language but different languages benefit to various degrees.

M3 - Conference contribution

SN - 9798891761964

T3 - Proceedings – International Conference on Computational Linguistics

SP - 2575

EP - 2581

BT - Proceedings of the 31st International Conference on Computational Linguistics

A2 - Rambow, Owen

A2 - Wanner, Leo

A2 - Apidianaki, Marianna

A2 - Al-Khalifa, Hend

A2 - Eugenio, Barbara Di

A2 - Schockaert, Steven

PB - Association for Computational Linguistics

CY - Abu Dhabi, UAE

T2 - The 31st International Conference on Computational Linguistics

Y2 - 19 January 2025 through 24 January 2025

ER -

Ji S, Chen P. How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM. In Rambow O, Wanner L, Apidianaki M, Al-Khalifa H, Eugenio BD, Schockaert S, editors, Proceedings of the 31st International Conference on Computational Linguistics. Abu Dhabi, UAE: Association for Computational Linguistics. 2025. p. 2575-2581. (Proceedings – International Conference on Computational Linguistics).

How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this