The unreasonable effectiveness of large language models for low-resource clause-level morphology: In-context generalization or prior exposure?

Coleman Haley

doi:10.18653/v1/2024.americasnlp-1.20

The unreasonable effectiveness of large language models for low-resource clause-level morphology: In-context generalization or prior exposure?

Coleman Haley

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This paper describes the submission of Team “Giving it a Shot” to the AmericasNLP 2024 Shared Task on Creation of Educational Materials for Indigenous Languages. We use a simple few-shot prompting approach with several state of the art large language models, achieving competitive performance on the shared task, with our best system placing third overall. We perform a preliminary analysis to determine to what degree the performance of our model is due to prior exposure to the task languages, finding that generally our performance is better explained as being derived from in-context learning capabilities.

Original language	English
Title of host publication	Proceedings of the 4th Workshop on Natural Language Processing for Indigenous Languages of the Americas (AmericasNLP 2024)
Editors	Manuel Mager, Abteen Ebrahimi, Shruti Rijhwani, Arturo Oncevay, Luis Chiruzzo, Robert Pugh, Katharina von der Wense
Publisher	Association for Computational Linguistics (ACL)
Pages	174-178
Number of pages	5
ISBN (Electronic)	9798891761087
DOIs	https://doi.org/10.18653/v1/2024.americasnlp-1.20
Publication status	Published - 21 Jun 2024
Event	The 4th Workshop on NLP for Indigenous Languages of the Americas - Mexico City, Mexico Duration: 21 Jun 2024 → 21 Jun 2024

Workshop

Workshop	The 4th Workshop on NLP for Indigenous Languages of the Americas
Abbreviated title	AmericasNLP 2024
Country/Territory	Mexico
City	Mexico City
Period	21/06/24 → 21/06/24

Access to Document

10.18653/v1/2024.americasnlp-1.20Licence: Creative Commons: Attribution (CC-BY)

HaleyAmericasNLP2024TheUnreasonableEffectivenessOfLarge-languageModelsFinal published version, 129 KBLicence: Creative Commons: Attribution (CC-BY)

Cite this

Haley, C. (2024). The unreasonable effectiveness of large language models for low-resource clause-level morphology: In-context generalization or prior exposure? In M. Mager, A. Ebrahimi, S. Rijhwani, A. Oncevay, L. Chiruzzo, R. Pugh, & K. von der Wense (Eds.), Proceedings of the 4th Workshop on Natural Language Processing for Indigenous Languages of the Americas (AmericasNLP 2024) (pp. 174-178). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2024.americasnlp-1.20

Haley, Coleman. / The unreasonable effectiveness of large language models for low-resource clause-level morphology : In-context generalization or prior exposure?. Proceedings of the 4th Workshop on Natural Language Processing for Indigenous Languages of the Americas (AmericasNLP 2024). editor / Manuel Mager ; Abteen Ebrahimi ; Shruti Rijhwani ; Arturo Oncevay ; Luis Chiruzzo ; Robert Pugh ; Katharina von der Wense. Association for Computational Linguistics (ACL), 2024. pp. 174-178

@inproceedings{a1a3cfb198e94b06b041b89013648b31,

title = "The unreasonable effectiveness of large language models for low-resource clause-level morphology: In-context generalization or prior exposure?",

abstract = "This paper describes the submission of Team “Giving it a Shot” to the AmericasNLP 2024 Shared Task on Creation of Educational Materials for Indigenous Languages. We use a simple few-shot prompting approach with several state of the art large language models, achieving competitive performance on the shared task, with our best system placing third overall. We perform a preliminary analysis to determine to what degree the performance of our model is due to prior exposure to the task languages, finding that generally our performance is better explained as being derived from in-context learning capabilities.",

author = "Coleman Haley",

year = "2024",

month = jun,

day = "21",

doi = "10.18653/v1/2024.americasnlp-1.20",

language = "English",

pages = "174--178",

editor = "Manuel Mager and Abteen Ebrahimi and Shruti Rijhwani and Arturo Oncevay and Luis Chiruzzo and Robert Pugh and {von der Wense}, Katharina",

booktitle = "Proceedings of the 4th Workshop on Natural Language Processing for Indigenous Languages of the Americas (AmericasNLP 2024)",

publisher = "Association for Computational Linguistics (ACL)",

note = "The 4th Workshop on NLP for Indigenous Languages of the Americas, AmericasNLP 2024 ; Conference date: 21-06-2024 Through 21-06-2024",

}

Haley, C 2024, The unreasonable effectiveness of large language models for low-resource clause-level morphology: In-context generalization or prior exposure? in M Mager, A Ebrahimi, S Rijhwani, A Oncevay, L Chiruzzo, R Pugh & K von der Wense (eds), Proceedings of the 4th Workshop on Natural Language Processing for Indigenous Languages of the Americas (AmericasNLP 2024). Association for Computational Linguistics (ACL), pp. 174-178, The 4th Workshop on NLP for Indigenous Languages of the Americas, Mexico City, Mexico, 21/06/24. https://doi.org/10.18653/v1/2024.americasnlp-1.20

The unreasonable effectiveness of large language models for low-resource clause-level morphology: In-context generalization or prior exposure? / Haley, Coleman.
Proceedings of the 4th Workshop on Natural Language Processing for Indigenous Languages of the Americas (AmericasNLP 2024). ed. / Manuel Mager; Abteen Ebrahimi; Shruti Rijhwani; Arturo Oncevay; Luis Chiruzzo; Robert Pugh; Katharina von der Wense. Association for Computational Linguistics (ACL), 2024. p. 174-178.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - The unreasonable effectiveness of large language models for low-resource clause-level morphology

T2 - The 4th Workshop on NLP for Indigenous Languages of the Americas

AU - Haley, Coleman

PY - 2024/6/21

Y1 - 2024/6/21

N2 - This paper describes the submission of Team “Giving it a Shot” to the AmericasNLP 2024 Shared Task on Creation of Educational Materials for Indigenous Languages. We use a simple few-shot prompting approach with several state of the art large language models, achieving competitive performance on the shared task, with our best system placing third overall. We perform a preliminary analysis to determine to what degree the performance of our model is due to prior exposure to the task languages, finding that generally our performance is better explained as being derived from in-context learning capabilities.

AB - This paper describes the submission of Team “Giving it a Shot” to the AmericasNLP 2024 Shared Task on Creation of Educational Materials for Indigenous Languages. We use a simple few-shot prompting approach with several state of the art large language models, achieving competitive performance on the shared task, with our best system placing third overall. We perform a preliminary analysis to determine to what degree the performance of our model is due to prior exposure to the task languages, finding that generally our performance is better explained as being derived from in-context learning capabilities.

U2 - 10.18653/v1/2024.americasnlp-1.20

DO - 10.18653/v1/2024.americasnlp-1.20

M3 - Conference contribution

SP - 174

EP - 178

BT - Proceedings of the 4th Workshop on Natural Language Processing for Indigenous Languages of the Americas (AmericasNLP 2024)

A2 - Mager, Manuel

A2 - Ebrahimi, Abteen

A2 - Rijhwani, Shruti

A2 - Oncevay, Arturo

A2 - Chiruzzo, Luis

A2 - Pugh, Robert

A2 - von der Wense, Katharina

PB - Association for Computational Linguistics (ACL)

Y2 - 21 June 2024 through 21 June 2024

ER -

Haley C. The unreasonable effectiveness of large language models for low-resource clause-level morphology: In-context generalization or prior exposure? In Mager M, Ebrahimi A, Rijhwani S, Oncevay A, Chiruzzo L, Pugh R, von der Wense K, editors, Proceedings of the 4th Workshop on Natural Language Processing for Indigenous Languages of the Americas (AmericasNLP 2024). Association for Computational Linguistics (ACL). 2024. p. 174-178 doi: 10.18653/v1/2024.americasnlp-1.20

The unreasonable effectiveness of large language models for low-resource clause-level morphology: In-context generalization or prior exposure?

Abstract

Workshop

Access to Document

Fingerprint

Cite this