First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models

Naomi Saphra, Eve Fleisig, Kyunghyun Cho, Adam Lopez

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Many NLP researchers are experiencing an existential crisis triggered by the astonishing success of ChatGPT and other systems based on large language models (LLMs). After such a disruptive change to our understanding of the field, what is left to do? Taking a historical lens, we look for guidance from the first era of LLMs, which began in 2005 with large ngram models for machine translation (MT). We identify durable lessons from the first era, and more importantly, we identify evergreen problems where NLP researchers can continue to make meaningful contributions in areas where LLMs are ascendant. We argue that disparities in scale are transient and researchers can work to reduce them; that data, rather than hardware, is still a bottleneck for many applications; that meaningful realistic evaluation is still an open problem; and that there is still room for speculative approaches.
Original languageEnglish
Title of host publicationProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics
PublisherAssociation for Computational Linguistics
Number of pages17
Publication statusAccepted/In press - 13 Mar 2024
Event2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics - Mexico City, Mexico
Duration: 16 Jun 202421 Jun 2024
https://2024.naacl.org/

Conference

Conference2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Abbreviated titleNAACL 2024
Country/TerritoryMexico
CityMexico City
Period16/06/2421/06/24
Internet address

Fingerprint

Dive into the research topics of 'First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models'. Together they form a unique fingerprint.

Cite this