Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems

Evgeniia Razumovskaia, Goran Glavaš, Olga Majewska, Edoardo M. Ponti, Anna Korhonen, Ivan Vulić

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

In task-oriented dialogue (ToD), a user holds a conversation with an artificial agentwith the aim of completing a concrete task. Although this technology represents one ofthe central objectives of AI and has been the focus of ever more intense research anddevelopment efforts, it is currently limited to a few narrow domains (e.g., food ordering,ticket booking) and a handful of languages (e.g., English, Chinese). This work provides anextensive overview of existing methods and resources in multilingualToDas an entry pointto this exciting and emerging field. We find that the most critical factor preventing thecreation of truly multilingualToDsystems is the lack of datasets in most languages forboth training and evaluation. In fact, acquiring annotations or human feedback for eachcomponent of modular systems or for data-hungry end-to-end systems is expensive andtedious. Hence, state-of-the-art approaches to multilingualToDmostly rely on (zero- orfew-shot) cross-lingual transfer from resource-rich languages (almost exclusively English),either by means of (i) machine translation or (ii) multilingual representations. These approaches are currently viable only for typologically similar languages and languages with parallel / monolingual corpora available. On the other hand, their effectiveness beyond theseboundaries is doubtful or hard to assess due to the lack of linguistically diverse benchmarks(especially for natural language generation and end-to-end evaluation). To overcome this limitation, we draw parallels between components of the ToD pipeline and other NLP tasks,which can inspire solutions for learning in low-resource scenarios. Finally, we list additional challenges that multilinguality poses for related areas (such as speech, fluency in generated text, and human-centred evaluation), and indicate future directions that hold promise to further expand language coverage and dialogue capabilities of current ToD systems.
Original languageEnglish
Pages (from-to)1351-1402
Number of pages52
JournalJournal of Artificial Intelligence Research
Publication statusPublished - 13 Jul 2022

Keywords / Materials (for Non-textual outputs)

  • Computation and Language (cs.CL)
  • FOS: Computer and information sciences


Dive into the research topics of 'Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems'. Together they form a unique fingerprint.

Cite this