Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers

Thomas Reitmaier; Electra Wallington; Dani Kalarikalayil Raju; Ondrej Klejch; Jennifer Pearson; Matt Jones; Peter Bell; Simon Robinson

doi:10.1145/3491102.3517639

Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers

Thomas Reitmaier, Electra Wallington, Dani Kalarikalayil Raju, Ondrej Klejch, Jennifer Pearson, Matt Jones, Peter Bell, Simon Robinson

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

Automatic Speech Recognition (ASR) researchers are turning their attention towards supporting low-resource languages, such as isiXhosa or Marathi, with only limited training resources. We report and reflect on collaborative research across ASR & HCI to situate ASR-enabled technologies to suit the needs and functions of two communities of low-resource language speakers, on the outskirts of Cape Town, South Africa and in Mumbai, India. We build on longstanding community partnerships and draw on linguistics, media studies and HCI scholarship to guide our research. We demonstrate diverse design methods to: remotely engage participants; collect speech data to test ASR models; and ultimately field-test models with users. Reflecting on the research, we identify opportunities, challenges, and use-cases of ASR, in particular to support pervasive use of WhatsApp voice messaging. Finally, we uncover implications for collaborations across ASR & HCI that advance important discussions at CHI surrounding data, ethics, and AI.

Original language	English
Title of host publication	Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
Editors	Simone Barbosa, Cliff Lampe, Caroline Appert, David A. Shamma, Steven Drucker, Julie Williamson, Koji Yatani
Place of Publication	New York, NY, USA
Publisher	Association for Computing Machinery, Inc
Number of pages	17
ISBN (Print)	9781450391573
DOIs	https://doi.org/10.1145/3491102.3517639
Publication status	Published - 29 Apr 2022
Event	The ACM CHI Conference on Human Factors in Computing Systems - New Orleans Ernest N. Morial Convention Center, New Orleans, United States Duration: 30 Apr 2022 → 5 May 2022 https://chi2022.acm.org/

Publication series

Name	CHI '22
Publisher	Association for Computing Machinery

Conference

Conference	The ACM CHI Conference on Human Factors in Computing Systems
Abbreviated title	CHI 2022
Country/Territory	United States
City	New Orleans
Period	30/04/22 → 5/05/22
Internet address	https://chi2022.acm.org/

Keywords / Materials (for Non-textual outputs)

automatic speech recognition
Speech/language
mobile devices

Access to Document

10.1145/3491102.3517639Licence: Creative Commons: Attribution (CC-BY)

Opportunities and Challenges_REITMAIER_DOA10022022_VOR_CC-BYFinal published version, 1.77 MBLicence: Creative Commons: Attribution (CC-BY)

https://dl.acm.org/doi/10.1145/3491102.3517639Licence: Creative Commons: Attribution (CC-BY)

Unmute : Opening Spoken Language Interaction to the Currently Unheard
Bell, P., Goldwater, S. & Renals, S.
EPSRC
1/12/20 → 30/11/23
Project: Research

Cite this

Reitmaier, T., Wallington, E., Kalarikalayil Raju, D., Klejch, O., Pearson, J., Jones, M., Bell, P., & Robinson, S. (2022). Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers. In S. Barbosa, C. Lampe, C. Appert, D. A. Shamma, S. Drucker, J. Williamson, & K. Yatani (Eds.), Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems Article 299 (CHI '22). Association for Computing Machinery, Inc. https://doi.org/10.1145/3491102.3517639

Reitmaier, Thomas ; Wallington, Electra ; Kalarikalayil Raju, Dani et al. / Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. editor / Simone Barbosa ; Cliff Lampe ; Caroline Appert ; David A. Shamma ; Steven Drucker ; Julie Williamson ; Koji Yatani. New York, NY, USA : Association for Computing Machinery, Inc, 2022. (CHI '22).

@inproceedings{967a09e27e2f4b5cba6da1fa58a2482d,

title = "Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers",

abstract = "Automatic Speech Recognition (ASR) researchers are turning their attention towards supporting low-resource languages, such as isiXhosa or Marathi, with only limited training resources. We report and reflect on collaborative research across ASR & HCI to situate ASR-enabled technologies to suit the needs and functions of two communities of low-resource language speakers, on the outskirts of Cape Town, South Africa and in Mumbai, India. We build on longstanding community partnerships and draw on linguistics, media studies and HCI scholarship to guide our research. We demonstrate diverse design methods to: remotely engage participants; collect speech data to test ASR models; and ultimately field-test models with users. Reflecting on the research, we identify opportunities, challenges, and use-cases of ASR, in particular to support pervasive use of WhatsApp voice messaging. Finally, we uncover implications for collaborations across ASR & HCI that advance important discussions at CHI surrounding data, ethics, and AI.",

keywords = "automatic speech recognition, Speech/language, mobile devices",

author = "Thomas Reitmaier and Electra Wallington and {Kalarikalayil Raju}, Dani and Ondrej Klejch and Jennifer Pearson and Matt Jones and Peter Bell and Simon Robinson",

note = "Funding Information: We would like to thank Minah and Manik as well as the workshop participants in Langa & Dharavi for their contribution to this work. This work was supported by Engineering and Physical Sciences Research Council grants EP/T024976/1 & EP/M022722/1. Publisher Copyright: {\textcopyright} 2022 Owner/Author.; The ACM CHI Conference on Human Factors in Computing Systems, CHI 2022 ; Conference date: 30-04-2022 Through 05-05-2022",

year = "2022",

month = apr,

day = "29",

doi = "10.1145/3491102.3517639",

language = "English",

isbn = "9781450391573",

series = "CHI '22",

publisher = "Association for Computing Machinery, Inc",

editor = "Simone Barbosa and Cliff Lampe and Caroline Appert and Shamma, {David A.} and Steven Drucker and Julie Williamson and Koji Yatani",

booktitle = "Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems",

url = "https://chi2022.acm.org/",

}

Reitmaier, T, Wallington, E, Kalarikalayil Raju, D, Klejch, O, Pearson, J, Jones, M, Bell, P & Robinson, S 2022, Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers. in S Barbosa, C Lampe, C Appert, DA Shamma, S Drucker, J Williamson & K Yatani (eds), Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems., 299, CHI '22, Association for Computing Machinery, Inc, New York, NY, USA, The ACM CHI Conference on Human Factors in Computing Systems, New Orleans, Louisiana, United States, 30/04/22. https://doi.org/10.1145/3491102.3517639

Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers. / Reitmaier, Thomas; Wallington, Electra; Kalarikalayil Raju, Dani et al.
Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. ed. / Simone Barbosa; Cliff Lampe; Caroline Appert; David A. Shamma; Steven Drucker; Julie Williamson; Koji Yatani. New York, NY, USA: Association for Computing Machinery, Inc, 2022. 299 (CHI '22).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers

AU - Reitmaier, Thomas

AU - Wallington, Electra

AU - Kalarikalayil Raju, Dani

AU - Klejch, Ondrej

AU - Pearson, Jennifer

AU - Jones, Matt

AU - Bell, Peter

AU - Robinson, Simon

N1 - Funding Information: We would like to thank Minah and Manik as well as the workshop participants in Langa & Dharavi for their contribution to this work. This work was supported by Engineering and Physical Sciences Research Council grants EP/T024976/1 & EP/M022722/1. Publisher Copyright: © 2022 Owner/Author.

PY - 2022/4/29

Y1 - 2022/4/29

N2 - Automatic Speech Recognition (ASR) researchers are turning their attention towards supporting low-resource languages, such as isiXhosa or Marathi, with only limited training resources. We report and reflect on collaborative research across ASR & HCI to situate ASR-enabled technologies to suit the needs and functions of two communities of low-resource language speakers, on the outskirts of Cape Town, South Africa and in Mumbai, India. We build on longstanding community partnerships and draw on linguistics, media studies and HCI scholarship to guide our research. We demonstrate diverse design methods to: remotely engage participants; collect speech data to test ASR models; and ultimately field-test models with users. Reflecting on the research, we identify opportunities, challenges, and use-cases of ASR, in particular to support pervasive use of WhatsApp voice messaging. Finally, we uncover implications for collaborations across ASR & HCI that advance important discussions at CHI surrounding data, ethics, and AI.

AB - Automatic Speech Recognition (ASR) researchers are turning their attention towards supporting low-resource languages, such as isiXhosa or Marathi, with only limited training resources. We report and reflect on collaborative research across ASR & HCI to situate ASR-enabled technologies to suit the needs and functions of two communities of low-resource language speakers, on the outskirts of Cape Town, South Africa and in Mumbai, India. We build on longstanding community partnerships and draw on linguistics, media studies and HCI scholarship to guide our research. We demonstrate diverse design methods to: remotely engage participants; collect speech data to test ASR models; and ultimately field-test models with users. Reflecting on the research, we identify opportunities, challenges, and use-cases of ASR, in particular to support pervasive use of WhatsApp voice messaging. Finally, we uncover implications for collaborations across ASR & HCI that advance important discussions at CHI surrounding data, ethics, and AI.

KW - automatic speech recognition

KW - Speech/language

KW - mobile devices

U2 - 10.1145/3491102.3517639

DO - 10.1145/3491102.3517639

M3 - Conference contribution

SN - 9781450391573

T3 - CHI '22

BT - Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

A2 - Barbosa, Simone

A2 - Lampe, Cliff

A2 - Appert, Caroline

A2 - Shamma, David A.

A2 - Drucker, Steven

A2 - Williamson, Julie

A2 - Yatani, Koji

PB - Association for Computing Machinery, Inc

CY - New York, NY, USA

T2 - The ACM CHI Conference on Human Factors in Computing Systems

Y2 - 30 April 2022 through 5 May 2022

ER -

Reitmaier T, Wallington E, Kalarikalayil Raju D, Klejch O, Pearson J, Jones M et al. Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers. In Barbosa S, Lampe C, Appert C, Shamma DA, Drucker S, Williamson J, Yatani K, editors, Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: Association for Computing Machinery, Inc. 2022. 299. (CHI '22). doi: 10.1145/3491102.3517639

Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers

Abstract / Description of output

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Projects

Unmute : Opening Spoken Language Interaction to the Currently Unheard

Cite this