Context-sensitive evaluation of automatic speech recognition: Considering user experience & language variation

Nina Markl; Catherine Lai

Context-sensitive evaluation of automatic speech recognition: Considering user experience & language variation

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Commercial Automatic Speech Recognition (ASR) systems tend to show systemic predictive bias for marginalised speaker/user groups. We highlight the need for an interdisciplinary and context-sensitive approach to documenting this bias incorporating perspectives and methods from sociolinguistics, speech and language technology and human-computer interaction in the context of a case study. We argue evaluation of ASR systems should be disaggregated by speaker group, include qualitative error analysis, and consider user experience in a broader sociolinguistic and social context.

Original language	English
Title of host publication	Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing
Editors	Su Lin Blodgett
Publisher	ACL Anthology
Pages	34-40
Publication status	Published - Apr 2021
Event	Bridging Human--Computer Interaction and Natural Language Processing Workshop: Workshop at EACL 2021 - Duration: 19 Apr 2021 → 20 Apr 2021 https://sites.google.com/view/hciandnlp

Workshop

Workshop	Bridging Human--Computer Interaction and Natural Language Processing Workshop
Abbreviated title	HCI + NLP
Period	19/04/21 → 20/04/21
Internet address	https://sites.google.com/view/hciandnlp

Keywords

ASR
HCI
sociolinguistics
speech technology
bias

Access to Document

MarklLai2021ContextSensitiveFinal published version, 205 KBLicence: Creative Commons: Attribution (CC-BY)

https://www.aclweb.org/anthology/2021.hcinlp-1.6

1 Article

The Lothian Diary Project: Sociolinguistic methods during the COVID-19 lockdown
Hall-Lew, L., Cowie, C., Lai, C., Markl, N., McNulty, S., Liu, S-J. S., Llewellyn, C., Alex, B., Elliott Slosarova, Z. & Klingler, A., 22 Feb 2022, (E-pub ahead of print) In: Linguistics Vanguard. p. 1 10 p.
Research output: Contribution to journal › Article › peer-review

Open Access
File

Cite this

@inproceedings{e2c7bfc27ed2477eab907467ec980a1e,

title = "Context-sensitive evaluation of automatic speech recognition: Considering user experience & language variation",

abstract = "Commercial Automatic Speech Recognition (ASR) systems tend to show systemic predictive bias for marginalised speaker/user groups. We highlight the need for an interdisciplinary and context-sensitive approach to documenting this bias incorporating perspectives and methods from sociolinguistics, speech and language technology and human-computer interaction in the context of a case study. We argue evaluation of ASR systems should be disaggregated by speaker group, include qualitative error analysis, and consider user experience in a broader sociolinguistic and social context.",

keywords = "ASR, HCI, sociolinguistics, speech technology, bias",

author = "Nina Markl and Catherine Lai",

year = "2021",

month = apr,

language = "English",

pages = "34--40",

editor = "Blodgett, {Su Lin}",

booktitle = "Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing",

publisher = "ACL Anthology",

note = "Bridging Human--Computer Interaction and Natural Language Processing Workshop : Workshop at EACL 2021, HCI + NLP ; Conference date: 19-04-2021 Through 20-04-2021",

url = "https://sites.google.com/view/hciandnlp",

}

Markl, N & Lai, C 2021, Context-sensitive evaluation of automatic speech recognition: Considering user experience & language variation. in SL Blodgett (ed.), Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing. ACL Anthology, pp. 34-40, Bridging Human--Computer Interaction and Natural Language Processing Workshop, 19/04/21. <https://www.aclweb.org/anthology/2021.hcinlp-1.6>

Context-sensitive evaluation of automatic speech recognition : Considering user experience & language variation. / Markl, Nina; Lai, Catherine.

Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing. ed. / Su Lin Blodgett. ACL Anthology, 2021. p. 34-40.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Context-sensitive evaluation of automatic speech recognition

T2 - Bridging Human--Computer Interaction and Natural Language Processing Workshop

AU - Markl, Nina

AU - Lai, Catherine

PY - 2021/4

Y1 - 2021/4

N2 - Commercial Automatic Speech Recognition (ASR) systems tend to show systemic predictive bias for marginalised speaker/user groups. We highlight the need for an interdisciplinary and context-sensitive approach to documenting this bias incorporating perspectives and methods from sociolinguistics, speech and language technology and human-computer interaction in the context of a case study. We argue evaluation of ASR systems should be disaggregated by speaker group, include qualitative error analysis, and consider user experience in a broader sociolinguistic and social context.

AB - Commercial Automatic Speech Recognition (ASR) systems tend to show systemic predictive bias for marginalised speaker/user groups. We highlight the need for an interdisciplinary and context-sensitive approach to documenting this bias incorporating perspectives and methods from sociolinguistics, speech and language technology and human-computer interaction in the context of a case study. We argue evaluation of ASR systems should be disaggregated by speaker group, include qualitative error analysis, and consider user experience in a broader sociolinguistic and social context.

KW - ASR

KW - HCI

KW - sociolinguistics

KW - speech technology

KW - bias

M3 - Conference contribution

SP - 34

EP - 40

BT - Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing

A2 - Blodgett, Su Lin

PB - ACL Anthology

Y2 - 19 April 2021 through 20 April 2021

ER -

Context-sensitive evaluation of automatic speech recognition: Considering user experience & language variation

Abstract

Workshop

Keywords

Access to Document

Fingerprint

Research output

The Lothian Diary Project: Sociolinguistic methods during the COVID-19 lockdown

Cite this