Self-supervised speech representations display some human-like cross-linguistic perceptual abilities

Joselyn Rodriguez; Kamala Sreepada; Ruolan Leslie Famularo; Sharon Goldwater; Naomi H. Feldman

Self-supervised speech representations display some human-like cross-linguistic perceptual abilities

Joselyn Rodriguez^*, Kamala Sreepada, Ruolan Leslie Famularo, Sharon Goldwater, Naomi H. Feldman

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

State of the art models in automatic speech recognition have shown remarkable improvements due to modern self-supervised (SSL) transformer-based architectures such as wav2vec 2.0 (Baevski et al., 2020). However, how these models encode phonetic information is still not well understood. We explore whether SSL speech models display a linguistic property that characterizes human speech perception: language specificity. We show that while wav2vec 2.0 displays an overall language specificity effect when tested on Hindi vs. English, it does not resemble human speech perception when tested on finer-grained differences in Hindi speech contrasts.

Original language	English
Title of host publication	Proceedings of the 28th Conference on Computational Natural Language Learning
Publisher	ACL Anthology
Pages	1-6
Number of pages	6
Publication status	Accepted/In press - 24 Sept 2024
Event	The 28th Conference on Computational Natural Language Learning - Hyatt Regency Miami Hotel, Miami, United States Duration: 15 Nov 2024 → 16 Nov 2024 Conference number: 28 https://conll.org/2024

Conference

Conference	The 28th Conference on Computational Natural Language Learning
Abbreviated title	CoNLL 2024
Country/Territory	United States
City	Miami
Period	15/11/24 → 16/11/24
Internet address	https://conll.org/2024

Access to Document

RodriguezEtalCoNLL2024Self-supervisedSpeechRepresentationsAccepted author manuscript, 1.46 MBLicence: Creative Commons: Attribution (CC-BY)

Cite this

@inproceedings{e02d07442455482291217e91ecbe315a,

title = "Self-supervised speech representations display some human-like cross-linguistic perceptual abilities",

abstract = "State of the art models in automatic speech recognition have shown remarkable improvements due to modern self-supervised (SSL) transformer-based architectures such as wav2vec 2.0 (Baevski et al., 2020). However, how these models encode phonetic information is still not well understood. We explore whether SSL speech models display a linguistic property that characterizes human speech perception: language specificity. We show that while wav2vec 2.0 displays an overall language specificity effect when tested on Hindi vs. English, it does not resemble human speech perception when tested on finer-grained differences in Hindi speech contrasts.",

author = "Joselyn Rodriguez and Kamala Sreepada and Famularo, {Ruolan Leslie} and Sharon Goldwater and Feldman, {Naomi H.}",

year = "2024",

month = sep,

day = "24",

language = "English",

pages = "1--6",

booktitle = "Proceedings of the 28th Conference on Computational Natural Language Learning",

publisher = "ACL Anthology",

note = "The 28th Conference on Computational Natural Language Learning , CoNLL 2024 ; Conference date: 15-11-2024 Through 16-11-2024",

url = "https://conll.org/2024",

}

Rodriguez, J, Sreepada, K, Famularo, RL, Goldwater, S & Feldman, NH 2024, Self-supervised speech representations display some human-like cross-linguistic perceptual abilities. in Proceedings of the 28th Conference on Computational Natural Language Learning. ACL Anthology, pp. 1-6, The 28th Conference on Computational Natural Language Learning , Miami, Florida, United States, 15/11/24.

TY - GEN

T1 - Self-supervised speech representations display some human-like cross-linguistic perceptual abilities

AU - Rodriguez, Joselyn

AU - Sreepada, Kamala

AU - Famularo, Ruolan Leslie

AU - Goldwater, Sharon

AU - Feldman, Naomi H.

N1 - Conference code: 28

PY - 2024/9/24

Y1 - 2024/9/24

N2 - State of the art models in automatic speech recognition have shown remarkable improvements due to modern self-supervised (SSL) transformer-based architectures such as wav2vec 2.0 (Baevski et al., 2020). However, how these models encode phonetic information is still not well understood. We explore whether SSL speech models display a linguistic property that characterizes human speech perception: language specificity. We show that while wav2vec 2.0 displays an overall language specificity effect when tested on Hindi vs. English, it does not resemble human speech perception when tested on finer-grained differences in Hindi speech contrasts.

AB - State of the art models in automatic speech recognition have shown remarkable improvements due to modern self-supervised (SSL) transformer-based architectures such as wav2vec 2.0 (Baevski et al., 2020). However, how these models encode phonetic information is still not well understood. We explore whether SSL speech models display a linguistic property that characterizes human speech perception: language specificity. We show that while wav2vec 2.0 displays an overall language specificity effect when tested on Hindi vs. English, it does not resemble human speech perception when tested on finer-grained differences in Hindi speech contrasts.

M3 - Conference contribution

SP - 1

EP - 6

BT - Proceedings of the 28th Conference on Computational Natural Language Learning

PB - ACL Anthology

T2 - The 28th Conference on Computational Natural Language Learning

Y2 - 15 November 2024 through 16 November 2024

ER -

Self-supervised speech representations display some human-like cross-linguistic perceptual abilities

Abstract / Description of output

Conference

Access to Document

Fingerprint

Cite this