Speaker-Independent Classification of Phonetic Segments from Raw Ultrasound in Child Speech

Manuel Sam Ribeiro; Aciel Eshky; Korin Richmond; Steve Renals

doi:10.1109/ICASSP.2019.8683564

Speaker-Independent Classification of Phonetic Segments from Raw Ultrasound in Child Speech

Manuel Sam Ribeiro, Aciel Eshky, Korin Richmond, Steve Renals

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

Ultrasound tongue imaging (UTI) provides a convenient way to visualize the vocal tract during speech production. UTI is increasingly being used for speech therapy, making it important to develop automatic methods to assist various time-consuming manual tasks currently performed by speech therapists. A key challenge is to generalize the automatic processing of ultrasound tongue images to previously unseen speakers. In this work, we investigate the classification of phonetic segments (tongue shapes) from raw ultrasound recordings under several training scenarios: speaker-dependent, multi-speaker, speaker-independent, and speaker-adapted. We observe that models underperform when applied to data from speakers not seen at training time. However, when provided with minimal additional speaker information, such as the mean ultrasound frame, the models generalize better to unseen speakers.

Original language	English
Title of host publication	ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Place of Publication	Brighton, United Kingdom
Publisher	Institute of Electrical and Electronics Engineers
Pages	1328-1332
Number of pages	5
ISBN (Electronic)	978-1-4799-8131-1
ISBN (Print)	978-1-4799-8132-8
DOIs	https://doi.org/10.1109/ICASSP.2019.8683564
Publication status	Published - 17 Apr 2019
Event	44th International Conference on Acoustics, Speech, and Signal Processing: Signal Processing: Empowering Science and Technology for Humankind - Brighton , United Kingdom Duration: 12 May 2019 → 17 May 2019 Conference number: 44 https://2019.ieeeicassp.org/

Publication series

Name
Publisher	IEEE
ISSN (Print)	1520-6149
ISSN (Electronic)	2379-190X

Conference

Conference	44th International Conference on Acoustics, Speech, and Signal Processing
Abbreviated title	ICASSP 2019
Country/Territory	United Kingdom
City	Brighton
Period	12/05/19 → 17/05/19
Internet address	https://2019.ieeeicassp.org/

Keywords / Materials (for Non-textual outputs)

ultrasound
ultrasound tongue imaging
speaker independent
speech therapy
Child Speech

Access to Document

10.1109/ICASSP.2019.8683564

ribeiro_icassp19
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Accepted author manuscript, 2.01 MBLicence: All Rights Reserved

https://ieeexplore.ieee.org/document/8683564

Cite this

Ribeiro, M. S., Eshky, A., Richmond, K., & Renals, S. (2019). Speaker-Independent Classification of Phonetic Segments from Raw Ultrasound in Child Speech. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1328-1332). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICASSP.2019.8683564

@inproceedings{87071137b5394896b73b550151ee5cd0,

title = "Speaker-Independent Classification of Phonetic Segments from Raw Ultrasound in Child Speech",

abstract = "Ultrasound tongue imaging (UTI) provides a convenient way to visualize the vocal tract during speech production. UTI is increasingly being used for speech therapy, making it important to develop automatic methods to assist various time-consuming manual tasks currently performed by speech therapists. A key challenge is to generalize the automatic processing of ultrasound tongue images to previously unseen speakers. In this work, we investigate the classification of phonetic segments (tongue shapes) from raw ultrasound recordings under several training scenarios: speaker-dependent, multi-speaker, speaker-independent, and speaker-adapted. We observe that models underperform when applied to data from speakers not seen at training time. However, when provided with minimal additional speaker information, such as the mean ultrasound frame, the models generalize better to unseen speakers.",

keywords = "ultrasound, ultrasound tongue imaging, speaker independent, speech therapy, Child Speech",

author = "Ribeiro, {Manuel Sam} and Aciel Eshky and Korin Richmond and Steve Renals",

year = "2019",

month = apr,

day = "17",

doi = "10.1109/ICASSP.2019.8683564",

language = "English",

isbn = "978-1-4799-8132-8",

publisher = "Institute of Electrical and Electronics Engineers",

pages = "1328--1332",

booktitle = "ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

address = "United States",

note = "44th International Conference on Acoustics, Speech, and Signal Processing : Signal Processing: Empowering Science and Technology for Humankind, ICASSP 2019 ; Conference date: 12-05-2019 Through 17-05-2019",

url = "https://2019.ieeeicassp.org/",

}

Ribeiro, MS, Eshky, A, Richmond, K & Renals, S 2019, Speaker-Independent Classification of Phonetic Segments from Raw Ultrasound in Child Speech. in ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Institute of Electrical and Electronics Engineers, Brighton, United Kingdom, pp. 1328-1332, 44th International Conference on Acoustics, Speech, and Signal Processing, Brighton , United Kingdom, 12/05/19. https://doi.org/10.1109/ICASSP.2019.8683564

Speaker-Independent Classification of Phonetic Segments from Raw Ultrasound in Child Speech. / Ribeiro, Manuel Sam; Eshky, Aciel; Richmond, Korin et al.
ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Brighton, United Kingdom: Institute of Electrical and Electronics Engineers, 2019. p. 1328-1332.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Speaker-Independent Classification of Phonetic Segments from Raw Ultrasound in Child Speech

AU - Ribeiro, Manuel Sam

AU - Eshky, Aciel

AU - Richmond, Korin

AU - Renals, Steve

N1 - Conference code: 44

PY - 2019/4/17

Y1 - 2019/4/17

N2 - Ultrasound tongue imaging (UTI) provides a convenient way to visualize the vocal tract during speech production. UTI is increasingly being used for speech therapy, making it important to develop automatic methods to assist various time-consuming manual tasks currently performed by speech therapists. A key challenge is to generalize the automatic processing of ultrasound tongue images to previously unseen speakers. In this work, we investigate the classification of phonetic segments (tongue shapes) from raw ultrasound recordings under several training scenarios: speaker-dependent, multi-speaker, speaker-independent, and speaker-adapted. We observe that models underperform when applied to data from speakers not seen at training time. However, when provided with minimal additional speaker information, such as the mean ultrasound frame, the models generalize better to unseen speakers.

AB - Ultrasound tongue imaging (UTI) provides a convenient way to visualize the vocal tract during speech production. UTI is increasingly being used for speech therapy, making it important to develop automatic methods to assist various time-consuming manual tasks currently performed by speech therapists. A key challenge is to generalize the automatic processing of ultrasound tongue images to previously unseen speakers. In this work, we investigate the classification of phonetic segments (tongue shapes) from raw ultrasound recordings under several training scenarios: speaker-dependent, multi-speaker, speaker-independent, and speaker-adapted. We observe that models underperform when applied to data from speakers not seen at training time. However, when provided with minimal additional speaker information, such as the mean ultrasound frame, the models generalize better to unseen speakers.

KW - ultrasound

KW - ultrasound tongue imaging

KW - speaker independent

KW - speech therapy

KW - Child Speech

UR - https://ieeexplore.ieee.org/document/8683564

U2 - 10.1109/ICASSP.2019.8683564

DO - 10.1109/ICASSP.2019.8683564

M3 - Conference contribution

SN - 978-1-4799-8132-8

SP - 1328

EP - 1332

BT - ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - Institute of Electrical and Electronics Engineers

CY - Brighton, United Kingdom

T2 - 44th International Conference on Acoustics, Speech, and Signal Processing

Y2 - 12 May 2019 through 17 May 2019

ER -

Speaker-Independent Classification of Phonetic Segments from Raw Ultrasound in Child Speech

Abstract / Description of output

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

Ultrax2020: Ultrasound Technology for Optimising the Treatment of Speech Disorders

Cite this

Speaker-Independent Classification of Phonetic Segments from Raw Ultrasound in Child Speech

Abstract / Description of output

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

Projects

Ultrax2020: Ultrasound Technology for Optimising the Treatment of Speech Disorders

Cite this