Ultrasound tongue imaging for diarization and alignment of child speech therapy sessions

Manuel Ribeiro, Aciel Eshky, Korin Richmond, Steve Renals

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

We investigate the automatic processing of child speech therapy sessions using ultrasound visual biofeedback, with a specific focus on complementing acoustic features with ultrasound images of the tongue for the tasks of speaker diarization and time-alignment of target words. For speaker diarization, we propose an ultrasound-based time-domain signal which we call estimated tongue activity. For word-alignment, we augment an acoustic model with low-dimensional representations of ultrasound images of the tongue, learned by a convolutional neural network. We conduct our experiments using the Ultrasuite repository of ultrasound and speech recordings for child speech therapy sessions. For both tasks, we observe that systems augmented with ultrasound data outperform corresponding systems using only the audio signal.
Original languageEnglish
Title of host publicationINTERSPEECH 2019: Proceedings of the 20th Annual Conference of the International Speech Communication Association (ISCA)
Place of PublicationGraz, Austria
PublisherInternational Speech Communication Association
Number of pages5
Publication statusPublished - 19 Sept 2019
EventInterspeech 2019 - Graz, Austria
Duration: 15 Sept 201919 Sept 2019

Publication series

PublisherInternational Speech Communication Association
ISSN (Electronic)1990-9772


ConferenceInterspeech 2019
Internet address

Keywords / Materials (for Non-textual outputs)

  • speech recognition
  • speaker diarization
  • ultrasound tongue imaging
  • speech therapy
  • child speech
  • Ultrasuite


Dive into the research topics of 'Ultrasound tongue imaging for diarization and alignment of child speech therapy sessions'. Together they form a unique fingerprint.

Cite this