On the Evaluation of Inversion Mapping Performance in the Acoustic Domain

Korin Richmond, Zhenhua Ling, Junichi Yamagishi, Benigno Ur?a

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The two measures typically used to assess the performance of an inversion mapping method, where the aim is to estimate what articulator movements gave rise to a given acoustic signal, are root mean squared (RMS) error and correlation. In this paper, we investigate whether ``task-based'' evaluation using an articulatory-controllable HMM-based speech synthesis system can give useful additional information to complement these measures. To assess the usefulness of this evaluation approach, we use articulator trajectories estimated by a range of different inversion mapping methods as input to the synthesiser, and measure their performance in the acoustic domain in terms of RMS error of the generated acoustic parameters and with a listening test involving 30 participants. We then compare these results with the standard RMS error and correlation measures calculated in the articulatory domain. Interestingly, in the acoustic evaluation we observe one method performs with no statistically significant difference from measured articulatory data, and cases where statistically significant differences between methods exist which are not reflected in the results of the two standard measures. From our results, we conclude such task-based evaluation can indeed provide interesting extra information, and gives a useful way to compare inversion methods.
Original languageEnglish
Title of host publicationProc. Interspeech
Pages1012-1016
Number of pages5
Publication statusPublished - Aug 2013

Keywords / Materials (for Non-textual outputs)

  • Inversion mapping, evaluation, HMM synthesis

Fingerprint

Dive into the research topics of 'On the Evaluation of Inversion Mapping Performance in the Acoustic Domain'. Together they form a unique fingerprint.

Cite this