Analysis of statistical parametric and unit selection speech synthesis systems applied to emotional speech

Roberto Barra-Chicote, Junichi Yamagishi, Simon King, Juan Manuel Montero, Javier Macias-Guarasa

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

We have applied two state-of-the-art speech synthesis techniques (unit selection and HMM-based synthesis) to the synthesis of emotional speech. A series of carefully designed perceptual tests to evaluate speech quality, emotion identification rates and emotional strength were used for the six emotions which we recorded - happiness, sadness, anger, surprise, fear, disgust. For the HMM-based method, we evaluated spectral and source components separately and identified which components contribute to which emotion.

Our analysis shows that, although the HMM method produces significantly better neutral speech, the two methods produce emotional speech of similar quality, except for emotions having context-dependent prosodic patterns. Whilst synthetic speech produced using the unit selection method has better emotional strength scores than the HMM-based method, the HMM-based method has the ability to manipulate the emotional strength. For emotions that are characterized by both spectral and prosodic components, synthetic speech using unit selection methods was more accurately identified by listeners. For emotions mainly characterized by prosodic components, HMM-based synthetic speech was more accurately identified. This finding differs from previous results regarding listener judgements of speaker similarity for neutral speech. We conclude that unit selection methods require improvements to prosodic modeling and that HMM-based methods require improvements to spectral modeling for emotional speech. Certain emotions cannot be reproduced well by either method.
Original languageEnglish
Pages (from-to)394-404
Number of pages11
JournalSpeech Communication
Issue number5
Publication statusPublished - 1 May 2010

Keywords / Materials (for Non-textual outputs)

  • Emotional speech synthesis
  • HMM-based synthesis
  • Unit selection


Dive into the research topics of 'Analysis of statistical parametric and unit selection speech synthesis systems applied to emotional speech'. Together they form a unique fingerprint.

Cite this