An overview of voice conversion and its challenges: From statistical modeling to deep learning

Berrak Sisman, Junichi Yamagishi, Simon King, Haizhou Li

Research output: Contribution to journalArticlepeer-review

Abstract

Speaker identity is one of the important characteristics of human speech. In voice conversion, we change the speaker identity from one to another, while keeping the linguistic content unchanged. Voice conversion involves multiple speech processing techniques, such as speech analysis, spectral conversion, prosody conversion, speaker characterization, and vocoding. With the recent advances in theory and practice, we are now able to produce human-like voice quality with high speaker similarity. In this paper, we provide a comprehensive overview of the state-of-the-art of voice conversion techniques and their performance evaluation methods from the statistical approaches to deep learning, and discuss their promise and limitations. We will also report the recent Voice Conversion Challenges (VCC), the performance of the current state of technology, and provide a summary of the available resources for voice conversion research.
Original languageEnglish
Pages (from-to)132-157
Number of pages26
JournalIEEE/ACM Transactions on Audio, Speech and Language Processing
Volume29
Early online date17 Nov 2020
DOIs
Publication statusE-pub ahead of print - 17 Nov 2020

Keywords

  • vocoding
  • training data
  • speech analysis
  • deep learning
  • speech synthesis
  • pipelines
  • training
  • voice conversion
  • speaker characterization
  • voice conversion evaluation
  • voice conversion challenges

Fingerprint

Dive into the research topics of 'An overview of voice conversion and its challenges: From statistical modeling to deep learning'. Together they form a unique fingerprint.

Cite this