High-Quality Nonparallel Voice Conversion Based On Cycle-Consistent Adversarial Network

Fuming  Fang; Junichi Yamagishi; Isao Echizen; Jaime Lorenzo-Trueba

doi:10.1109/ICASSP.2018.8462342

High-Quality Nonparallel Voice Conversion Based On Cycle-Consistent Adversarial Network

Fuming Fang, Junichi Yamagishi, Isao Echizen, Jaime Lorenzo-Trueba

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Although voice conversion (VC) algorithms have achieved remarkable success along with the development of machine learning, superior performance is still difficult to achieve when using nonparallel data. In this paper, we propose using a cycle-consistent adversarial network (CycleGAN) for nonparallel data-based VC training. A CycleGAN is a generative adversarial network (GAN) originally developed for unpaired image-to-image translation. A subjective evaluation of inter-gender conversion demonstrated that the proposed method significantly outperformed a method based on the Merlin open source neural network speech synthesis system (a parallel VC system adapted for our setup) and a GAN-based parallel VC system. This is the first research to show that the performance of a nonparallel VC method can exceed that of state-of-the-art parallel VC methods.
Index Terms— Voice conversion, deep learning, cycle-consistent adversarial network, generative adversarial network

Original language	English
Title of host publication	2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Subtitle of host publication	Calgary, AB, Canada
Place of Publication	Calgary, Alberta, Canada
Publisher	Institute of Electrical and Electronics Engineers
Pages	5279-5283
Number of pages	5
ISBN (Electronic)	978-1-5386-4658-8
ISBN (Print)	978-1-5386-4659-5
DOIs	https://doi.org/10.1109/ICASSP.2018.8462342
Publication status	Published - 13 Sept 2018
Event	2018 IEEE International Conference on Acoustics, Speech and Signal Processing - Calgary, Canada Duration: 15 Apr 2018 → 20 Apr 2018 https://2018.ieeeicassp.org/ https://2018.ieeeicassp.org/default.asp

Publication series

Name
Publisher	IEEE
ISSN (Electronic)	2379-190X

Conference

Conference	2018 IEEE International Conference on Acoustics, Speech and Signal Processing
Abbreviated title	ICASSP 2018
Country/Territory	Canada
City	Calgary
Period	15/04/18 → 20/04/18
Internet address	https://2018.ieeeicassp.org/ https://2018.ieeeicassp.org/default.asp

Access to Document

10.1109/ICASSP.2018.8462342

HIGH-QUALITY NONPARALLEL VOICE CONVERSION BASED ON CYCLE-CONSISTENT ADVERSARIAL NETWORKAccepted author manuscript, 217 KB

Cite this

Fang, F., Yamagishi, J., Echizen, I., & Lorenzo-Trueba, J. (2018). High-Quality Nonparallel Voice Conversion Based On Cycle-Consistent Adversarial Network. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP): Calgary, AB, Canada (pp. 5279-5283). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICASSP.2018.8462342

@inproceedings{16904e6f458541e0a185ee3fd523c13c,

title = "High-Quality Nonparallel Voice Conversion Based On Cycle-Consistent Adversarial Network",

abstract = "Although voice conversion (VC) algorithms have achieved remarkable success along with the development of machine learning, superior performance is still difficult to achieve when using nonparallel data. In this paper, we propose using a cycle-consistent adversarial network (CycleGAN) for nonparallel data-based VC training. A CycleGAN is a generative adversarial network (GAN) originally developed for unpaired image-to-image translation. A subjective evaluation of inter-gender conversion demonstrated that the proposed method significantly outperformed a method based on the Merlin open source neural network speech synthesis system (a parallel VC system adapted for our setup) and a GAN-based parallel VC system. This is the first research to show that the performance of a nonparallel VC method can exceed that of state-of-the-art parallel VC methods. Index Terms— Voice conversion, deep learning, cycle-consistent adversarial network, generative adversarial network",

author = "Fuming Fang and Junichi Yamagishi and Isao Echizen and Jaime Lorenzo-Trueba",

year = "2018",

month = sep,

day = "13",

doi = "10.1109/ICASSP.2018.8462342",

language = "English",

isbn = "978-1-5386-4659-5",

publisher = "Institute of Electrical and Electronics Engineers",

pages = "5279--5283",

booktitle = "2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

address = "United States",

note = "2018 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2018 ; Conference date: 15-04-2018 Through 20-04-2018",

url = "https://2018.ieeeicassp.org/, https://2018.ieeeicassp.org/default.asp",

}

Fang, F, Yamagishi, J, Echizen, I & Lorenzo-Trueba, J 2018, High-Quality Nonparallel Voice Conversion Based On Cycle-Consistent Adversarial Network. in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP): Calgary, AB, Canada. Institute of Electrical and Electronics Engineers, Calgary, Alberta, Canada, pp. 5279-5283, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, Calgary, Canada, 15/04/18. https://doi.org/10.1109/ICASSP.2018.8462342

High-Quality Nonparallel Voice Conversion Based On Cycle-Consistent Adversarial Network. / Fang, Fuming ; Yamagishi, Junichi; Echizen, Isao et al.
2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP): Calgary, AB, Canada. Calgary, Alberta, Canada: Institute of Electrical and Electronics Engineers, 2018. p. 5279-5283.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - High-Quality Nonparallel Voice Conversion Based On Cycle-Consistent Adversarial Network

AU - Fang, Fuming

AU - Yamagishi, Junichi

AU - Echizen, Isao

AU - Lorenzo-Trueba, Jaime

PY - 2018/9/13

Y1 - 2018/9/13

N2 - Although voice conversion (VC) algorithms have achieved remarkable success along with the development of machine learning, superior performance is still difficult to achieve when using nonparallel data. In this paper, we propose using a cycle-consistent adversarial network (CycleGAN) for nonparallel data-based VC training. A CycleGAN is a generative adversarial network (GAN) originally developed for unpaired image-to-image translation. A subjective evaluation of inter-gender conversion demonstrated that the proposed method significantly outperformed a method based on the Merlin open source neural network speech synthesis system (a parallel VC system adapted for our setup) and a GAN-based parallel VC system. This is the first research to show that the performance of a nonparallel VC method can exceed that of state-of-the-art parallel VC methods. Index Terms— Voice conversion, deep learning, cycle-consistent adversarial network, generative adversarial network

AB - Although voice conversion (VC) algorithms have achieved remarkable success along with the development of machine learning, superior performance is still difficult to achieve when using nonparallel data. In this paper, we propose using a cycle-consistent adversarial network (CycleGAN) for nonparallel data-based VC training. A CycleGAN is a generative adversarial network (GAN) originally developed for unpaired image-to-image translation. A subjective evaluation of inter-gender conversion demonstrated that the proposed method significantly outperformed a method based on the Merlin open source neural network speech synthesis system (a parallel VC system adapted for our setup) and a GAN-based parallel VC system. This is the first research to show that the performance of a nonparallel VC method can exceed that of state-of-the-art parallel VC methods. Index Terms— Voice conversion, deep learning, cycle-consistent adversarial network, generative adversarial network

U2 - 10.1109/ICASSP.2018.8462342

DO - 10.1109/ICASSP.2018.8462342

M3 - Conference contribution

SN - 978-1-5386-4659-5

SP - 5279

EP - 5283

BT - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - Institute of Electrical and Electronics Engineers

CY - Calgary, Alberta, Canada

T2 - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing

Y2 - 15 April 2018 through 20 April 2018

ER -

Fang F, Yamagishi J, Echizen I, Lorenzo-Trueba J. High-Quality Nonparallel Voice Conversion Based On Cycle-Consistent Adversarial Network. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP): Calgary, AB, Canada. Calgary, Alberta, Canada: Institute of Electrical and Electronics Engineers. 2018. p. 5279-5283 doi: 10.1109/ICASSP.2018.8462342

High-Quality Nonparallel Voice Conversion Based On Cycle-Consistent Adversarial Network

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this